see the sound

What it does

A system combining hardware and software can precisely identify information gaps for hearing-impaired children. It captures sign language and sound via machines for analysis, converting them into visual text to ensure smooth information transmission

Your inspiration

In the process of product design, I turned my attention to the vulnerable group of hearing-impaired children. After learning about the basic situation of school-age hearing-impaired children in special education through literature and online materials, I conducted in-depth interviews with my friend who is a special education teacher. Through in-depth communication with him and an investigation into the educational status of hearing-impaired children, I deeply realized that many problems urgently need to be solved, especially the severe lack of auxiliary devices during the education of hearing-impaired children.

How it works

In response to the information reception barriers faced by hearing-impaired children during the educational process, a smart wearable AR glasses has been designed. Leveraging multi-modal information collection and intelligent interaction technologies, it constructs a comprehensive information perception system for them. The device features a split modular design. The frame is equipped with a dual-axis联动 (synergistic) sign language recognition camera set, including a front wide-angle lens and a rear blind-spot compensation lens, enabling 360° sign language motion capture. It is also integrated with a high-sensitivity ambient sound array microphone, supporting full-frequency sound signal collection. The system can instantly perform sign language gesture analysis and speech semantic recognition. Processed information is presented on the display of the glasses in the form of AR (augmented reality), transmitted through visual means.

Design process

After defining the core functional architecture and technical concepts of the smart AR glasses, I systematically iterated and optimized the design to achieve a dual upgrade in product form and user experience: 1. Lightweight Appearance Reconstruction The initial design suffered from a bulky modular integration, resulting in an overly blocky appearance that was ill-suited to the daily use scenarios of hearing-impaired children. Through multiple rounds of ergonomic verification, the thickness of components was reduced. Streamlined curved transitions were employed to eliminate redundant edges, creating a lightweight appearance that balances a sense of technology with approachability. 2. Dynamic Adaptive Structure Innovation In response to the dynamic changes in head circumference and interpupillary distance during the growth and development of hearing-impaired children, the traditional ear-hook fixing structure was abandoned in favor of an integrated head-mounted design. The elastic silicone material allows for adjustable fitting, while the one-piece curved lens design ensures comfortable wear and visual stability for users of different ages.

How it is different

At the core of our product lies breakthrough AR (Augmented Reality) technology, which is deeply focused on the real pain points of educational scenarios for hearing - impaired children, offering innovative solutions to information reception barriers. The built - in high - precision eye - tracking analyzer, like a perceptive "digital nerve," can instantly capture the gaze trajectories and fixation durations of hearing - impaired children, intelligently analyzing the areas where they are unaware of missing information. The system accurately identifies blind spots for crucial information and proactively delivers important content into the children's field of vision, enabling intelligent supplementation and efficient transmission of information. This "active information service" model not only breaks the limitations of traditional passive information reception but also ensures that hearing - impaired children do not miss any key knowledge during the learning proces

Future plans

The smart AR glasses designed for the education of hearing - impaired children are currently in the conceptual prototype stage. The core technical architecture has been designed, and preliminary functional verification has been completed. By integrating AR (Augmented Reality) and eye - tracking analysis technologies, the product demonstrates innovative potential in addressing the information reception challenges faced by hearing - impaired children. Although it is still some way from large - scale commercialization, a solid foundation has been laid for subsequent research and development.

Awards

The Silver Award of the Chinese Division of the 4th FA International Frontier Innovative Art and Design Competition in 20252025