More

    Reader Submitted: An Assistive Communication Ecosystem Enabling Real-Time Deaf-Hearing Conversation

    Dari is an assistive communication ecosystem designed to bridge the communication gap between Deaf and hearing individuals through seamless, real-time, two-way interaction. Developed over a 26-week design process, the project was grounded in extensive user research, interviews, and co-design activities with members of the Deaf community to ensure that the solution addressed real challenges experienced in everyday life. The name Dari, meaning “bridge” in Korean, reflects the project’s core mission of fostering meaningful connections and creating a world where communication is more accessible, inclusive, and equitable for everyone.

    Many existing communication tools for Deaf individuals rely heavily on interpreters, smartphones, text-based exchanges, or one-way translation systems that can interrupt the natural flow of conversation. These solutions often create dependency, introduce delays, and fail to capture the emotional and contextual nuances that are essential to human interaction. Through research, key challenges were identified, including difficulties communicating in public environments, reliance on third parties for interpretation, and barriers to spontaneous social interactions. Dari was created to address these issues by enabling both Deaf and hearing individuals to communicate naturally using their preferred methods of communication.

    The ecosystem consists of three integrated components. The first is Woori, an AI-powered EMG smart armband that recognizes sign language gestures through muscle movement and translates them into spoken or written language. The second is Sari, a pair of smart glasses that captures spoken language and provides real-time captions and contextual information directly within the user’s field of view. The third component is a companion mobile application that supports device management, customization, communication history, and personalized settings. Together, these components create a communication experience that feels intuitive, unobtrusive, and human-centered.

    Beyond translating words, Dari aims to preserve the natural rhythm of conversation and reduce the social barriers that often exist between Deaf and hearing communities. By allowing both users to communicate in the way that is most comfortable for them, the system promotes independence, confidence, and equal participation in conversations. The solution is designed to support a wide range of environments, including workplaces, educational settings, healthcare facilities, public services, and everyday social interactions.

    Dari represents a vision of accessibility that goes beyond compliance and accommodation. It demonstrates how emerging technologies such as artificial intelligence, wearable computing, and human-centered design can be combined to create meaningful social impact. Rather than asking users to adapt to technology, Dari adapts technology to the needs of its users, making communication more natural and inclusive. Through its focus on empathy, accessibility, and human connection, the project seeks to empower individuals, strengthen communities, and contribute to a future where communication barriers no longer limit opportunities for participation, understanding, and belonging.

    Sejoon Kim(UX Designer, Team Lead), Suji Kim(UX Designer, UI Designer), Lukas Wienser(UX Engineer), Lara Kurt(UX Designer, Branding)


    image


    image


    image


    image


    image


    image


    image


    image


    image

    Dari is an assistive communication ecosystem designed to enable seamless, real-time communication between Deaf and hearing individuals. At its core, the project seeks to address a fundamental challenge that millions of Deaf and hard-of-hearing people face every day: the inability to communicate effortlessly with those who do not know sign language. While modern technology has transformed how people connect across distances, communication barriers still exist in many everyday situations, limiting independence, social participation, and equal access to opportunities. Dari was created to explore how emerging technologies can be used to foster more inclusive interactions and create a future where communication barriers are significantly reduced.

    The project began in 2024 with a simple question: Why does communication between Deaf and hearing individuals still require so much adaptation from one side of the conversation? In many situations, Deaf individuals are expected to rely on interpreters, written notes, smartphone applications, or family members to facilitate communication. These solutions can be effective in specific circumstances, but they often interrupt the natural flow of conversation, create dependency on third parties, and limit spontaneous interactions. More importantly, they can remove the emotional and social elements that make communication meaningful.

    The name Dari, which means “bridge” in Korean, reflects the central vision of the project. Rather than functioning as a tool that simply translates language, Dari aims to serve as a bridge between Deaf and hearing communities, enabling both groups to communicate naturally while maintaining their preferred methods of communication. The goal was not to force users to adapt to technology, but to design technology that adapts to users.

    The project was developed over a 26-week design process that combined user research, accessibility studies, emerging technology exploration, concept development, prototyping, and iterative testing. Throughout the project, members of the Deaf community played a critical role in shaping the direction of the solution. Their experiences, perspectives, and feedback informed design decisions at every stage, ensuring that the project remained grounded in real-world needs rather than assumptions.

    The research phase began with a broad exploration of Deaf culture, accessibility challenges, communication methods, and existing assistive technologies. This included reviewing academic literature, studying accessibility guidelines, analyzing existing products, and conducting interviews with Deaf individuals. While existing technologies have made significant progress, many solutions still present limitations. Smartphone-based translation applications require users to constantly shift their attention between a screen and the conversation. Human interpreters provide high-quality communication support but are not always available and can reduce personal privacy. Camera-based sign language recognition systems often depend on controlled environments, adequate lighting conditions, and specific positioning to function accurately.

    As interviews and research progressed, several recurring themes emerged. One major finding was the importance of independence. Many participants described situations where communication barriers forced them to rely on family members, friends, or interpreters to perform tasks that hearing individuals often take for granted. This dependency could affect confidence, privacy, and autonomy. Another insight involved the importance of spontaneity. Communication does not only occur during planned meetings or appointments. It happens during casual conversations, unexpected encounters, and social interactions. Many participants expressed frustration that existing communication solutions often required preparation or additional effort before a conversation could take place.

    The research also highlighted the complexity of communication itself. Language is only one part of how people communicate. Facial expressions, eye contact, body language, emotional tone, and contextual awareness all contribute to understanding. Several participants emphasized that communication barriers are not solely caused by the inability to translate words. They are also caused by the loss of these social and emotional cues during interactions. This insight became one of the most influential findings of the entire project and shaped the design direction moving forward.

    Based on the research findings, the project established three primary design goals. First, enable natural two-way communication between Deaf and hearing individuals. Second, reduce dependency on third-party assistance and external devices that interrupt conversation. Third, preserve the emotional and social aspects of communication that are often lost in existing solutions.

    These goals led to the development of the Dari ecosystem, which consists of three interconnected components: Woori, Sari, and a companion mobile application.

    Woori is an AI-powered EMG (electromyography) smart armband designed to recognize sign language through muscle activity in the forearm. Traditional sign language recognition systems frequently rely on cameras to track hand gestures. While effective in some cases, camera-based systems often face challenges related to lighting conditions, field of view, background clutter, and privacy concerns. Woori approaches the problem differently by capturing electrical signals generated by muscle movements. These signals are processed using machine learning algorithms that interpret signing patterns and translate them into spoken or written language in real time.

    The decision to explore EMG technology was driven by both technical and user-centered considerations. By eliminating the need for cameras, the system could function in a wider range of environments while providing a more discreet user experience. It also allowed users to communicate naturally through signing without worrying about whether their hands were visible to a camera system. The result is a solution that supports the user’s existing behavior rather than requiring new communication habits.

    The second component, Sari, is a pair of smart glasses designed to support Deaf users during conversations with hearing individuals. Sari captures spoken language and provides real-time captions directly within the user’s field of view. Rather than requiring users to constantly look down at a smartphone screen, the glasses allow them to maintain eye contact and remain visually engaged with the person they are speaking with. This seemingly small change has a significant impact on the quality of interaction because it helps preserve social cues and conversational flow.

    Beyond transcription, Sari was designed to provide contextual awareness that contributes to more natural communication. During research, participants frequently discussed how traditional captioning solutions often felt disconnected from the social dynamics of conversation. By integrating information directly into the user’s field of view, Sari helps reduce cognitive load and supports a more immersive communication experience.

    The third component is a companion mobile application, which serves as the central hub of the ecosystem. The application manages device connectivity, personalization settings, communication history, and system updates. It also provides customization options that allow users to tailor the experience to their preferences and communication needs. Rather than functioning as the primary communication interface, the mobile application supports the wearable devices and ensures a cohesive user experience across the ecosystem.

    The design process involved extensive ideation and concept development. Hundreds of sketches, interaction flows, and system diagrams were created to explore different approaches to communication support. Early concepts ranged from smartphone-only applications to wearable translation devices and environmental communication systems. Each concept was evaluated against the project’s core design goals, and feedback from potential users helped identify strengths and weaknesses.

    User journey mapping played a major role throughout development. Scenarios were created to examine how communication barriers affect different aspects of daily life, including healthcare appointments, workplace meetings, classroom participation, public transportation, retail experiences, and social gatherings. These journeys revealed moments where communication breakdowns commonly occur and helped identify opportunities where technology could provide meaningful support.

    Prototyping was conducted throughout the project to test both hardware and software concepts. Low-fidelity sketches and wireframes were initially used to explore interaction patterns and information architecture. As the project progressed, higher-fidelity prototypes were developed to visualize the ecosystem and evaluate usability. Feedback from reviews and testing sessions continuously informed refinements, helping ensure that the solution remained aligned with user needs.

    The accessibility should not be viewed solely as a functional requirement. Accessibility is fundamentally about creating opportunities for participation, inclusion, and independence. Many existing assistive technologies are designed to help users overcome barriers, but they often unintentionally emphasize differences between users. Dari instead seeks to create shared experiences where communication feels natural and equal for everyone involved.

    The project also encouraged deeper reflection on the broader societal impact of communication barriers. Communication influences nearly every aspect of life, including education, employment, healthcare, relationships, and community engagement. When communication barriers exist, they can create isolation and limit opportunities for personal and professional growth. Addressing these barriers therefore has implications that extend far beyond individual conversations.

    Dari demonstrates how emerging technologies such as artificial intelligence, wearable computing, machine learning, and human-centered design can be combined to create meaningful social impact. Rather than focusing solely on technological innovation, the project prioritizes human experiences and real-world needs. Every design decision was guided by the belief that technology should empower individuals rather than requiring them to adapt to technological constraints.

    Ultimately, Dari is more than a communication tool. It is a vision for a more inclusive future where communication is accessible to everyone regardless of hearing ability. By enabling natural interactions, reducing dependency, and preserving the social and emotional dimensions of conversation, Dari seeks to strengthen connections between Deaf and hearing communities. The project demonstrates that thoughtful design can transform emerging technologies into solutions that not only solve problems but also promote dignity, understanding, and belonging. Through its focus on accessibility, empathy, and human connection, Dari represents a step toward a future where communication barriers no longer prevent people from fully participating in the world around them.

    Learn More About This Projectx

    • o

      Favorite This

     

    Latest articles

    spot_imgspot_img

    Related articles