Description:The Smart Wearable Intercom device designed for deaf and mute individuals is illustrated
through two key diagrams. Fig. 1 provides an overview of the system’s functional architecture.
The system integrates various input devices, including a microphone for capturing voice input
and a camera for detecting gestures and sign language. An analog-to-digital converter processes
these inputs for further interpretation. The central microcontroller acts as the processing unit,
handling speech recognition, gesture detection, and text conversion, ensuring efficient
management of user inputs. The processed data is then relayed to output modules, including an
LCD on the smartwatch for visual feedback, a speaker for audio output, and a vibration motor
for haptic notifications. Connectivity between the components is achieved through Bluetooth,
which enables seamless communication between the microcontroller, a smartphone, and the
smartwatch. Additionally, a dedicated mobile application facilitates user interaction and data
visualization. This integrated setup allows for real-time conversion of speech, gestures, and lip
movements into accessible feedback for effective communication.
Fig. 2 depicts the workflow of the system, detailing its operational process. The system begins
by capturing inputs such as voice, gestures, and lip movements. Speech recognition, camerabased gesture detection, and deep learning algorithms for lip reading are employed to ensure
accurate input interpretation. These inputs are then processed through a translation module that
converts them into text, facilitating seamless communication. Based on the processed data, the
system determines the most appropriate output format, which can include text-to-speech
feedback, gesture translations, or lip-reading outputs. The final output is delivered through text
displays on the smartwatch, audio outputs via the speaker, or vibratory feedback for critical
alerts. A feedback loop ensures that the system continuously listens for new inputs, enabling
uninterrupted communication. This comprehensive process highlights how the system effectively
transforms multimodal inputs into actionable outputs, empowering users with hearing or speech
impairments to communicate intuitively and efficiently. , Claims:I/We claim:
1. A Smart Wearable Intercom device System for facilitating communication for deaf and mute
individuals, comprising:
a speech recognition system;
a gesture detection system;
a lip-reading detection system;
a microcontroller for processing data from the speech recognition, gesture detection, and
lip-reading systems;
a smartwatch interface for displaying text and providing feedback;
a Bluetooth communication module for data exchange;
a mobile application for configuration and remote monitoring of the system.
2. The speech recognition system as claimed in claim 1, wherein the system converts spoken
language into text using a microphone and sends the text to the microcontroller for processing.
3. The gesture detection system as claimed in claim 1, wherein the system captures visual data from
a camera and recognizes sign language gestures, converting them into text for display on the
smartwatch.
4. The lip-reading detection system as claimed in claim 1, wherein the system uses deep learning
algorithms to analyse lip movements and translate them into text for further processing by the
microcontroller.
5. The microcontroller as claimed in claim 1, wherein the microcontroller processes the data from
the speech recognition, gesture detection, and lip-reading systems, determines the type of input,
and outputs the translated text to the smartwatch display or provides audio feedback through a
speaker.
6. The smartwatch interface as claimed in claim 1, wherein the smartwatch displays translated text
in real-time and provides haptic feedback using a vibrating motor to alert the user to important
information or messages.
7. The Bluetooth communication module as claimed in claim 1, wherein the Bluetooth module
enables wireless communication between the speech recognition system, gesture detection
system, lip-reading detection system, microcontroller, and the mobile application, facilitating
seamless data transfer and synchronization.
8. The mobile application as claimed in claim 1, wherein the mobile application allows users to
configure system settings, monitor performance, and manage communication preferences,
providing remote access to the Smart Wearable Intercom device for enhanced usability and
control.
9. The feedback system as claimed in claim 1, wherein the system outputs feedback through
multiple channels, including text-to-speech, haptic vibrations, or visual text display, to ensure
effective communication based on the user's needs.
10. The real-time communication system as claimed in claim 1, wherein the system continuously
listens for inputs, translates speech or gestures into text, and outputs the feedback in real-time,
facilitating an uninterrupted and smooth communication experience for the user