Description:FIELD OF THE INVENTION
[001] The present invention relates to the field of Internet of Thing (IoT), Machine Learning and more particularly, the present invention relates to the blink-based home access system for disabled individuals.

BACKGROUND FOR THE INVENTION:
[002] The following discussion of the background to the invention is intended to facilitate an understanding of the present invention. However, it should be appreciated that the discussion is not an acknowledgment or admission that any of the material referred to was published, known, or part of the common general knowledge in any jurisdiction as of the priority date of the application. The details provided herein the background if belongs to any publication is taken only as a reference for describing the problems, in general terminologies or principles or both of science and technology in the associated prior art.
[003] The inspiration for this design came from extensive interactions and feedback from individuals living with paralysis, caregivers, and healthcare professionals who expressed a strong desire for accessible, secure, and user-friendly solutions to empower disabled individuals in their daily lives. This system can not only be used to save electricity, but also to help paralysed patients to lead their own life without anyone else’s assistance. Paralysis is a condition where people are paralysed below their neck. Therefore, there is a growing need for innovative solutions that cater to the specific needs of disabled individuals and empower them to independently control their environments. One promising approach to address this challenge is through the integration of Eye Blink-Based Authentication System (EBAS) with smart home devices. EBAS leverages the unique characteristics of an individual's eye movements, particularly blinking patterns, to authenticate their identity and grant access to smart home controls. By using convolutional neural network (CNN) technology, EBAS can accurately analyze and interpret eye blink data, ensuring reliable and secure authentication for users with disabilities.
[004] Applicant has conducted a worldwide prior art search for elucidating prior arts. Following are some of the cited prior art documents by the Applicant:
[005] Patent Application No. [EP3140780B1]: "Method and System for Eye Blink-Based Authentication in Smart Home Environments" - This patent application describes a method and system for authenticating users based on eye blink patterns, similar to the present invention. However, it focuses on general smart home authentication rather than specifically catering to disabled individuals.
[006] Patent Application No. US20150002676A1]: "Smart Glasses with Integrated Eye Tracking and Authentication System" - This patent application proposes smart glasses equipped with eye tracking technology for authentication purposes. While it shares similarities with the present invention in terms of eye tracking, it does not specifically address the needs of disabled individuals or integrate CNN technology for authentication.
[007] Patent Application No. [US 2016/0259308A1]: "Enhanced Accessibility Features for Smart Home Devices" - This patent application discusses various accessibility features for smart home devices, including voice recognition and gesture control. While it acknowledges the importance of accessibility, it does not specifically explore eye blink-based authentication or CNN technology.
[008] Limitation of related invention are giving below over some prior patents:
[009] [EP3140780B1]: The method and system described in this application lack specificity for disabled individuals, potentially overlooking their unique needs and challenges (EP3140780B1).
[010] [US20150002676A1]: While smart glasses with eye tracking technology offer potential benefits for authentication, they may not provide sufficient accessibility options for individuals with disabilities beyond eye tracking (US20150002676A1).
[011] [US 2016/0259308A1]: Although this application discusses enhanced accessibility features, it does not address the specific challenges faced by disabled individuals in accessing and controlling smart home devices using eye blink patterns (US 2016/0259308A1).
[012] [EP3140780B1]: The generic focus on smart home authentication may result in a lack of tailored solutions for disabled users, potentially limiting their ability to interact effectively with smart home environments (EP3140780B1).
[013] [US20150002676A1]: The integration of eye tracking technology in smart glasses may pose usability challenges for individuals with certain disabilities, such as those with limited eye mobility or dexterity issues (US20150002676A1).
[014] In light of the foregoing, there is a need for the blink-based home access system for disabled individuals that overcomes problems prevalent in the prior art.

OBJECTS OF THE INVENTION:
[015] Some of the objects of the present disclosure, which at least one embodiment herein satisfies, are as follows.
[016] The principal object of the present invention is to overcome the disadvantages of the prior art by providing the Blink-based home access system for disabled individuals.
[017] An object of the present invention is to provide the Blink-based home access system for disabled individuals that utilize convolutional neural networks (CNNs) for eye authentication, resulting in higher accuracy compared to traditional methods.
[018] An object of the present invention is to provide the Blink-based home access system for disabled individuals, wherein that enables individuals with disabilities to control home appliances using eye blinks, enhancing their accessibility and independence.
[019] An object of the present invention is to provide the Blink-based home access system for disabled individuals, wherein that allows for real-time response to user commands, ensuring swift and efficient control of smart home devices.
[020] An object of the present invention is to provide the Blink-based home access system for disabled individuals, wherein users are allowed to customize settings based on their preferences, such as adjusting the sensitivity of eye blink detection or assigning specific actions to different blinking patterns.
[021] An object of the present invention is to provide the blink-based home access system for disabled individuals which integrate with existing smart home systems, requiring minimal modifications for implementation.
[022] An object of the present invention is to provide the blink-based home access system for disabled individuals, wherein the modular design of the system facilitates scalability, enabling easy expansion to accommodate additional smart home devices or user functionalities.
[023] Other objects and advantages of the present disclosure will be more apparent from the following description, which is not intended to limit the scope of the present disclosure.

SUMMARY OF THE INVENTION:
[024] The present invention provides Blink-based home access system for disabled individuals. The system aims to enhance accessibility for individuals, particularly those with disabilities, by providing them with a convenient and intuitive means of controlling household devices using eye blinks. The system offers an intuitive interaction method, allowing users to control lights and fans effortlessly through simple eye movements, eliminating the need for traditional physical controls. The system seamlessly integrates advanced technologies such as face detection, facial landmark extraction, and Long-term Recurrent Convolutional Networks (LRCN) with IoT platforms, ensuring accurate detection and interpretation of intentional eye blinks. By enabling individuals with disabilities to independently manage their environment, the system contributes to improving their overall quality of life and fostering greater autonomy and independence. The system undergoes rigorous testing to ensure reliable performance, accurate detection of eye blinks, and seamless control of household devices, providing users with a dependable and effective solution.
[025] Eye Region Extraction: A rectangular region surrounding the eye landmarks is extracted from the aligned face areas to create a new sequence of input frames.
[026] LRCN Implementation: The cropped eye area sequences are input into Long-term Recurrent Convolutional Networks (LRCN) for dynamic state prediction, utilizing the VGG16 framework for CNN-based implementation.
[027] Eye Blink Detection: Patient intentional eye blinks are detected using an IR-based eye blink sensor.
[028] Blink Duration Measurement: The duration of eye blinks is measured using ESP32-S3-EYE development board using IoT protocols such as MQTT or CoAP.
[029] Blink Counting: If the blink duration falls within a specified range, the count value is incremented.
[030] Application Execution: Depending on the count value, different applications are executed, such as turning on lights or sending SMS alerts.
BRIEF DESCRIPTION OF DRAWINGS:
[031] Reference will be made to embodiments of the invention, examples of which may be illustrated in accompanying figures. These figures are intended to be illustrative, not limiting. Although the invention is generally described in the context of these embodiments, it should be understood that it is not intended to limit the scope of the invention to these particular embodiments.
[032] Fig.1 illustrates a schematic diagram of an IoT Based Irrigation with Security System and method thereof, in accordance with an embodiment of the present invention.
[033] Fig. 2 Working incarnation Eye Detection
[034] Fig. 3 After Eye authentication IOT senses the frequency of Eye Blink to Control home appliance.
[035] Fig. 4 IOT sense the frequency of Eye Blink to Control home appliance workflow
DETAILED DESCRIPTION OF DRAWINGS:
[036] While the present invention is described herein by way of example using embodiments and illustrative drawings, those skilled in the art will recognize that the invention is not limited to the embodiments of drawing or drawings described and are not intended to represent the scale of the various components. Further, some components that may form a part of the invention may not be illustrated in certain figures, for ease of illustration, and such omissions do not limit the embodiments outlined in any way. It should be understood that the drawings and the detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the scope of the present invention as defined by the appended claim.
[037] As used throughout this description, the word "may" is used in a permissive sense (i.e. meaning having the potential to), rather than the mandatory sense, (i.e. meaning must). Further, the words "a" or "an" mean "at least one” and the word “plurality” means “one or more” unless otherwise mentioned. Furthermore, the terminology and phraseology used herein are solely used for descriptive purposes and should not be construed as limiting in scope. Language such as "including," "comprising," "having," "containing," or "involving," and variations thereof, is intended to be broad and encompass the subject matter listed thereafter, equivalents, and additional subject matter not recited, and is not intended to exclude other additives, components, integers, or steps. Likewise, the term "comprising" is considered synonymous with the terms "including" or "containing" for applicable legal purposes. Any discussion of documents, acts, materials, devices, articles, and the like are included in the specification solely for the purpose of providing a context for the present invention. It is not suggested or represented that any or all these matters form part of the prior art base or were common general knowledge in the field relevant to the present invention.
[038] In this disclosure, whenever a composition or an element or a group of elements is preceded with the transitional phrase “comprising”, it is understood that we also contemplate the same composition, element, or group of elements with transitional phrases “consisting of”, “consisting”, “selected from the group of consisting of, “including”, or “is” preceding the recitation of the composition, element or group of elements and vice versa.
[039] The present invention is described hereinafter by various embodiments with reference to the accompanying drawing, wherein reference numerals used in the accompanying drawing correspond to the like elements throughout the description. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiment set forth herein. Rather, the embodiment is provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those skilled in the art. In the following detailed description, numeric values and ranges are provided for various aspects of the implementations described. These values and ranges are to be treated as examples only and are not intended to limit the scope of the claims. In addition, several materials are identified as suitable for various facets of the implementations. These materials are to be treated as exemplary and are not intended to limit the scope of the invention.
[040] The present invention provides a blink-based home access system for disabled individuals that presents a innovative approach to enhance accessibility in smart home environments for disabled individuals using an Eye Blink-Based Authentication System (EBAS) integrated with Convolutional Neural Network (CNN) technology. Through the utilization of eye-gaze tracking via specialized glasses, the system employs CNN for robust eye authentication. Once authenticated, users can effortlessly control home appliances such as lights, fans, and alarms through simple eye blinks, facilitating seamless interaction and improving overall accessibility for individuals with disabilities.
[041] By using convolutional neural network (CNN) technology, EBAS can accurately analyze and interpret eye blink data, ensuring reliable and secure authentication for users with disabilities.
[042] CNNs are a class of deep learning algorithms that have shown remarkable success in image recognition tasks. They are particularly well-suited for analyzing complex visual data, making them ideal for interpreting eye movement patterns captured by cameras or other sensors. In the context of EBAS, CNNs play a crucial role in processing raw eye blink data and extracting meaningful features that can be used for authentication.
[043] The design begins with the collection of eye blink data from the user, typically through a camera or sensor integrated into a pair of smart glasses or other wearable device. The raw data is then preprocessed to remove noise and artifacts, ensuring that only relevant information is passed to the CNN for analysis.
[044] The CNN architecture consists of multiple layers, including convolutional layers, pooling layers, and fully connected layers. These layers work together to extract hierarchical features from the input data, gradually transforming it into a representation that is suitable for classification.
[045] During training, the CNN learns to distinguish between genuine eye blink patterns associated with the user and false positives caused by involuntary movements or environmental factors. This is achieved through a process known as backpropagation, where the network adjusts its internal parameters based on the discrepancy between predicted and actual outputs.
[046] Once trained, the CNN can be deployed in real-time to authenticate users based on their eye blink patterns. When a user attempts to access smart home controls, such as lighting, heating, or security systems, the EBAS system captures their eye movements and feeds them into the CNN for analysis. If the pattern matches the user's unique profile, access is granted, allowing them to control the desired devices.
[047] By integrating EBAS with smart home devices, individuals with disabilities can enjoy greater independence and autonomy in their daily lives. They no longer have to rely on cumbersome physical interfaces or assistance from caregivers to interact with their surroundings. Instead, they can simply blink their eyes to access and control a wide range of devices, enhancing their quality of life and promoting inclusivity in smart home environments.
1. The first step in our method is to locate the face areas in each frame of the video using a face detector. Then facial landmarks, which are locations on the face carrying important structural information such as tip of the eyes, noses and mouths and contours of the cheek, are extracted from each detected face area.
2. We align the face regions to a unified coordinate space using landmark based face alignment algorithms.
3. From the aligned face areas, we can extract a surrounding rectangular region of the landmarks corresponding to the contours of the eyes into a new sequence of input frames.
4. The cropped eye area sequences are passed into Long-term Recurrent Convolutional Networks (LRCN) for dynamic state prediction. For implementation we will use Convolutional Neural Network (CNN) based on the VGG16 framework.
5. Patient will provide the intentional eye blink to operate the system by IR based eye blink sensor.
6. Blink Detection using MobileNet:
7. Preprocess eye images captured by the spectacles to enhance features and remove noise.
8. Implement MobileNet architecture on the edge device for efficient blink detection.
9. Fine-tune the MobileNet model if necessary to optimize performance for blink counting.
10. Utilize techniques like sliding window approach or temporal differencing to detect eye blinks from consecutive frames.
11. Integrate blink detection with the eye authentication system to ensure that only authenticated blinks are considered for controlling the lights.
12. IoT Integration for Light Control:
13. Establish communication between the edge device and the ESP32-S3-EYE development board using IoT protocols such as MQTT or CoAP.
14. Develop firmware for the ESP32-S3-EYE to receive blink count data from the edge device and control the light switches accordingly.
15. Implement logic to interpret blink frequency and toggle the lights on or off based on predefined thresholds, considering factors like user preferences and ambient lighting conditions.
16. Ensure reliability and fault tolerance in the IoT communication protocol to handle potential network disruptions or packet losses.
17. User Interface and Accessibility:
18. Design a user-friendly interface for configuring the system settings and monitoring blink counts, leveraging graphical user interface (GUI) libraries or web-based interfaces.
19. Incorporate accessibility features such as voice commands, tactile feedback, or customizable gesture recognition to accommodate users with different abilities.
20. Provide clear instructions and troubleshooting guidance for setup and operation, considering the diverse needs and preferences of the target user group.
21. Details of eye blink for controlling the device:
22. Total number of blinks and Applications Executed
a) Electrical Bulb will glow on
b) The SMS will be sent to the assistant mobile
c) It will play pre-recorded audio messages
d) Alarm ring for calling a person
[048] Based on the intentional blinks, the count value increases and the corresponding application gets executed. If the count value is one, then the home appliance (bulb) will glow and if we repeat the same the bulb gets off. If the count value is two, then the SMS will be sent to the assistant mobile. If the count value is three, the prerecorded audio messages will be sent to the mobile for help.
[049] The design will Improved Authentication Accuracy: The present invention utilizes convolutional neural networks (CNNs) for eye authentication, resulting in higher accuracy compared to traditional methods.
[050] Enhanced Accessibility: By integrating IoT technology, the invention enables individuals with disabilities to control home appliances using eye blinks, enhancing their accessibility and independence.
[051] Real-time Response: The system's integration with IoT allows for real-time response to user commands, ensuring swift and efficient control of smart home devices.
[052] Customizable Settings: Users can customize settings based on their preferences, such as adjusting the sensitivity of eye blink detection or assigning specific actions to different blinking patterns.
[053] Seamless Integration: The invention seamlessly integrates with existing smart home systems, requiring minimal modifications for implementation.
[054] Scalability: The modular design of the system facilitates scalability, enabling easy expansion to accommodate additional smart home devices or user functionalities.
[055] To Control Lights and Fans Using Eye Blinks through IoT:
- Install an IoT-enabled camera to capture eye movements.
- Develop software algorithms, including threshold-based, machine learning-based, feature-based, template matching, and hybrid algorithms, to detect and interpret eye blinks as commands.
- Connect the software to an IoT platform capable of controlling lights and fans remotely.
- Configure the IoT platform to trigger actions based on detected eye blinks, such as turning lights on/off or adjusting fan speed.
- Test the system to ensure accurate detection and reliable control of lights and fans based on eye blinks.
[056] The proposed "Eye Blink-Based Password Authentication System for Controlling Smart Home Accessibility for Disabled Individuals" distinguishes itself from existing products in several ways:
[057] Non-Verbal and Non-Physical Control: Unlike voice and gesture control systems that rely on speech or physical movements, the proposed system offers non-verbal and non-physical control through eye blinks. This makes it accessible to individuals with various disabilities, including those with limited mobility or speech impairments.
[058] Enhanced Security: Biometric authentication based on eye blink patterns provides a high level of security. It ensures that only authorized users can access and control smart home devices, addressing concerns about unauthorized access.
[059] Customization for Individual Needs: The system can be tailored to the unique capabilities and preferences of each user, allowing them to define their own blink patterns for specific commands or devices.
[060] Independence and Privacy: By offering independent control, the proposed system enhances users' privacy and reduces their dependence on caregivers or family members for routine tasks.
[061] Accessibility-First Design: The project places a strong emphasis on accessibility and inclusion, aiming to create a solution that can be used by a diverse range of individuals, including those with disabilities.
[062] IoT Integration: The integration of this technology into existing IoT and smart home ecosystems allows users to seamlessly control a wide range of devices and appliances.
[063] The disclosure has been described with reference to the accompanying embodiments herein and the various features and advantageous details thereof are explained with reference to the non-limiting embodiments in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein.
[064] The foregoing description of the specific embodiments so fully revealed the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the scope of the embodiments as described herein.
, Claims:1) A blink-based home access system for disabled individuals, the system comprises:
- a spectacles for blink detection comprising: a camera for capturing user's eyes and an edge device;
- a processor for processing predefined instructions that authenticate the user's eyes using convolutional neural networks (CNNs) and utilize MobileNet implemented;
- a ESP32-S3-EYE development board as an IoT interface for controlling the lights based on blink frequency.
2) The system as claimed in claim 1, wherein the MobileNet is implemented on an edge device attached to spectacles.
3) The system as claimed in claim 1, wherein the Blink Detection comprises steps of:
- preprocessing eye images capturesd by the spectacles to enhance features and remove noise;
- implementing MobileNet architecture on the edge device for efficient blink detection;
- fine-tuning the MobileNet model if necessary to optimize performance for blink counting;
- utilizing techniques like sliding window approach or temporal differencing to detect eye blinks from consecutive frames; and
- integrating blink detection with the eye authentication system to ensure that only authenticated blinks are considered for controlling the lights.
4) The system as claimed in claim 1, wherein the patient is provided with an intentional eye blink to operate the system by IR based eye blink sensor.
5) The system as claimed in claim 1, wherein the communication between the edge device and the ESP32-S3-EYE development board is established using IoT protocols such as MQTT or CoAP.
6) A method for blink-based home access for disabled individuals; the method comprises steps of:
- Eye Region Extraction; wherein a rectangular region surrounding the eye landmarks is extracted from the aligned face areas to create a new sequence of input frames;
- LRCN Implementation; wherein the cropped eye area sequences are input into Long-term Recurrent Convolutional Networks (LRCN) for dynamic state prediction, utilizing the VGG16 framework for CNN-based implementation.
- Eye Blink Detection; wherein Patient intentional eye blinks are detected using an IR-based eye blink sensor;
- Blink Duration Measurement; wherein the duration of eye blinks is measured using ESP32-S3-EYE development board using IoT protocols such as MQTT or CoAP.
- Blink Counting; wherein if the blink duration falls within a specified range, the count value is incremented;
- Application Execution; wherein depending on the count value, different predefined applications are executed.