Claims:1. A vehicle driver drowsiness warning system (100) comprising:

an in-vehicle driver identification system (210) configured to recognize the driver based on biometrics and load profile of a recognised driver;

an in-vehicle eyelid closing pattern recognition system (130) configured to capture a first set of facial data corresponding to the driver at a first time instant, and second set of facial data corresponding to the driver at a second time instant, and identify first eyelid closing pattern of the driver based on the first set of facial data and second eyelid closing pattern of the driver based on the second set of facial data,
wherein the second set of facial data being captured at a predetermined sampling interval continuously;

a central data processing system (110) configured to
detect drowsiness of the driver based on comparison of the first eyelid closing pattern of the driver with the second eyelid closing pattern of the driver pattern of the driver,
wherein the first eyelid closing pattern of the driver act as a reference point,
wherein the drowsiness is detected when degree of variation of second eyelid closing pattern is more than 10 degrees and time of eyelid closing is more than one second as compared to the first eyelid closing pattern,
capture geolocation of the vehicle and corresponding time stamp upon detection of the drowsiness of the driver;
an on-board memory system being configured to store profiling data, wherein the profiling data comprises the first eyelid closing pattern of the vehicle driver, detected drowsiness of the driver, captured one or more geolocations and corresponding timestamps with respect to one or more recognized drivers, route and terrain covered by the vehicle in an instant; duration of driving over the driving path, at least one finger print associated with the one or more drivers, health of the vehicle, and lists of emergency contact, and

a profile generation module configured to generate profile of the vehicle driver based on the profiling data.

2. The vehicle driver drowsiness warning system as claimed in claim 1, further comprising a centralized server (120) configured to store the first set of facial data corresponding to the driver, the second set of facial data corresponding to the driver, the first eyelid closing pattern of the driver, the second eyelid closing pattern of the driver continuously captured at the predetermined sampling interval, the profiling data.

3. The vehicle driver drowsiness warning system as claimed in claims 1 and 2, wherein the central data processing system (110) and the centralized server (120) independently generate one or more alerts upon detection of drowsiness by the central data processing system (110) by at least one of an audio means or visual means to the vehicle driver, and send the one or more alerts to a number of pre-fed contact list on the geo-location of the vehicle, alertness level of the driver and physical condition of the said vehicle.

4. The vehicle driver drowsiness warning system (100) as claimed in claim 1, wherein said in-vehicle eyelid closing pattern recognition system (130) comprises a camera (220).

5. The vehicle driver drowsiness warning system (100) as claimed in claim 1, wherein said driver identification system (210) for recognizing the driver comprises at least one of a fingerprint scanner and a camera for facial recognition.

6. The vehicle driver drowsiness warning system (100) as claimed in claim 5, wherein said driver identification system (210) for recognizing the driver is accompanied by proximity sensor for determining and ascertaining recognized driver for the vehicle.

7. The vehicle driver drowsiness warning system (100) as claimed in claim 1, wherein said profile generation module alerts the vehicle driver about nearest location of rest basing on the usual route taken on detection of the drowsiness.

8. The vehicle driver drowsiness warning system (100) as claimed in claim 1, wherein the rate of sampling of eye lid closing pattern is from one to thirty seconds.

9. A method of vehicle driver drowsiness warning system (100) comprising:
acquiring in-vehicle driver identification for each authorized driver by an in-vehicle driver identification system (210) based on biometrics;
loading profile of an identified driver;
extracting the relevant in-vehicle eyelid closing pattern of said identified driver from an on-board memory system based on a first set of facial data corresponding to the driver at a first time instant, and second set of facial data corresponding to the driver at a second time instant, wherein the first set of facial data and the second set of facial data being captured via a camera (220),
wherein a centralized server (120) storing the first set of facial data corresponding to the driver and the second set of facial data corresponding to the driver, the first eyelid closing pattern of the driver, the second eyelid closing pattern of the driver is continuously captured at the predetermined sampling interval;
comparing said eyelid closing data from said on-board memory system to the eyelid closing pattern of the identified driver;
establishing a relationship on drowsiness of the driver by continuous sampling of said eyelid closing pattern of the identified driver;
enabling an onboard alarm to the driver informing on the identification of drowsiness; and
sending alerts to a number of pre-fed contact list on the geo-location of the vehicle, alertness level of the driver and physical condition of the said vehicle.

10. The method of vehicle driver drowsiness warning system (100) as claimed in claim 9, further comprises capturing the geolocation of the vehicle, health of the vehicle, route and terrain of the vehicle, lists the point of emergency contact and continuously updates them through machine learning upon the identification of drowsiness.
Dated this 22nd day of May 2020
Signature

Vidya Bhaskar Singh Nandiyal
Patent Agent (IN/PA-2912)
Agent for the Applicant
, Description:FIELD OF THE INVENTION
The present invention relates to a vehicle driver drowsiness warning system. More particularly, the present invention relates to an inbuilt vehicle smart system that detects and alarms the vehicle rider as well as a central server on the detection of drowsiness by the rider while driving the vehicle.

BACKGROUND
There have been a number of developments in detecting and monitoring the alertness of a driver so that falling into sleep could be avoided. Such methods include monitoring body signals and observing a continued physical reaction. To monitor various body signals and to observe physical reactions involve placing sensors at various places on the vehicle as well as on the body of the driver. This could be very troublesome for the driver owing to the inconvenience of the location and wirings associated with the sensors. Additionally, such techniques are quite inaccurate owing to the indirect judgment of the body behavior of the driver and associated leaning pattern of the software. Another technique is to observe a pattern of blinking of the driver to differentiate between active and dosing off. However, such techniques have the following factors associated with them: a. The monitoring system must be accessible and portable. b. The system must be reliable. c. The driving conditions must not be hindered with and, d. The system performance should be easily replicated.
For instance, US 9783162 illustrates a system and method for facilitating user access to vehicles based on biometric information. However, the system doesn’t mention about onboard training module for constantly updating and training the close loop processing unit in a single unit. Additionally, US 5689241 illustrates sleep detection and driver alert apparatus. However, this system doesn’t include a fingerprint scanner or any other proximity sensor. Additionally, any close loop training and processing module is absent in such prior art.
Therefore, to obviate the prior art, the system must have the ability to self-learn the driving pattern of the driver, in addition, to be reliably pattern and predict the driving behavior to predict the drowsiness warning system. Additionally, there is also a need for performing real-time data processing of the facial and alertness pattern of the driver and a means for alerting the driver, record, and retrieval of the pattern via a central server.

SUMMARY OF THE INVENTION
In accordance with one of the embodiments of the present invention, a vehicle driver drowsiness warning system is provided. The drowsiness detection system includes in-vehicle driver identification system, an in-vehicle eyelid closing pattern recognition system, a central data processing system, and a centralized server. In an embodiment, the in-vehicle driver identification system is configured to recognize the driver and load profile of a recognized driver. In one embodiment of the present invention, the in-vehicle eyelid closing pattern recognition system comprises a camera that continuously monitors the facial graph, including the blinking pattern of the driver. The camera could be placed at any suitable location inside the vehicle for effective monitoring of the driver's blinking pattern of eyes. The data gathered from the said camera is stored and processed at the central data processing system, which located inside the vehicle at a suitable place. The central data processing system, apart from collecting data, also compares it with its previously stored data of the same user to accurately determine if the driver is distracted from driving owing to drowsiness or partial closing of eyes.
In another embodiment of the present invention, the data gathered is directly transferred to a server and processed therein. The server also stores data over a central server and compares the eyelid closing pattern with its previously stored data. Once the data is processed and compared with the previously stored data, the server or the central data processing system, on a continual basis, compares and analyzes the data to determine if the driver of the vehicle is experiencing drowsiness or uneasiness. On determining the same, an alarm activated. Additionally, an alert via phone is also generated for the driver to appraise his condition of drowsiness. Additionally, a copy of the alert is also sent to the pre-fed contact number in the system, which could alert the given list persons on the impeding condition of the driver. This is done via an Internet of Things (IoT) device performing connectivity as well as partial processing of data.
In accordance with another embodiment of the present invention, a method of vehicle driver drowsiness warning system is provided. The method includes following steps acquiring in-vehicle driver identification for each authorized driver by a finger print scanner; extracting the relevant in-vehicle eyelid closing pattern of said driver from an on-board memory system; comparing said eyelid closing data from said on-board memory system to the eyelid closing pattern of the driver; establishing a relationship on drowsiness of the driver by continuous sampling of said eyelid closing pattern of the driver; enabling an onboard alarm to the driver informing on the identification of drowsiness; and sending alerts to a number of pre-fed contact list on the geo-location of the vehicle, alertness level of the driver and physical condition of the said vehicle.
To further clarify the advantages and features of the present disclosure, a more particular description of the disclosure will follow by reference to specific embodiments thereof, which are illustrated in the appended figures. It is to be appreciated that these figures depict only typical embodiments of the disclosure and are therefore not to be considered limiting in scope. The disclosure will be described and explained with additional specificity and detail with the appended figures.

BRIEF DESCRIPTION OF THE DRAWINGS
The disclosure will be described and explained with additional specificity and detail with the accompanying figures in which:
Figure 1 illustrates the system level diagram of the drowsiness detection system according to the present invention.
Figure 2 illustrates the block diagram of the drowsiness detection system architecture according to the present invention.
Figure 3 illustrates a block diagram of a computer or a server in accordance with an embodiment of the present disclosure.
Figure 4 illustrates the flow diagram of the working of the drowsiness detection system architecture according to the present invention.
Figure 5 illustrates the block diagram of the control and data process unit according to the present invention.
Figure 6 and Figure 7 illustrate the means of eyelid closing pattern on the drowsiness recognition system.
Further, those skilled in the art will appreciate that elements in the figures are illustrated for simplicity and may not have necessarily been drawn to scale. Furthermore, in terms of the construction of the device, one or more components of the device may have been represented in the figures by conventional symbols, and the figures may show only those specific details that are pertinent to understanding the embodiments of the present disclosure so as not to obscure the figures with details that will be readily apparent to those skilled in the art having the benefit of the description herein.

DETAILED DESCRIPTION OF THE INVENTION
For the purpose of promoting an understanding of the principles of the disclosure, reference will now be made to the embodiment illustrated in the figures, and specific language will be used to describe them. It will nevertheless be understood that no limitation of the scope of the disclosure is thereby intended. Such alterations and further modifications in the illustrated system, and such further applications of the principles of the disclosure as would normally occur to those skilled in the art are to be construed as being within the scope of the present disclosure.
The terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process or method that comprises a list of steps does not include only those steps but may include other steps not expressly listed or inherent to such a process or method. Similarly, one or more devices or sub-systems or elements or structures or components preceded by "comprises... a" does not, without more constraints, preclude the existence of other devices, sub-systems, elements, structures, components, additional devices, additional sub-systems, additional elements, additional structures or additional components. Appearances of the phrase "in an embodiment", "in another embodiment" and similar language throughout this specification may, but not necessarily do, all refer to the same embodiment.
Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this disclosure belongs. The system, methods, and examples provided herein are only illustrative and not intended to be limiting.
In the following specification and the claims, reference will be made to a number of terms, which shall be defined to have the following meanings. The singular forms “a”, “an”, and “the” include plural references unless the context clearly dictates otherwise.
Figure 1 illustrates the system level diagram of the drowsiness detection system (100) according to the present invention. The drowsiness detection system (100) includes an in-vehicle eyelid closing pattern recognition system (130), a central data processing system (110), and a centralized server (120). In an embodiment, the centralized server may be a cloud server. In one embodiment of the present invention, the in-vehicle eyelid closing pattern recognition system (130) comprises a camera that continuously monitors the facial graph, including the blinking pattern of the driver. In an embodiment, the camera may be an infrared enabled camera. The in-vehicle eyelid closing pattern recognition system (130) is configured to capture a first set of facial data corresponding to the driver at a first time instant, and second set of facial data corresponding to the driver at a second time instant, and identify first eyelid closing pattern of the driver based on the first set of facial data and second eyelid closing pattern of the driver based on the second set of facial data. The facial data is captured with a predefined interval on a continuous basis. The duration of the interval could be from 1 second to 1 minute. The camera could be placed at any suitable location inside the vehicle for effective monitoring of the driver's blinking pattern of eyes. The data gathered from the said camera is stored and processed at the central data processing system (110), which located inside the vehicle at a suitable place. The central data processing system (110), apart from collecting data, also compares it with its previously stored data of the same user to accurately determine if the driver is distracted from driving owing to drowsiness or partial closing of eyes. In other words, the central data processing system (110) detects drowsiness of the driver based on comparison of the first eyelid closing pattern of the driver with the second eyelid closing pattern of the driver pattern of the driver. The first eyelid closing pattern of the driver act as a reference point and the drowsiness is detected when degree of variation of second eyelid closing pattern is more than 10 degrees and time of eyelid closing is more than one second as compared to the first eyelid closing pattern as illustrated in Figure 6 and Figure 7.
Here the in another embodiment of the present invention, the data gathered is directly transferred to a server (120) and processed therein. The server also stores data over a central server and compares the eyelid closing pattern with its previously stored data. Once the data is processed and compared with the previously stored data, the server (120) or the central data processing system (110), on a continual basis, compares and analyzes the data to determine if the driver of the vehicle is experiencing drowsiness or uneasiness. On determining the same, an alarm activated. Additionally, an alert via phone is also generated for the driver to appraise his condition of drowsiness. Additionally, a copy of the alert is also sent to the pre-fed contact number in the system, which could alert the given list persons on the impeding condition of the driver. This is done via an Internet of Things (IoT) device performing connectivity as well as partial processing of data. The information sent to the pre-fed list of contacts includes the geolocation of the vehicle, the health of the vehicle as well as information on next possible location of rest for the driver. A copy of the information is also displayed to the driver through in vehicle display as well as via mobile communication device. Additionally, the central data processing system (110) on a continual level, studies the route taken by the driver, studying its terrain and traffic, possible places of rest in case of emergency, nearest point of emergency assistance and creates a profile based on such route. Additionally, the driver driving pattern is also studied that includes his usual route, the duration of driving, the rest stops, the total time period of driving on detection of drowsiness etc. to provide a predictive model in the said central data processing system (110) for oncoming drowsiness identification. This is done through continuous update of the central data processing system (110) through machine learning of the route, driver and his driving pattern and vehicle health. Thus, the central data processing system (110) and the centralized server (120) independently generates one or more alerts upon detection of drowsiness by the central data processing system (110) by at least one of an audio means or visual means to the vehicle driver, and sends one or more alerts to a number of pre-fed contact list on the geo-location of the vehicle, alertness level of the driver and physical condition of the said vehicle.

Figure 2 and figure 3 illustrates the block diagram of the drowsiness detection system architecture according to one of the aspects of the present invention. The working of the drowsiness detection system (200) could be defined under the functioning of its various sub-systems.
The system includes an in-vehicle driver identification system (210) which is configured to recognize the driver based on biometrics. The biometrics may include fingerprint and facial features. In an embodiment, the in-vehicle driver identification system (210) may be fingerprint scanner adapted to receive the fingerprint of the driver and authenticate. In such an embodiment, in-vehicle driver identification system (210) is also configured to load profile of a recognized driver via fingerprint. In another embodiment, the in-vehicle driver identification system (210) may be a camera adapted to facial features of the driver and authenticate. In such an embodiment, the in-vehicle driver identification system (210) uses a facial recognition system to map facial features from a photograph or video of the driver captured. It compares the information with a database of known faces of drivers to find a match. In one of the aspects of the present invention, the driver logs in his presence via a fingerprint scanner (210). This fingerprint scanner could be replaced with other proximity sensors or any other means to determine the presence of the authorized driver or user in the vehicle. This feature is very useful in case of multiple drivers operating same vehicle. It becomes easy to track all the drivers driven a given vehicle for a single trip, over a period of time, routes covered, time taken by all the drivers in completing a trip along with their individual contributions. This will ensure that in case of public transport vehicles or in case of multiple drivers for same vehicles, a single device will support all drivers.
This system not only allows restricted access to the vehicle while contributing to the security and safety of the vehicle but also helps in tracking the users and their eyelid closing pattern for both in-site (local memory) and off-site (cloud) resources. Additionally, there is camera (220) put at a suitable place in the vehicle, more particularly either at the dashboard or at the rearview mirror mounting point.
Alternatively, in another embodiment of the present invention, the camera could be replaced with any other camera or sensor system for determining the level on the eyelids and their opening and closing patterns for accurately monitoring and activating vehicle driver drowsiness. The input signals and information from said camera (220) and fingerprint scanner (210) is fed to the memory (209).
The memory comprises a local frame memory (230) that stores and compares the face and eye blinking pattern stored at the specialized unit (240). The specialized face and eye pattern storage unit (240) is updated continuously, and the performance of the overall system gradually improves as more patterns of data are recorded and updated in the pattern storage unit (240). The data stored in the face and eye pattern storage unit is further analyzed and processed at the control and data process unit (250).
In another embodiment of the present invention, the data processing can also happen at a remote location, and the transfer of data is done through suitable data transfer means which are predominantly wireless. The data processing unit (250) further stores the data at the server simultaneously, giving alert and sounding alarm (270) to the vehicle driver.
Additionally, in another embodiment of the present invention, an alert is also sent to the pre-defined list of contacts by the driver/user on the condition of the driver.
Figure 3 illustrates a block diagram of a computer or a server in accordance with an embodiment of the present disclosure. The drowsiness detection system architecture includes processor(s) (250), and memory (209) operatively coupled to a bus (212).
The processor(s) (250), as used herein, means any type of computational circuit, such as, but not limited to, a microprocessor, a microcontroller, a complex instruction set computing microprocessor, a reduced instruction set computing microprocessor, a very long instruction word microprocessor, an explicitly parallel instruction computing microprocessor, a digital signal processor, or any other type of processing circuit, or a combination thereof.
The memory (209) includes several subsystems stored in the form of executable program which instructs the processor (250) to perform the method steps illustrated in Figure 1 and Figure 4. The memory (209) has following subsystems: a frame memory subsystem (230) and said face and an eye pattern storage unit (240).
Figure 4 illustrates the working (300) of the vehicle driver drowsiness warning system. In the first step (310), the driver is recognized based on the biometrics by the system by suitable detection means (substantially similar to the in-vehicle driver identification system (210) such as cameras and/or any other suitable means of detection such as proximity sensors or camera configured to capture facial features. The extraction could be done via the on-board storage unit or via cloud wirelessly. This could be achieved via suitable IoT connectivity. The profile of the recognized driver is also loaded to the vehicle driver drowsiness warning system.
In the second and third step (320) (330), there is extraction of data from storage or cloud, and it is compared with the previously stored pattern, and a closed-loop logic circuit is built around it which gives continual and closed-loop feedback to the processor for assessing the eyelid closing pattern of the driver.
In the fourth step (340), the eye pattern is compared with the previously stored and analyzed data to give a close loop control and continual improvisation over the recognition pattern of the driver.
In the fifth step (350), there is close-loop processing of data that would establish a relationship between eyelids closing patterns with the stored data to determine if there is indeed initiation of drowsiness on behalf of the driver. The pattern is continuously studied to prevent any false occurrence of alarm to the user, and the past data stored either in the cloud or the local memory helps in overcoming such a situation.
In the sixth step (360), the driver is informed through a sound alert in case the processor determines if the driver’s eyes close owing to drowsiness or being sick. In another embodiment of the present invention, the sound system is substituted with other suitable means such as the vibration of the seat or automated call over the phone to alert the driver on being drowsy. The alert may also include the geographical position of the vehicle and information about its occupants via means of GPS or Glonass or any such service provider on global coordinate position of man and materials.
Figure 5 illustrates the IoT device that is incorporated in the control and data processing unit (250). The data or information from the face and eyelid pattern storage unit (240) (figure 2) is fed to the control and data processing unit (250). The information is acquired by an eyelid closing pattern recognition system (130) which compares the information with the previous stored driver data in the data analysis (420) module. The information is fed to the machine learning module (430), which apart from considering the information generated from data analysis module (420) takes into consideration factors such as its geolocation through either GPS or any other suitable means. Additionally, the machine learning module (430) includes the driving pattern of the driver, the health of the vehicle, route map of the driver of the route frequently taken, etc. This helps in determining preferred place of rest or access during driving emergency as well as monitoring fatigue level of the driver. This information is continuously update in the memory update module (440) that generates signal in the signal module (450). The signal generated in the signal module (450) is projected through visual and sonic or audio alert system (270) and stored as back-up in the cloud storage (260). In addition to storage of information as back-up in the cloud storage (260), alerts are also independently generated in addition to in-vehicle alert system for removal of any chance of missing communication link. This is done via a suitable communication device such as a wireless communication system promoted through a IoT device.
Figure 6 and figure 7 illustrates the three dimensional extraction of the face data by the infra-red camera. As illustrated in the figures, the camera studies the face inclination, the eye-lid position relative to the face and stores in the local or the cloud memory. The analysis is done with the previously stored data and in case there is variation of face angle or eye-lid position, for example, in the range of 10-20 degrees, it prompts the vehicle drowsiness alarm system.
Additionally, the time factor is also taken into consideration wherein if the duration of closure of eye-lids is more than one second, in addition to range of variation of closure of eyelids (10-20 degrees), this gives an instant notification to the pre-fed contact list in addition to the vehicle user on various aspects of the vehicle such as its geolocation such as exact GPS coordinates, the condition of vehicle, probable area of rest, et al. This visual analysis using camera or any other suitable means can provide three dimensional data without mechanical scanning of the face and eyelids. This sampling is done at a preset interval, for determining rate of alertness of the driver in the vehicle. The sampling rate could vary from one to thirty seconds.
Figure 6 and Figure 7 illustrate the three dimensional eyelid closing pattern recognition pattern which is done at a periodic intervals via sampling system. Figure 6 illustrates the grid pattern learning of the face data by said camera (220), transferring the data to the control and data processing unit (250) and thereafter to the cloud storage (260). When the closure of eyelids is more than a predetermined level, for example, ten degrees for a particular driver, and time of closure is more than one second, this generates a value that the driver is experiencing drowsiness. Determining so gives signal to the local sonic device to raise an alarm as well as initiate an communication alert through a suitable communication device such as IoT devices to a pre-fed list of contacts. The alert through the communication device may include information such as its geo-location or its GPS position, next possible resting place to the driver as well as to the list of pre-fed contacts, and health of the vehicle including but not limited to physical condition of the vehicle.
It will be understood by those skilled in the art that the foregoing general description and the following detailed description are exemplary and explanatory of the disclosure and are not intended to be restrictive thereof.
While specific language has been used to describe the disclosure, any limitations arising on account of the same are not intended. As would be apparent to a person skilled in the art, various working modifications may be made to the method in order to implement the inventive concept as taught herein.
The figures and the foregoing description give examples of embodiments. Those skilled in the art will appreciate that one or more of the described elements may well be combined into a single functional element. Alternatively, certain elements may be split into multiple functional elements. Elements from one embodiment may be added to another embodiment. For example, the order of processes described herein may be changed and are not limited to the manner described herein. Moreover, the actions of any flow diagram need not be implemented in the order shown; nor do all of the acts need to be necessarily performed. Also, those acts that are not dependent on other acts may be performed in parallel with the other acts. The scope of embodiments is by no means limited by these specific examples.