Description:FORM 2
THE PATENTS ACT 1970
(39 of 1970)
&
The Patents Rules, 2003
COMPLETE SPECIFICATION
(See section 10 and rule 13)

1. TITLE OF THE INVENTION: “A PROCESS FOR FACIAL RECOGNITION IN A DYNAMIC ENVIRONMENT”

2. APPLICANT:

(a) NAME : Nervanik AI Labs Pvt. Ltd.
(b) NATIONALITY : Indian
(c) ADDRESS : A – 1111, World Trade Tower,
Off. S G Road, B/H Skoda Showroom,
Makarba, Ahmedabad – 380051
Gujarat, INDIA.
3. PREAMBLE TO THE DESCRIPTION
PROVISIONAL

The following specification describes the invention. þ COMPLETE

The following specification particularly describes the invention and the manner in which it is to be performed.

Field of the invention
The present invention relates to a process for facial recognition in a dynamic environment. More particularly, the present invention relates to the field of biometric recognition, and specifically relates to identify a human face through an image, a video stream or a live stream through an installed camera. The main aim of the present invention is centered around the recognition of faces within dynamic environment, where factors such as lighting conditions, pose variations, transient expressions, and occlusions are constantly changing.

Background of the invention
In recent years, with the rapid development of technologies and the increasing actual demand, face recognition has become more and more popular in the field of biometrics. With the development of technology, image processing technology has been applied to more and more fields. Generally, machine learning methods can be used to train face recognition models, and face recognition models can be used to recognize faces in the represented images.

The existing face detection and recognition mainly use visible light images for recognition. For example, color images are used for face detection and recognition. However, face detection and recognition based on color images are more sensitive to light. The color image in the environment has poor recognition effect. Another face recognition system relied entirely on the potential of the administrator to identify and extract feature points such as eyes, nose, ears, and mouth from photographs.

However, this technology could be used for simple tasks in our day-to-day lives. Organizations can prepare themselves for a post-COVID work atmosphere by replacing fingerprints in attendance logs with facial recognition system. Further, with the education system, which has currently shifted learning online, facial recognition could be integrated with video conferencing platforms to track students’ attention on the screen.

There is still a certain gap between domestic intelligent face recognition systems and foreign cutting-edge technologies in terms of face image collection, recognition accuracy and speed. The stability of products in practical is also not ideal, which is mainly reflected in most products. It is sensitive to changes in lighting, age, expression, posture, distance and other conditions. When certain conditions change slightly, the recognition effect will be greatly reduced, and the search and comparison speed cannot meet the requirements of real-time processing. More importantly, when the database size increases, the false recognition rate (FAR) is too high, and the resulting false alarm phenomenon makes many businesses and public security departments shy away from security alarm systems based on facial recognition. The above factors have seriously restricted the promotion and facial recognition technology in country's high-end intelligent monitoring and large-scale database accurate searches that have security value.

In most cases, a static environment (such as a photograph or a single frame from a video) is used for facial recognition, but in a non-static environment, such as real-time video surveillance or live streaming, facial recognition systems need to handle additional challenges , such as changes in lighting, pose, expression, and occlusions.

In view of the above technical problems, the purpose of the present invention is to provide a process for facial recognition in a dynamic environment. Face recognition is the process of identifying or verifying the identity of a person using their face based on a person’s facial details, it captures, analyses, and compares patterns with stored database.

The present invention process for facial recognition in a dynamic environment is to solve the problem of the robustness of the existing face recognition system to illumination. The main aim of the present invention is centered around the recognition of faces within dynamic environment, where factors such as lighting conditions, pose variations, transient expressions, and occlusions are constantly changing. These challenges have traditionally hindered the accuracy and reliability of facial recognition systems, limiting their applicability in scenarios where movement is a constant factor. In continuation, for moving environments, the present invention process incorporates techniques to adapt to changing environmental conditions, such as adjusting for variations in lighting or handling pose and expression changes. This may involve utilizing multiple facial images of an individual captured from different angles or under different conditions to enhance recognition accuracy.

Object of the invention
The main object of the present invention is to disclose a process for facial recognition in a dynamic environment.
Another object of the present invention is to detect and identify a human face in a real-time through an image, a video stream, or a live stream through camera device.
The further object of the present invention is to provide the device installed in vehicle dedicated for capturing video and a more specialized and effective approach to identifying faces in real time and showcasing.
The other object of the present invention is to provide the dual-camera device setup that is equipped with stored database and advance machine learning module which monitors both the external driving environment and the driver's actions inside cabin of the vehicle in real-time.
Another object of the present invention is to establish a cloud based storage framework for data storage and face recognition and also manage best facial images efficiently that serves as a secure and scalable solution for preserving the selected facial data.
Further object of the present invention is to capture the best face from the camera stream involves a multi-step face capture process includes a face detection, extraction of identity features, pixel quality evaluation, yaw and pitch thresholding and region-specific quality assessments.
Another object of the present invention is to provide real-time face recognition system in dynamic environments with moving cameras and jerks.
The further object of the present invention is to provide the device equipped with GNSS (Global Navigation Satellite System) and IMU sensors for essential vehicle data, enabling features like route optimization and accident reconstruction.
Another object of the present invention is to provide a driver monitoring and identification, driver behaviour analysis and driver score measurement.
Still another object of the present invention is to provide multi-level filtering for face recognition in dynamic environments with moving cameras, ensuring a more robust and accurate identification process.

Summary of the Invention
The present invention relates to a process for facial recognition in a dynamic environment. A facial recognition is the process of detecting and identifying a human face in real-time through an image, a video stream, or a live stream through camera device. The present invention includes a hardware platform configured with the two camera devices; one camera device displays the road side view and the other camera device display the driver side cabin view of vehicle configured with stored database and machine learning model. The process of a face recognition system establishes a cloud based storage framework to store and manage best facial images. The process of the present invention includes: initiating face detection on image to identify face area through face detection model; performing face extraction to identify facial features on the image through face landmark detection model and assessment of pixel quality index to analyze the pixel quality of the image of the detected face. Calculating yaw and pitch values of the detected face to check whether the yaw and pitch values lies in predefined criteria of threshold values. Perform region specific quality assessment for eye region’s quality assessment, mouth region quality assessment and gauging features like lip to select qualitative facial images.

Another embodiment of the present invention includes the process of training a facial recognition system (FRS) acquiring good quality videos of drivers to serve for training the FRS to ensure that the videos capture of the drivers in various scenarios, lighting conditions, and angles to simulate real-world situations. Identifying optimal facial representation once the video data is gathered based on deep convolutional neural network model to locate and extract the best straight face from the video frames. Further, the best face captured is then passed through face recognition model. Said model generates a face embedding, a numerical representation that encapsulates the unique facial features of the driver. A separate database is configured with the FRS training to stored face embedding in to the database with given Face id or name to identify the person.

Further, another embodiment of the present invention involves inference model to retrieve the qualitative face images from previously captured from the device stored in the cloud based storage framework. Each retrieved qualitative face image is then passed through the inference model of the FRS to calculate face embedding of the best-face image. The calculated face embedding data is then compared against a pre-existing database of known faces. Said database contains the face embeddings of drivers has been previously onboarded into the FRS training. The FRS employs machine learning module or distance method to compute similarity scores between the calculated face embedding and the embedding previously stored in the database of the FRS training leading to a recognition decision based on predefined thresholds. The present invention FRS inference results further integrate with DMS (Driver monitoring system) event videos to enrich the process of refining accuracy and ensuring driver safety. Furthermore, for dynamic environment the FRS configured to adapt changing environmental conditions, such as adjusting for variations in lighting or handling pose and expression changes that involve utilizing multiple facial images of an individual captured from different angles or under different conditions to enhance recognition accuracy.

Brief Description of the Drawings
FIG. 1 illustrates a schematic structural diagram of a process for facial recognition in a dynamic environment according to the present invention.
Fig. 2 illustrates a flowchart of one embodiment of a process for facial recognition in a dynamic environment according to the present invention.
Fig. 3 illustrates a flowchart of another embodiment of a process for facial recognition in a dynamic environment according to the present invention.
FIG. 4 illustrates a flowchart of still another embodiment of a process for facial recognition in a dynamic environment according to the present invention.
Fig. 5 illustrates a flowchart of one specific embodiment of process for facial recognition in a dynamic environment according to the present invention.

Detailed description of the Invention
Before explaining the present invention in detail, it is to be understood that the invention is not limited in its application. The nature of invention and the manner in which it is performed is clearly described in the specification. The invention has various components and they are clearly described in the following pages of the complete specification. It is to be understood that the phraseology and terminology employed herein is for the purpose of description and not of limitation.
As used herein, the term “FRS”, refers to a face recognition system.
As used herein, the term “ADAS”, refers to an advance driver-assistance system.
As used herein, the term “DMS”, refers to an driver monitoring system.
As used herein, the term "module", and “model” refers to as the unique and addressable components of the software implemented in hardware which can be solved and modified independently without disturbing (or affecting in very small amount) other modules of the software implemented in hardware.
As used herein, the term “device”, refers to a unit of hardware, outside or inside the case or housing that is capable of providing input or of receiving output or of both.
As used herein, the term "database" refers to either a body of data, a relational database management system (RDBMS), or to both. The database includes any collection of data including hierarchical databases, relational databases, flat file databases, object-relational databases, object oriented databases, and any other structured collection of records or data that is stored in a computer system. The above examples are example only, and thus are not intended to limit in any way the definition and/or meaning of the term database.
The present invention disclosure is related to face detection. The face detection is used to find and identify human faces in digital images and video.
Another disclosure of the present invention is face landmark that basically represent individual’s facial features of facial images as multi-dimensional vectors and stores the data face image.
Further, disclosure of the present invention is a facial feature extraction is process of extracting face component features like eyes, nose, mouth, etc from human face image. Facial feature extraction is very much important for the initialization of processing techniques like face tracking, facial expression recognition or face recognition.
As per Fig. 1, a process for facial recognition in a dynamic environment according to the present invention includes a dual camera device configured with stored database and machine learning model. The present invention is a hardware platform configured with two cameras device, one camera device display the road side view and the other camera devcie display the driver side cabin view of the vehicle. The machine learning module to identify or verify the identity of a person/driver using their face and based on a person’s/driver facial details captures, analyses, and compares patterns as stored in the database. The primary focus of the driver-side camera device is to detect and identify a human face/driver in real-time through an image, a video stream, or a live stream to ensure their attentiveness and emotional state during the driving. Specially, such as lighting conditions, pose variations, transient expressions, and occlusions are constantly changing.
Said camera device further configured with GNSS (Global Navigation Satellite System) and IMU (Inertial Measurement Unit) sensors to get real-time location and orientation details of the vehicular movement. In continuation, integration of video data and Global Navigation Satellite System (GNSS) information in vehicles for monitoring and identification of the driver. Said integration enables real-time analysis of frames in conjunction with location data, ensure accurate and timely capture of crucial driver information. As the vehicle operates, the integrated system analyzes every frame from the camera device in real time and captured best facial image within a timeframe. The captured best face is then processed and compared against stored facial profiles to verify the driver's identity. This step aids in confirming whether the driver has changed or remained consistent since the last session.
Fig. 2 illustrates a flowchart of one embodiment of a process for facial recognition in a dynamic environment according to the present invention. This embodiment of the present invention provides an efficientprocess for facial recognition in a dynamic environment comprises:
S11: receiving image/video frames captured by a camera device on a vehicle, the image frames including a plurality of images depicting a face of a driver in the vehicle;
S12: establishing a cloud based storage framework to store and manage the qualitative facial images.
S13: performing face detection on image to identify face area through face detection model.
S14: identifying facial features on the image through face landmark detection model.
S15: calculating pixel quality index to analyze the pixel quality of the image of the detected face.
In above mentioned step S15, calculate the pixel quality index to ensure that the captured images are clear, well-defined and free from noise or artifacts.
S16: calculating yaw and pitch values of the detected face to check whether the yaw and pitch values lies in predefined criteria of threshold values.
S17: Performing region specific quality assessment for eye region’s quality assessment, mouth region quality assessment and gauging features like lip to select qualitative facial images.
In Step S16, the yaw corresponds to the transition along a horizontal axis, while the pitch relates to a transition along a vertical axis. By defining specific threshold values, the process can filter out images where the face orientation deviates significantly from the desired angle, ensuring consistency and accuracy.
According to step18 of the present invention, the eye region's quality is evaluated, considering factors such as clarity, openness, and symmetry of the eyes. Similarly, the mouth region undergoes assessment, gauging features like lip visibility and symmetry calculation can be perform in the region specific quality assessment to capture the best quality face image
As per the above mentioned process of the present invention, selecting the qualitative face image based on predefined criteria through machine learning module and said qualitative faces are stored in to the cloud based storage framework.
Fig. 3 illustrates a flowchart of another embodiment of a process for facial recognition in a dynamic environment according to the present invention. The process of training a Facial Recognition System (FRS) for driver on boarding is a multi-step process that endeavors on capturing high-quality data and to ensure accurate identification. The initial step involves acquiring good-quality videos of drivers for training the FRS to ensure that the videos captured in various scenarios, lighting conditions, and angles to simulate real-world situations. Once the video data is gathered, identifying optimal facial representation through face detection model is employed to locate and extract best straight face from the video frames. Selecting the best straight face for further analysis and same stored in to separate database.
In continuation and with reference to Fig.3, captured best-face further proceeds for analysis through the face recognition model. Said model calculate face embedding, a numerical representation that encapsulates the unique feature of the driver. Said face embedding is associated with the driver's name and stored in a separate database. The separate set of data to evaluate the FRS ability to correctly match faces with the associated driver’s labels. The separate database stored the face embeddings along with associated labels. The next step is to establish a link between the individual's face and their corresponding label. Additionally, to enhance the accuracy of the face recognition system, it incorporates a diverse range of driver images and can be achieved by on boarding multiple images of the same driver captured from different angles and under varying conditions. By introducing said variations, the FRS becomes more robust and adept at recognizing drivers in dynamic settings.
In continuation and with reference to fig. 4 illustrates a flowchart of still another embodiment of a process for facial recognition in a dynamic environment according to the present invention. As shown in Fig.4, the process of this embodiment includes: retrieving the best face images previously captured from the camera devices during driving sessions and subsequently saved in the cloud based storage framework. Said images represent the optimal facial representation in varying conditions. After retrieving the best face image is systematically proceed through inference model of the FRS to calculate the face embedding of the best face image. The next step is, comparing the calculated face embedding data of the inference model against a pre-existing database of known faces. The pre-existing database contains the face embedding of drivers have been previously on boarded in to FRS.
Further, the FRS employs machine learning module or distance methods to calculate similarity scores between the calculated face embedding and the embeddings in the separate database. The higher similarity scores indicate a stronger resemblance and increase the probability of a successful match. Based on the calculated similarity scores, the FRS makes a recognition decision. Furthermore, if calculated face embedding closely similar with an existing database entry and greater than the predefined threshold the FRS identifies the driver associated with that entry. Said recognition decision communicated as the recognized driver for the given best-image.
Fig. 5 illustrates a flowchart of one specific embodiment of process for facial recognition in a dynamic environment according to the present invention. As shown in Fig. 5, this embodiment comprises driver monitoring system (DMS) configured with the camera device to monitors the driver’s state and behavior. The DMS detecting driver distraction and drowsiness in real-time, improves road safety, saving lives based on eye openness, gaze, head pose, facial expression, body activities, and much more. In continuation with the fig. 5 the FRS inference results integrate with the DMS events videos to enhance the accuracy of the Facial Recognition System (FRS) and to address failed driver recognition through advanced machine learning module and human intervention.
According to the present invention, following are the steps of the process of improving the FRS and mitigating recognition, receiving face image by the FRS to determine the recognition confidence. The FRS analyzes the facial features, lighting, and other factors to evaluate whether the face can be confidently recognized or it falls under an "unknown" category. In recognizing the face based on the existing data if system has high confidence than the recognition proceeds without human intervention. Whether the system has lower confidence or the face is categorized as unknown category the system triggers a manual annotation process.
The manual annotation process is for refining accuracy which extends to annotating DMS events videos, which capture diverse driver-related scenarios. A human reviewer examines these videos, identifying and annotating events that provide context for the recognition process. Said step contributes valuable insights to enhance both FRS and DMS functionalities. Through the annotations of DMS events videos deeper understanding of driver behaviors and actions and said annotations serve to enrich the FRS's, facilitating improved recognition accuracy based on real-world scenarios.
The next step is, if the manual annotation confirms a new face, the system initiates a cloud-based onboarding process. The new face is added in to the FRS dataset and all previous appearances of the same face are labeled as a newly onboarded driver. Said process is facilitated by machine learning module, streamlining data organization. Further in such cases where manual annotation underscores low recognition confidence, the system is updated with the new face image and introduced different views of the same face into the FRS and retaining the existing driver label. The FRS also employs a feedback loop mechanism where data from manual annotation, new face onboarding, and ADAS integration contribute to model refinement. Through the iterative process of manual annotation, new face onboarding, diverse views, and continuous learning, the FRS gradually achieves high accuracy in recognizing drivers. The system becomes adept at identifying individuals even in challenging scenarios, leading to a robust and reliable driver recognition system.
In the another embodiment of the present invention, the facial recognition system (FRS) configured into Advanced Driver Assistance Systems (ADAS) devices contributing to enhanced driver safety, behavior analysis, and overall driving experience. Said ADAS includes driver Monitoring and Identification, Driver Behavior Analysis and Driver's Score Measurement. The driver Monitoring and Identification through ADAS devices enables real-time recognition of the driver identities. By accurately identifying the driver behind the wheel, the system can personalize vehicle settings, preferences, and safety measures based on individual profiles. It ensures a tailored driving experience and seamless transition between different drivers sharing the same vehicle.
Furthermore, driver behavior analysis through ADAS devices can analyze various aspects of a driver's behavior. It can detect signs of drowsiness, distraction, and other potentially unsafe behaviors. By tracking facial movements, eye movements, and head positions, the system can alert the driver and prompt corrective actions, thus reducing the risk of accidents caused by driver inattention.
The driver's score measurement through ADAS device identity recognition, the FRS can determine specific behaviors with individual drivers. By analyzing patterns over time, the system generates a driver behavior score that reflects adherence to safe driving practices. Said score can be utilized for personalized coaching, improving driving habits, and fostering safer road behavior.
While various elements of the present invention have been described in detail, it is apparent that modification and adaptation of those elements will occur to those skilled in the art. It is expressly understood, however, that such modifications and adaptations are within the spirit and scope of the present invention as set forth in the following claims.
, Claims:We Claim:
1. A process for facial recognition in a dynamic environment comprising:
receiving image/video frames captured by a camera device on a vehicle, the image frames including a plurality of images depicting a face of a driver in the vehicle;
establishing a cloud based storage framework to store and manage qualitative facial images;
performing face detection on the image to identify face area through face detection model.;
identifying facial features on the image through face landmark detection model;
wherein,
calculating pixel quality index to analyze the pixel quality of the image of the detected face;
calculating yaw and pitch values of the detected face to check whether yaw and pitch values lies in predefined criteria of threshold values;
performing region specific quality assessment for eye region’s quality assessment, mouth region quality assessment and gauging features to select the qualitative facial images; and
obtaining the standardized face image based on the predefined criteria through machine learning model are stored in to a cloud based storage framework.

2. The process for facial recognition in a dynamic environment as claimed in claim 1, wherein training a facial recognition system(FRS) further comrpising:
selecting best straight face and stored in separate database;
analysing the captured best straight face through face recognition model of the FRS;
calculating face embedding a numerical representation encapsultes unique feature of the best straight face of the driver;
storing the face embeddings in the separate database with associated labels; and
establishing a link between individual’s face and corresponding labels.

3. The process for facial recognition in a dynamic environment as claimed in claim 1 and 2, further comprises:
retrieving the qualitative face images stored in the cloud based storage framework;
calculating face embedding of the qualitative face image of inference model through the macchine learning module of the FRS;
comparing calculated face embedding data of the inference model against a pre-existing data base of the faces stored in the separate database;
calculating similarity scores between the calculated face embedding and the embeddings in the separate database of the training FRS through the machine learning module;
obtaining higher similarity scores determine successful match based on the calculated similarity scores of the FRS;
determining match the calculated face embedding with an exisrting database entry greater than the predefined threshold;
identifying the driver associated with that entry the FRS determine a recogized driver of the given qualitative image;

4. The process for facial recognition in a dynamic environment as claimed in claim 1 and 3, further comprises:
configuring driver monitoring system(DMS) with the camera device to monitor the driver state and behvaiour;
detecting driver drowsiness and distraction in real-time through the DMS of the FRS;
integrating inference model results with the DMS events videos determine failed driver recognition through advanced machine learningmodel;

5. The system for facial recognition in a dynamic environment as claimed in claim 1, wherein a camera device to capture image/video of driver in vehicle, a cloud based storage framework to store and manage qualitative images captured by the camera device,
a face detection model identify face area of the images stored in the cloud based storage framework,
a face landmark detection model identify facial features of the image detected by the face detection model.

Dated this on 14th day of May 2024