Description:A SYSTEM/METHOD TO DETECT FACE MASKS AND RECOGNISE SOCIAL DISTANCING USING DEEP LEARNING ALGORITHMS
Field of Invention
The present invention is relating to identify the social distance between the humans and face mask detection in the public areas based on computer vision and deep learning algorithms.
The Objectives of this Invention
The main goals of this invention are to create safe environments that improve the public's safety. To this end, we've developed a practical computer vision-based tackle focusing on the real-time computerized surveillance of people that recognize both safe social distance and masks for facial protection in common areas.
Background of the Invention
According to (US2010/10045742B2), A person's samples are taken, the specimen is tested to see if the individual in question has ever been infected with a specific virus, and if not, the person is given a sensor that can be set to detect when their hand gets close to their face. This method for hindering virus infection requires collecting a specimen from the individual in question. The risk of associated infections with viruses can be decreased by alerting the person to unwanted hand-to-face encounters. Another type of application invented in (US2021/11436881B2), The disclosure of a mask, temperature, and social distance monitoring technique. The method entails receiving several pictures or videos from a detection device; determining an individual's body temperature; carrying out the detection of face masks using the images or videos and a pre-trained mask recognition model; and producing one or additional control signals depending on the accomplished face mask recognition and the determined temperature of the body, the one or more control signals being programmed to cause the mask detection sub-system to grant accessibility when the person's body temperatures is below a certain threshold. Another method was invented in (CN2014/110248019B), These current disclosures include voice-enabled dialogue interfaces, computer-readable media, and methodologies. A computing device obtains a first input corresponding to a demand to open an application in particular and, in reaction, functions the gadget following an assumption that the device is being functioned in a restricted distraction circumstance. The electronic gadget also presents a prohibited disorientation user interface, which includes displaying fewer programmable user interface opposes than in an unhindered user connection for the specific app.
According to Hu et al.'s research (Hu et al. [2017], Journal of biomedical optics, vol. 22, no. 3, pp. 036006), respiration can be used to determine a person's physiologic state, making pulmonary signals, signs and symptoms that can, to some extent, indicate a person's state of health. Abnormal breathing symptoms may be crucial indicators for identifying some particular diseases, according to a large body of clinical literature. Early, mild respiratory symptoms are frequently hard to identify. Therefore, possible COVID-19 patients can be partially screened by monitoring respiratory conditions. According to Jiang et al. (2020), ArXiv abs/2004.06912), may serve as an adjunct diagnostic function, assisting in the early identification of probable patients. According to Armann et al’s research (2020, medRxiv, pp-1-16), The SchoolCoviDD19 study welcomed students in grades 8 through 11 and their teachers from 13 secondary educational institutions in eastern Saxony, Germany. Blood samples were taken in May/June 2020, four weeks after the conclusion of the summer break, and again in September 2020, four weeks after the schools reopened during the lockdown in March 2020. The chemiluminescence diagnostic technique was used to measure SARS-CoV-2 IgG, and all specimens with positive or ambiguous test results underwent two further serological examinations. Nevertheless, there is an increasing need for more methods of diagnostics because there aren't reliable automated instruments. Deep learning algorithms with magnetic resonance imaging can help accurately detect COVID-19, especially in remote locations with a dearth of skilled medical personnel. This paper proposes a method for employing Capsule Neural Networks to recognize COVID-19 from unprocessed chest X-ray pictures. The suggested model outperformed existing convolution neural network models with an accuracy of 93.58% and a loss rate of just 0.32. Healthcare workers may find this strategy useful for quickly identifying COVID-19 instances by sritha et al (2023 2nd International Conference on Vision Towards Emerging Trends in Communication and Networking Technologies (ViTECoN), Vellore, India, 2023, pp. 1-5).
Summary of the Invention
The proposed invention presented a method that uses computer vision and MobileNet V2 designs to help preserve a safe atmosphere and ensure people's safety by continuously monitoring public places to prevent the spread of the COVID-19 virus. It also helps police reduce their physical observation work in confinement regions, and open spaces where observation is necessary by using real-time camera feeds with raspberry pi4.
Detailed Description of the Invention
The suggested approach helps to guarantee people's safety in public places by autonomously monitoring them to check if they maintain a proper distance from others and by determining whether or not they possess face masks. The architecture of the suggested treatment is briefly explained in the following section, along with how it will continually block the coronavirus from propagating. The suggested system uses a camera fitted inside a Raspberry Pi 4 to observe people in public areas automatically and, when it comes to efficiency optimization, can recognize persons sporting masks using the transfer of learning technique. We also apply fine-tuning, another transfer learning technique, and feature extraction.
Transmitted through RTSP from the Network Video Recorder (NVR). These frames are then converted to grayscale and given to the model inside the Raspberry Pi 4 to speed up and improve accuracy. Although the MobileNetV2 design offers a significant cost benefit over the conventional 2D CNN approach, we selected it as the primary model for detection. The SSD MultiBox Detector, a neural network architecture developed for excellent quality picture categorization on a substantial database of pictures comprising ImageNet and PascalVOC, also forms part of the procedure.
The network head is removed, a new FC head is created and attached to the base in place of the previous head, and the network's base layers are frozen. The MobileNet V2 is subsequently loaded with previously trained ImageNet weights. The head layer weights will change during the backpropagation's fine-tuning stage, irrespective of the weighting of these base layers. After the information is prepared, and the mathematical framework is ready for fine-tuning, the model is developed and trained. OpenCV, TensorFlow, Deep Learning, and Computer Vision investigations have been carried out to examine the safe social distance between recognized humans and the identification of face masks in real-time video streams. To prevent a significant departure from the previously learned convolutional filtering algorithms, a relatively low acquisition rate is implemented during the retraining process of the architecture. The three main characteristics of the suggested method are face mask recognition, determining the safe distance amongst recognized persons, and person detection. Real-time identification of individuals is possible with Single Shot object Detection (SSD), which outperforms the Faster R-CNN model by 91.2% mAP and MobileNet V2 with OpenCV. Everybody who is discovered will have a limitless space visible surrounding them. In this system, SSD can identify several objects in a frame, but it can only recognize one person at a time. To calculate the distance between two individuals, we must first calculate the distance they are from the camera's lens using the triangle comparison approach, ascertain the assumed focal length of the camera, assume their separation from the camera to be D, and their actual height to be H, which is 165 cms, and finally use SSD individual recognition to calculate their pixel height P using the box that contains directions. Using these numbers, the camera's focal length can be calculated using the equation.
After identifying the depth of the subject in the camera, we calculate the gap between the two people in the video. A video might show multiple people. As a result, the distance calculated by Euclid is used to calculate the median of the boxes that define the boundaries for each recognized person. We next converted the x and y values into cm after obtaining the x and y values. Each person's centimeter-based x, y, and z locations, which signify their distance from the camera, are knownUsing the (x, y, z) coordinates, the distance calculated from Euclid between each person found is determined. A red border line will appear around two people to show that they still need to maintain a social space between them if there are less than two metres (200 cm) separating them.
The suggested method uses mobileNet V2 construction as a foundation. It applies to learning through transfer on top of the exceptionally successful pre-trained SSD model for facial recognition to create a compact, accurate, and efficient computational model that is easier to deploy to a raspberry pi. Approximately 3165 pictures from special face crop information sets, . Annotated photos train a deep learning binary classification approach, which divides input images into groups with and without masks based on output class probability. A box with boundaries and an extracted human mask is displayed in the output of the SSD model. The suggested system constantly scans public areas, whenever an individual without a respirator is spotted, the person's face is photographed, and an alarm with an image of their face given to the officials. In the meantime, the gap between individuals is assessed in real-time. As well as whether more than 20 individuals have consistently been determined to be going beyond the recommended safe social distance, an alarm is transmitted to the State Police Headquarters' control center at an appropriate time to initiate further action. This method can be applied in immediate scenarios that demand secure monitoring of interpersonal distance and the recognition of masks on people's faces due to safety concerns resulting from the Covid-19 epidemic. This design was chosen because it may ease the responsibility of physical surveillance by disseminating our approach to edge units for continuing public monitoring. This sort of equipment can be utilized in train stations, airports, offices, schools, and other gathering areas to guarantee that public safety regulations are observed. This system is also compatible with edge devices.
The proposed method uses a special data set labeled and utilised to train our models, consisting of face pictures with various face masks. We use the current subtraction of background [21,22] technique in the pre-processing stage. The SSD methodology deals with both the validation of mask wear and the real-time computerized determination of social distance control. 3165 photos comprise the dataset used to train our proposed face mask classifier. The data collection is split into testing and training data sets before the customized face mask image data is labeled. There should be 80% of pictures in the original training data set and 20% in the data used for the testing set. The classifications for the images in the initial data gathering are mask and no mask.
The recommended approach facilitates the automatic execution of the psychological distance inspection operation. After the model has been trained using the unique data set and predetermined weights, we evaluate its performance on the test dataset by presenting a bounding box with the tag's name and its the top of the box. The proposed approach acknowledges everyone in the field of view of the cameras first and displays a green bounding box around each person separated from other people. The hue of the bounding box transforms to red for a person who disobeys social distance rules in a public situation. In addition, whether the identified person is wearing a mask or not, identification of face masks is carried out by projecting box boundaries onto the identified individual's face. If the equipment detects the mask in the faces, but the social distance fails to be preserved, it generates an advisory and transmits an alert to surveillance institutions with a face image. The system can recognize social separation and coverings with a score for precision of 91.7%, a score for trust of 0.7, an accuracy rating of 0.91, an average recall of 0.91, and a false-positive rate of 28.07.

5 Claims & 1 Figure
Brief description of Drawing
In the figure which are illustrate exemplary embodiments of the invention.
Figure 1, The Process of Proposed Invention of Safe Social Distancing and Face Mask Detection , Claims:The scope of the invention is defined by the following claims:

Claim:
1. A system/method for identifying the social distance between the humans and face mask detection in the public areas based on computer vision and deep learning algorithms, said system/method comprising the steps of:
a) The system starts with live camera (1), from this data frame will be read (2) and persons detection inference will trigger (3) to calculate the scoring box values (4).
b) After that, the face detection inference box (5) is started along with face mask detection model (6) based on centroid value of the boxes (7).
c) The system continuously calculates the measurement between two humans (8), if anyone constraints is not satisfied then the invention will trigger the alerts (9) to the public places (10).
2. As mentioned in claim 1, the live camera will record the activity of the humans in the public places like Malls or markets. The photo frame will capture and analyse the scores of the frames.
3. According to claim 1, the extracted features are used as input to machine learning algorithms to calculate the distance between the humans and also the human wearing the face mask or not.
4. As per claim 1, to trained model, alert a system based on anyone of the conditions if distance is very close alert to the human via display or announcements.
5. As per claim 1, if the user not wearing the mask, the trained model will announce to the public to alert.