Claims:I/We claim:
1. A method for training a model for detecting a Standard Operating Procedure (SOP) compliance, the method comprising:
receiving, by a processor, a media file from a user;
annotating, by the processor, at least an object in the media file;
assigning, by the processor, a Standard Operating Procedure (SOP) for the object;
augmenting, by the processor, the media file in runtime to obtain augmented data;
training, by the processor, a deep learning model with the augmented data to generate a baseline trained model, wherein the baseline trained model is generated after the training reaches a predefined threshold;
feeding, by the processor, a decoded camera stream to the baseline trained model;
detecting, by the processor, at least an SOP compliance from the decoded camera stream when the object is under observation based on the assigned SOP; and
recursively training, by the processor, the baseline trained model based on the SOP compliance, wherein the baseline trained model is recursively trained using semi-supervised learning techniques.

2. The method as claimed in claim 1, further comprises receiving feedback from the user on the detection of the SOP compliance.

3. The method as claimed in claim 2, wherein the baseline trained model is recursively trained based on the feedback received from the user.

4. The method as claimed in claim 1, further comprises alerting the user when the SOP is not complied.
5. The method as claimed in claim 1, wherein the augmentation is performed based on Colour Translations, Spatial Translations and Adversarial Training algorithms.

6. The method as claimed in claim 1, wherein the media file comprises at least an image and a video frame.

7. The method as claimed in claim 1, wherein the augmentation is performed by applying at least one of a shear effect, a noise effect, a modification of contrast, a modification of brightness, a blur effect and axes wise flipping.

8. The method as claimed in claim 1, wherein the SOP compliance indicates adhering to the assigned SOP.

9. A system for training a model for detecting a Standard Operating Procedure (SOP) violation, the system comprises:
a memory; and
a processor coupled to the memory, wherein the processor is configured to execute program instructions stored in the memory for:
receiving a media file from a user;
annotating at least an object in the media file;
assigning a Standard Operating Procedure (SOP) for the object;
augmenting the media file in runtime to obtain augmented data;
training a deep learning model with the augmented data to generate a baseline trained model, wherein the baseline trained model is generated after the training reaches a predefined threshold;
feeding a decoded camera stream to the baseline trained model;
detecting at least an SOP compliance from the decoded camera stream when the object is under observation based on the assigned SOP; and
recursively training the baseline trained model based on the SOP compliance, wherein the baseline trained model is recursively trained using semi-supervised learning techniques.

10. A non-transitory computer program product having embodied thereon a computer program for training a model for detecting a Standard Operating Procedure (SOP) violation, the computer program product storing instructions, the instructions for:
receiving a media file from a user;
annotating at least an object in the media file;
assigning a Standard Operating Procedure (SOP) for the object;
augmenting the media file in runtime to obtain augmented data;
training a deep learning model with the augmented data to generate a baseline trained model, wherein the baseline trained model is generated after the training reaches a predefined threshold;
feeding a decoded camera stream to the baseline trained model;
detecting at least an SOP compliance from the decoded camera stream when the object is under observation based on the assigned SOP; and
recursively training the baseline trained model based on the SOP compliance, wherein the baseline trained model is recursively trained using semi-supervised learning techniques. , Description:FORM 2

THE PATENTS ACT, 1970 (39 of 1970)
&
THE PATENT RULES, 2003

COMPLETE SPECIFICATION
(See Section 10 and Rule 13)

Title of invention:

Training a model for detecting a Standard Operating Procedure (SOP) compliance

Applicant:
Oureye.ai
Having address:
4th Floor, No 22, Salarpuria Towers-I Industrial Layout, Hosur Rd, 7th Block, Koramangala, Bengaluru, Karnataka 560095

The following specification describes the invention and the manner in which it is to be performed.

PRIORITY INFORMATION
[001] The present application does not claim a priority from any other application.
TECHNICAL FIELD
[002] The present subject matter described herein, in general, relates to training a model for detection of a Standard Operating Procedure (SOP) violation.
BACKGROUND
[003] Video surveillance is used for security purposes. In recent times, video surveillance technology has impacted hugely, because of the development in Computer Vision (CV) and Artificial Intelligence (AI). Usually, an Internet Protocol (IP) camera or a Closed-circuit television (CCTV) is used to capture a video. Further, the captured video is sent to a server for video surveillance. Currently, the analysis of the video is performed manually which is inefficient and time consuming. Thus, an efficient way for the analysis of the video is required.
SUMMARY
[004] Before the present system(s) and method(s), are described, it is to be understood that this application is not limited to the particular system(s), and methodologies described, as there can be multiple possible embodiments which are not expressly illustrated in the present disclosures. It is also to be understood that the terminology used in the description is for the purpose of describing the particular implementations or versions or embodiments only and is not intended to limit the scope of the present application. This summary is provided to introduce aspects related to a system and a method for training a model for detecting a Standard Operating Procedure (SOP) violation. This summary is not intended to identify essential features of the claimed subject matter nor is it intended for use in determining or limiting the scope of the claimed subject matter.
[005] In one embodiment, a method for training a model for detecting a Standard Operating Procedure (SOP) violation is disclosed. Initially, a media file may be received from a user. Subsequently, at least an object in the media file may be annotated. Further, a Standard Operating Procedure (SOP) for the object may be assigned. Furthermore, the media file may be augmented in runtime to obtain augmented data. Subsequently, a deep learning model may be trained with the augmented data to generate a baseline trained model. It may be noted that the baseline trained model may be generated after the training reaches a predefined threshold. Further, a decoded camera stream may be fed to the baseline trained model. Furthermore, at least an SOP compliance may be detected from the decoded camera stream when the object is under observation based on the assigned SOP. Finally, the baseline trained model may be recursively trained based on the SOP compliance. It may be noted that the baseline trained model is recursively trained using semi-supervised learning techniques. In one aspect, the aforementioned method for training a model for detecting an SOP violation may be performed by a processor using programmed instructions stored in a memory.
[006] In another embodiment, a non-transitory computer-readable medium embodying a program executable in a computing device for training a model for detecting a Standard Operating Procedure (SOP) violation is disclosed. The program may comprise a program code for receiving a media file from a user. Further, the program may comprise a program code for annotating at least an object in the media file. Subsequently, the program may comprise a program code for assigning a Standard Operating Procedure (SOP) for the object. Further, the program may comprise a program code for augmenting the media file in runtime to obtain augmented data. Furthermore, the program may comprise a program code for training a deep learning model with the augmented data to generate a baseline trained model. It may be noted that the baseline trained model is generated after the training reaches a predefined threshold. Further, the program may comprise a program code for feeding a decoded camera stream to the baseline trained model. Furthermore, the program may comprise a program code for detecting at least an SOP compliance from the decoded camera stream when the object is under observation based on the assigned SOP. Finally, the program may comprise a program code for recursively training the baseline trained model based on the SOP compliance. It may be noted that the baseline trained model is recursively trained using semi-supervised learning techniques.
BRIEF DESCRIPTION OF THE DRAWINGS
[007] The foregoing detailed description of embodiments is better understood when read in conjunction with the appended drawings. For the purpose of illustrating of the present subject matter, an example of a construction of the present subject matter is provided as figures, however, the invention is not limited to the specific method and system for training a model for detecting a Standard Operating Procedure (SOP) violation disclosed in the document and the figures.
[008] The present subject matter is described in detail with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to refer to various features of the present subject matter.
[009] Figure 1 illustrates a network implementation for training a model for detecting a Standard Operating Procedure (SOP) violation, in accordance with an embodiment of the present subject matter.
[010] Figure 2 illustrates a method for training a model for detecting a Standard Operating Procedure (SOP) violation, in accordance with an embodiment of the present subject matter.
[011] The figure depicts an embodiment of the present disclosure for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the disclosure described herein.

DETAILED DESCRIPTION
[012] Some embodiments of this disclosure, illustrating all its features, will now be discussed in detail. The words "receiving," "annotating," "assigning," "augmenting," "training," "feeding," "detecting," and other forms thereof, are intended to be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Although any system and methods similar or equivalent to those described herein can be used in the practice or testing of embodiments of the present disclosure, the exemplary system and methods are now described.
[013] The disclosed embodiments are merely examples of the disclosure, which may be embodied in various forms. Various modifications to the embodiment will be readily apparent to those skilled in the art and the generic principles herein may be applied to other embodiments. However, one of ordinary skill in the art will readily recognize that the present disclosure is not intended to be limited to the embodiments described but is to be accorded the widest scope consistent with the principles and features described herein.
[014] The present subject matter discloses a method and a system for training a model for detecting a Standard Operating Procedure (SOP) violation. The system receives a media file comprising an image or a video frame from a user. The user may annotate at least an object present in the media file. The user further assigns at least an SOP to the objects. It may be noted that the SOP may be a set of instructions for the object. Further, the system may augment the media files to obtain the augmented data. The augmented data may be created in runtime. It may be noted that the augmented data is not stored by the system. Further, a deep learning model may be trained based on the augmented data to obtain a baseline trained model.
[015] The objective of the invention is to create a model for detecting the object in a video stream in real-time. Further, a decoded camera stream may be fed to the baselined trained model. Furthermore, the system detects at least an SOP compliance from the decoded camera stream. The object may comply with the SOP or may violate the SOP. When the object violates the SOP, the system generates an alert. In an embodiment, the system also tracks the object in real-time in the video stream. For creating the model, the system may receive metadata related to a video stream. While aspects of described system and method for training a model for detecting a Standard Operating Procedure (SOP) violation may be implemented in any number of different computing systems, environments, and/or configurations, the embodiments are described in the context of the following exemplary system.
[016] Referring now to Figure 1, a network implementation 100 of a system 102 for training a model for detecting a Standard Operating Procedure (SOP) violation is disclosed. Initially, the system 102 receives a media file from a user. In an example, the software may be installed on a user device 104-1. It may be noted that the one or more users may access the system 102 through one or more user devices 104-2, 104-3…104-N, collectively referred to as user devices 104, hereinafter, or applications residing on the user devices 104. The system 102 receives a media file from one or more user devices 104. Further, the system may also 102 receive a feedback from a user using the user devices 104.
[017] Although the present disclosure is explained considering that the system 102 is implemented on a server, it may be understood that the system 102 may be implemented in a variety of computing systems, such as a laptop computer, a desktop computer, a notebook, a workstation, a virtual environment, a mainframe computer, a server, a network server, a cloud-based computing environment. It will be understood that the system 102 may be accessed by multiple users through one or more user devices 104-1, 104-2…104-N. In one implementation, the system 102 may comprise the cloud-based computing environment in which the user may operate individual computing systems configured to execute remotely located applications. Examples of the user devices 104 may include, but are not limited to, a portable computer, a personal digital assistant, a handheld device, and a workstation. The user devices 104 are communicatively coupled to the system 102 through a network 106.
[018] In one implementation, the network 106 may be a wireless network, a wired network, or a combination thereof. The network 106 can be implemented as one of the different types of networks, such as intranet, local area network (LAN), wide area network (WAN), the internet, and the like. The network 106 may either be a dedicated network or a shared network. The shared network represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), and the like, to communicate with one another. Further the network 106 may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, and the like.
[019] In one embodiment, the system 102 may include at least one processor 108, an input/output (I/O) interface 110, and a memory 112. The at least one processor 108 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, Central Processing Units (CPUs), state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the at least one processor 108 is configured to fetch and execute computer-readable instructions stored in the memory 112.
[020] The I/O interface 110 may include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like. The I/O interface 110 may allow the system 102 to interact with the user directly or through the client devices 104. Further, the I/O interface 110 may enable the system 102 to communicate with other computing devices, such as web servers and external data servers (not shown). The I/O interface 110 can facilitate multiple communications within a wide variety of networks and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. The I/O interface 110 may include one or more ports for connecting a number of devices to one another or to another server.
[021] The memory 112 may include any computer-readable medium or computer program product known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, Solid State Disks (SSD), optical disks, and magnetic tapes. The memory 112 may include routines, programs, objects, components, data structures, etc., which perform particular tasks or implement particular abstract data types. The memory 112 may include programs or coded instructions that supplement applications and functions of the system 102. In one embodiment, the memory 112, amongst other things, serves as a repository for storing data processed, received, and generated by one or more of the programs or the coded instructions.
[022] As there are various challenges observed in the existing art, the challenges necessitate the need to build the system 102 for training a model for detecting a Standard Operating Procedure (SOP) violation. At first, a user may use the user device 104 to access the system 102 via the I/O interface 110. The user may register the user devices 104 using the I/O interface 110 in order to use the system 102. In one aspect, the user may access the I/O interface 110 of the system 102. The detail functioning of the system 102 is described below with the help of figures.
[023] The present subject matter describes the system 102 for training a model for detecting an SOP violation. The system 102 may receive a media file from a user. The media file may comprise at least an image and a video frame. In an embodiment, the system may receive a video in real time from a user. The system may convert the video in a plurality of a video frame in real time.
[024] Further to receiving the media file, the user may annotate at least an object in the media file. In an example, the object may be present in the media file. Examples of the object may include, but not limited to, a mask, a hairnet, hand gloves, a helmet, an apron, and alike. In an embodiment, the user may also annotate a person. The user may provide the name of the person to the system 102. It may be noted that the user may annotate the media file by labelling or classifying the media file using at least a text, and annotation tools, to show the object or data features present in the media file.
[025] Further to annotating the media file, the user may assign a Standard Operating Procedure (SOP) for the object. The SOP may be an instruction for the object. It may be noted that the user is adding metadata to the media file by annotating and assigning the SOP for the object. The metadata may comprise at least an SOP assigned to the object, a name or a tag or a label associated to the object, any other information related to the media file such as location and time stamp. Consider an example, the system 102 receives a media file in real time from a user. It may be noted that the media file comprises one or more objects. Further, the user may annotate the media file by naming one or more objects. Assuming that, the objects are a glove and a mask. The user may assign an SOP for the objects (the glove and the mask). In the example, the SOP may be defined as ‘a person must be wearing the glove and the mask while working in a cooking area’.
[026] Further to assigning the SOP, the system 102 may augment the media file in runtime to obtain augmented data. The augmented data may comprise at least the media files and media files obtained by the augmentation. It may be noted that the augmentation is performed by applying at least one of a shear effect, a noise effect, a modification of contrast, a modification of brightness, a blur effect and axes wise flipping of the media file. Further, the augmentation may be performed based on Colour Translations, Spatial Translations and Adversarial Training algorithms. It may be noted that the size of the augmented data is generally high. Further, it must be noted that the system 102 does not store the augmented data locally or at the server (112).
[027] Consider the above example, the system 102 may receive the media file comprising images. Assuming the system 102 receives 100 images from the user. The objects present in the images may comprise a mask, a hairnet, hand gloves, a helmet, an apron, and alike. Further, the augmented data may comprise 1000 images. The system may apply at least one of a shear effect, a noise effect, a modification of contrast, a modification of brightness, a blur effect and axes wise flipping to the media file (100 images). It may be noted that the augmentation is performed in runtime and the augmented data is not stored in the system 102.
[028] Further to augmenting, the system 102 may train a deep learning model with the augmented data to generate a baseline trained model. It may be noted that the baseline trained model is generated after the training reaches a predefined threshold. Further, the predefined threshold may be defined by the user. It may be noted that the baseline trained model is obtained after training the deep learning model with the augment data.
[029] After training the model, the baseline trained model may be fed with a decoded camera stream. It is to be noted that the decoded camera stream is obtained from at least an IP camera, and a CCTV camera.
[030] Further to feeding, the system 102 may detect at least an SOP compliance from the decoded camera stream. The SOP compliance is detected when the object is under observation based on the assigned SOP. Further, the SOP compliance may indicate adhering to the assigned SOP. It may be noted that the object may follow the assigned SOP or may violate the assigned SOP. Further, the system may alert a user when the object violates the assigned SOP. In an embodiment, the system 102 may detect an SOP violation, when the object fails to follow the assigned SOP. It must be noted that the system 102, also detects the object that follows the assigned SOP. Consider an example, a plurality of cameras is installed at a place. The user may assign different SOP for different camera streams.
[031] Consider an example of a restaurant. A plurality of the IP cameras is installed in the restaurant. In the example, the system 102 receives the media file. Further, the user annotates the objects present in the media file. Let us assume that the annotated objects are a mask, a glove, and a hairnet. Further, the user assigns an SOP for each object. Let us assume that the SOP assigned for the mask is, “A person must wear the mask.” Further, the SOP assigned for the gloves is, “All the workers must wear the gloves”. Furthermore, the SOP assigned for the hairnet is, “Workers present inside the kitchen must wear the hairnet”. Subsequently, the system augments the media file to obtain the augmented data. Further, a deep learning model is trained with the augmented data to generate a baseline trained model. After the generation of the baseline trained model, the system is fed with a decoded camera stream. The system detects all the objects in the decoded camera stream in real time. In the example, the system also tracks all the objects in real time. Consider a situation, a worker present inside the kitchen removes the hairnet. In an embodiment, the system may alert a restaurant manager about the SOP violation as the hairnet is not present on the worker’s hair. In another embodiment, when the worker is not in the kitchen and the worker removes the hairnet, the system will not alert the restaurant manager. In addition, the system 102 may also highlight/indicate the person who violated the SOP. Similarly, when any worker is not wearing the gloves, the system generates an alert. In an embodiment, the system may also alert the restaurant manager when a customer who is not allowed to enter the kitchen, enters the kitchen.
[032] Further to detecting, the system may recursively train the baseline trained model based on the SOP compliance. It may be noted that the baseline trained model is recursively trained using semi-supervised learning techniques. In an embodiment, the system may receive feedback from the user on the detection of the SOP compliance. The user may validate the detection of the SOP compliance.
[033] Consider an example, the system 102 detects an SOP violation. When the SOP violation is not correct the user may provide a feedback to the system. It may be noted that the system is retrained based upon the user feedback. In an embodiment, the system also identifies at least age, gender, and name of a person present in the decoded camera stream. Further, the user may provide feedback for the identification of the age, gender, and name. It may be noted that the feedback improves the efficiency of the system 102.
[034] Referring now to figure 2, a method 200 for training a model for detecting a Standard Operating Procedure (SOP) violation is shown, in accordance with an embodiment of the present subject matter. The method 200 may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, functions, etc., that perform particular functions or implement particular abstract data types.
[035] The order in which the method 200 is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the method 200 or alternate methods for training a model for detecting an SOP violation. Additionally, individual blocks may be deleted from the method 200 without departing from the scope of the subject matter described herein. Furthermore, the method 200 for training a model for detecting an SOP violation can be implemented in any suitable hardware, software, firmware, or combination thereof. However, for ease of explanation, in the embodiments described below, the method 200 may be considered to be implemented in the above-described system 102.
[036] At block 202, a media file may be received from a user. The media file may comprise an image or a video frame.
[037] At block 204, at least an object in the media file may be annotated by the user.
[038] At block 206, a Standard Operating Procedure (SOP) for the object may be assigned.
[039] At block 208, the media file may be augmented in runtime to obtain augmented data.
[040] At block 210, a deep learning model may be trained with the augmented data to generate a baseline trained model. It may be noted that the baseline trained model may be generated after the training reaches a predefined threshold.
[041] At block 212, a decoded camera stream may be fed to the baseline trained model.
[042] At block 214, at least an SOP compliance may be detected from the decoded camera stream when the object is under observation based on the assigned SOP.
[043] At block 216, the baseline trained model may be recursively trained based on the SOP compliance. It may be noted that the baseline trained model is recursively trained using semi-supervised learning techniques.
[044] Exemplary embodiments discussed above may provide certain advantages. Though not required to practice aspects of the disclosure, these advantages may include those provided by the following features.
[045] Some embodiments of the system and the method enables to detect any object present in the media file.
[046] Some embodiments of the system and the method makes the video surveillance easier.
[047] Some embodiments of the system and the method enables to track the object in the decoded camera stream in real time.
[048] Some embodiments of the system and the method enables to build a deep learning model for detection of the object.
[049] Although implementations for methods and system for training a model for detecting a Standard Operating Procedure (SOP) violation have been described in language specific to structural features and/or methods, it is to be understood that the appended claims are not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as examples of implementations for training a model for detecting a Standard Operating Procedure (SOP) violation.