Claims:We claim:
1. A system for providing video enabled decisions and alerts at a workspace, the system comprising:
a set of cameras placed at various locations of the workspace for capturing a set of video streams of the workspace;
a stream parser configured to receive the set of video streams and generate one or more input images from the set of video streams;
a model inference unit configured to detect one or more objects and actions in the one or more input images using one or more machine learning algorithms, and generating corresponding one or more output images indicating detection of one or more objects;
a database configured to store the one or more input and output images;
a business logic unit configured to aggregate one or more output of the model inference unit for one or more predefined incidents, to generate one or more safety violations therein;
a reporting unit configured to generate real-time audio, video, and message alerts based on the one or more safety violations; and
a user interface configured to provide a dashboard to display a real-time aggregated output and images to highlight the safety violations.
2. The system of claim 1, wherein the camera is selected from at least one of: a Closed-circuit Television (CCTV) camera and a drone.
3. The system of claim 1, wherein the model inference unit use computer vision libraries and customized machine learning algorithms to detect and characterize objects.

4. The system of claim 1, wherein the model inference unit is configured to detect objects in the input images for safety gear detection of workers, intrusion detection for security, fire detection, vehicle detection, face mask detection, detecting whether people adhering to social distancing, facial recognition based attendance tracking and quality monitoring.

5. The system of claim 1, wherein the dashboard is configured to provide violation trends, snapshots of the violations, and comparison across various application areas.

6. A method for providing video enabled decisions and alerts at a workspace, the method comprising:
capturing a set of video streams of the workspace;
receiving the set of video streams and generating one or more input images from the set of video streams;
detecting one or more objects in the one or more input images using one or more machine learning algorithms, and generating corresponding one or more output images indicating detection of one or more objects;
storing the one or more input and output images;
aggregating one or more output of the model inference unit for one or more predefined incidents based on configuration of one or more business rules, to generate one or more safety violations therein;
generating real-time audio, video, and message alerts based on the one or more safety violations; and
providing a dashboard to display a real-time aggregated output and images to highlight the safety violations.
7. The method of claim 6, wherein the camera is selected from at least one of: a Closed-circuit Television (CCTV) camera and a drone.

8. The method of claim 6 comprising using computer vision libraries and customized machine learning algorithms to detect and characterize objects.

9. The method of claim 6 comprising detecting objects in the input images for at least one of: safety gear detection of workers, intrusion detection for security, fire detection, vehicle detection, face mask detection, detecting whether people adhering to social distancing, facial recognition based attendance tracking and quality monitoring.

10. The method of claim 6, wherein the dashboard is configured to provide violation trends, snapshots of the violations, and comparison across various application areas.
Dated this 18th Day of September 2020
-Digitally Signed-
M. Kisoth
IN/PA-2259
Agent for the applicant
, Description:SYSTEM AND METHOD FOR PROVIDING VIDEO ENABLED DECISIONS
TECHNICAL FIELD
[0001] The present disclosure relates generally to a video analytics platform, and more specifically to a video analytics platform with real time alerting on safety violations.
BACKGROUND
[0002] Closed-circuit Television (CCTV) video surveillance is commonly used within manufacturing plants to identify issues related to safety and compliance. However, usage of such systems for identifying compliance issues are manual and have low accuracy due to difficulty in maintaining focus for long durations. Further, such systems are labour intensive due to extended supervisor time spent on video analysis at each plant. Also, in some systems, video playback is analysed later, making actions taken less effective. Also, there are variations within safety rules from different plants requiring an understanding of the process detail and differences in practises across firms.
[0003] Also, existing video analytics works primarily on individual image frames within a video stream to identify specific safety violations. However, they have to be processed further and consolidated to be presented on a dashboard. Also, the system need to be able to identify and drop one off detections that could be false positives due to various conditions. The actions to be carried out based on the automated detections need to be carried out in real time while the reporting would have to be executed asynchronously over a period in order to ensure accuracy of incidents trends and timelines.
[0004] In addition, each individual models and the business logic would need to have configurable inputs from administrators such as the region of interest, distance mapping on ground, batch sizes, frames per second, window of false positives and window of alert triggers. Specific timelines and validities of each output from every module needs to be checked at every stage to manage real time alerts while consolidating these asynchronously for accurate trends on safety incidents.

SUMMARY
[0005] In an aspect of the present disclosure, there is provided a system for providing video enabled decisions and alerts at a workspace. The system includes a set of cameras placed at various locations of the workspace for capturing a set of video streams of the workspace, a stream parser configured to receive the set of video streams and generate one or more input images from the set of video streams, a model inference unit configured to detect one or more objects in the one or more input images using one or more machine learning algorithms, and generating corresponding one or more output images indicating detection of one or more objects, a database configured to store the one or more input and output images, a business logic unit configured to aggregate one or more output of the model inference unit for one or more predefined incidents, to generate one or more safety violations therein, a reporting unit configured to generate real-time audio, video, and message alerts based on the one or more safety violations, and a user interface configured to provide a dashboard to display a real-time aggregated output and images to highlight the safety violations.
[0006] In another aspect of the present disclosure, there is provided a method for providing video enabled decisions and alerts at a workspace. The method includes capturing a set of video streams of the workspace, receiving the set of video streams and generating one or more input images from the set of video streams, detecting one or more objects in the one or more input images using one or more machine learning algorithms, and generating corresponding one or more output images indicating detection of one or more objects, storing the one or more input and output images, aggregating one or more output of the model inference unit for one or more predefined incidents, to generate one or more safety violations therein, generating real-time audio, video, and message alerts based on the one or more safety violations, and providing a dashboard to display a real-time aggregated output and images to highlight the safety violations.
[0007] Various embodiments of the present disclosure provide a video enabled decisions and alerting (VEDA) platform that may deploy custom built Artificial Intelligence (AI)AI/Machine Learning (ML) models for analyzing videos and providing customized solutions for solving business problems. The VEDA platform may be used for multiple use-cases and to create products for benefits such as safety, automation, and reliable operations. The implementation of video analytics may aid transformation in a business environment by increasing process efficiencies, understanding customer behavior and enhancing safety in diverse environments while reducing considerably the manual intervention required. The VEDA platform is scalable and built for quick deployment to process multiple streams in parallel and apply advanced video analytics in real time. The VEDA platform is built on the model output through customizable business logic and push notifications/reports/dashboard views asynchronously. The VEDA platform has the ability to spot a much broader array of potential safety infringements and then generate automatic alerts to the company’s safety system. The AI enabled platform may detect potential infringements with a greater accuracy and consistency thereby reducing the number of high-risk events. This enables targeting safety awareness and process monitoring initiatives far more effectively, using data-driven insights from actual operations.
[0008] It will be appreciated that features of the present disclosure are susceptible to being combined in various combinations without departing from the scope of the present disclosure as defined by the appended claims.
BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The summary above, as well as the following detailed description of illustrative embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the present disclosure, exemplary constructions of the disclosure are shown in the drawings. However, the present disclosure is not limited to specific methods and instrumentalities disclosed herein. Moreover, those in the art will understand that the drawings are not to scale. Wherever possible, like elements have been indicated by identical numbers.
[0010] Embodiments of the present disclosure will now be described, by way of example only, with reference to the following figures wherein:
[0011] FIG. 1 illustrates an environment wherein various embodiments of the present invention can be practiced;
[0012] FIG. 2 illustrates a Video enabled decisions and alerts (VEDA) platform for controlling and managing the operations in the environment of FIG. 1, in accordance with an embodiment of the present disclosure;
[0013] FIG. 3 illustrates an exemplary input image received by the model inference unit, and an exemplary output image generated by the model inference unit, in accordance with an embodiment of the present disclosure;
[0014] FIG. 4 illustrates various use cases pertaining to the VEDA platform of FIG. 2, in accordance with an embodiment of the present disclosure;
[0015] FIG. 5 illustrates a dashboard provided on the user interface of FIG. 2 for monitoring incidents in the environment of FIG. 1, in accordance with an embodiment of the present disclosure; and
[0016] FIG. 6 is a flowchart illustrating a method for providing video enabled decisions and alerts in a workspace, in accordance with an embodiment of the present disclosure.
[0017] In the accompanying drawings, an underlined number is employed to represent an item over which the underlined number is positioned or an item to which the underlined number is adjacent. A non-underlined number relates to an item identified by a line linking the non-underlined number to the item. When a number is non-underlined and accompanied by an associated arrow, the non-underlined number is used to identify a general item at which the arrow is pointing.
DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS
[0018] The following detailed description illustrates embodiments of the present disclosure and ways in which they can be implemented. Although the best mode of carrying out the present disclosure has been disclosed, those skilled in the art would recognize that other embodiments for carrying out or practicing the present disclosure are also possible.
[0019] FIG. 1 illustrates an environment 100, wherein various embodiments of the present invention can be practiced. The environment 100 illustrates a sample workspace setting with placements of cameras 102a till 102e at various locations such as main gate, canteen, shop floor, office space, and storage. Examples of the cameras 102a till 102e include, but are not limited to Closed-circuit Television (CCTV) cameras. The deployment of cameras 102a till 102e across industrial plants and workplaces may vary depending on the automation of the processes and the count of workforce. The cameras 102a till 102e may be placed for periphery monitoring as well as for workplace surveillance for deterrence and any post incident review.
[0020] FIG. 2 illustrates a Video enabled decisions and alerts (VEDA) platform 200 for controlling and managing the operations in the environment 100, in accordance with an embodiment of the present disclosure.
[0021] The VEDA platform 200 includes a stream parser 202 that is configured to receive multiple incoming video streams from the cameras 102a till 102e (for e.g. FIG. 1) and split multiple incoming video streams based on specific configurable parameters into individual images.
[0022] The VEDA platform 100 further includes a model inference unit 204 that is configured to receive an input image from the stream parser 202, and detect and characterize objects in the input image. In the context of the present disclosure, the model inference unit 204 use computer vision libraries stored in a file system, and customize machine learning algorithms to detect and characterize objects. The model inference unit 204 may be configured to crop images, and analyze examples of safe behavior so as to detect and categorize objects pertaining to safe behavior. It may be noted that the model inference unit 204 is configured to iteratively use training information to detect objects, and manually validate the results through computer vision libraries.
[0023] In an embodiment of the present disclosure, the model inference unit 204 is configured to detect objects based on the application area in which it is used. The model inference unit 204 is highly configurable and provides simple UI to a define cases to be executed on specific cameras, set required frame rate (FPS), mark area of focus on the camera view in certain cases, specify feature specific parameters based on video feed, and provide time schedule in which the analysis should be carried out.
[0024] In an example, for the purpose of safety gear detection of workers in the environment 100, the model inference unit 204 is configured to analyze images to detect the safety gear worn by the workers. In another example, for the purpose of intrusion detection for security in the environment 100, the model inference unit 204 is configured to analyze images to detect presence of objects/human beings at one or more entry gates of the environment 100. In yet another example, for the purpose of fire detection in the environment 100, the model inference unit 204 is configured to analyze images to detect any fire images therein. In yet another example, for the purpose of vehicle detection in the environment 100, the model inference unit 204 is configured to analyze images to detect one or more type of vehicles. In yet another example, for the purpose of face mask detection in the environment 100, the model inference unit 204 is configured to analyze images to detect faces and face masks thereon. In yet another example, the model inference unit 204 is configured to analyze images to detect whether the people are adhering to social distancing. In yet another example, the model inference unit 204 is configured to analyze images to perform facial recognition based attendance tracking. In addition to the above-mentioned examples, the model inference unit 204 may be configured to track temperature values from a thermal camera, and provide same within a dashboard or provide alerts based on configured thresholds.
[0025] FIG. 3 illustrates an exemplary input image 302 received by the model inference unit 204, and an exemplary output image 304 generated by the model inference unit 204, in accordance with an embodiment of the present disclosure. The output image 304 illustrates the detection of a human image, and an area around the human image.
[0026] Referring back to FIG. 2, the model inference unit 204 is configured to store one or more output results in a database 206. The database 206 is configured to maintain a record of all images as highlighted by the model inferences.
[0027] The database 206 is coupled to a user interface 208, a business logic unit 210, and a reporting unit 212 through one or more hardware/software interfaces 214. The user interface 208 is configured to display one or more image outputs generated by the model inference unit 204. The business logic unit 210 is configured to carry out business specific transformations on the model output which are then pushed for storage on the database 206. The business logic unit 210 is configured to carry out aggregation of the model output for specific incidents in addition to the individual image output. The business logic unit 210 is further configured to drop potential false positives from the model output.
[0028] The reporting unit 212 is configured to take inputs from the model response to identify events and channel for playing a real-time alert. The reporting unit 212 includes an alerting engine 213 for generating alerts, an on-site delivery unit 214 for generating real-time audio alerts specific to the violation, and an email/SMS delivery unit 215 for generating email/SMS alerts. The alerting engine 213 may be configured to capture the necessary metadata in order to report exceptions. The on-site delivery unit 214 may be configured to raise real-time alerts on multiple channels including audio output to speakers at the location.
[0029] FIG. 4 illustrates various use cases 402, and corresponding analytics features 404, business logics 406, and actions 408 pertaining to the VEDA platform 200, in accordance with an embodiment of the present disclosure.
[0030] In a first use case – detection of PPE non-compliance, the corresponding model inference unit is configured to perform person detection, and PPE detection. The corresponding business logic unit is configured to aggregate detection of persons, and drop x frame violations based on region of interest. The corresponding reporting system is configured to generate real-time voice alerts, and capture the same on a dashboard on a user interface.
[0031] In a second use case – detection of incorrect vehicle, the corresponding model inference unit is configured to perform vehicle detection. The corresponding business logic unit is configured to aggregate same violation detection and estimate time of vehicle parking. The corresponding reporting system is configured to generate real-time voice alerts, and capture the same on a dashboard on a user interface.
[0032] Although two use cases have been illustrated herein, it would be apparent to one of ordinary skill in the art, that more than two use cases are possible by configuring the VEDA platform 200 accordingly.
[0033] FIG. 5 illustrates a dashboard 500 provided on the user interface 208 for monitoring incidents in the environment 100, in accordance with an embodiment of the present disclosure.
[0034] The dashboard 500 provide aggregated incident reports and trends along with specific images to highlight safety violations. In an embodiment of the present disclosure, the dashboard 500 is configured to provide violation trends, snapshots of the violation, comparison across various areas, and export feature to excel. The dashboard 500 also provide an option to categorize features based on various filters such as time period, areas/cameras, and type of violation.
[0035] Thus, the implementation of VEDA platform 200 may aid transformation and increase in process efficiencies by automated analysis and surveillance in various application areas. Also, the retailers may use video surveillance for strolling recognition, footfall measurement, and behavior analysis for following:
? Customer journey mapping
? Personalized shopping experience
? Store optimization
? Footfall tracking
? Customer segmentation
? Retail applications
[0036] FIG. 6 is a flowchart illustrating a method 600 for providing video enabled decisions and alerts in a workspace, in accordance with an embodiment of the present disclosure.
[0037] At step 602, a set of video streams of the workspace are captured.
[0038] At step 604, the set of video streams are received and one or more input images are generated from the set of video streams.
[0039] At step 606, one or more objects are detected in the one or more input images using one or more machine learning algorithms, and corresponding one or more output images are generated indicating detection of one or more objects;
[0040] At step 608, the one or more input and output images are stored in a database.
[0041] At step 610, one or more output of the model inference unit is aggregated for one or more predefined incidents, to generate one or more safety violations therein.
[0042] At step 612, real-time audio, video, and message alerts are generated based on the one or more safety violations.
[0043] At step 614, a dashboard is provided to display a real-time aggregated output and images to highlight the safety violations.
[0044] Modifications to embodiments of the present disclosure described in the foregoing are possible without departing from the scope of the present disclosure as defined by the accompanying claims. Expressions such as “including”, “comprising”, “incorporating”, “consisting of”, “have”, “is” used to describe and claim the present disclosure are intended to be construed in a non-exclusive manner, namely allowing for items, components or elements not explicitly described also to be present. Reference to the singular is also to be construed to relate to the plural.