Description:BACKGROUND
Technical Field
[0001] The embodiments herein generally relate to Artificial Intelligence (AI)-based incident management and surveillance, more particularly to a system and method for pre-emptive detection ofan untoward incident in a surveillance area and rapid response for the untoward incident in real-time using edge-AI, especially one or more deep learning modelswith user defined rules.
Description of the Related Art
[0002] Surveillance systems play a crucial role in ensuring public safety and security by monitoring and recording activities in various areas. Numerous surveillance systems are currently available. However, they are facing several challenges that are hindering their effectiveness.
[0003] Existing surveillance systems mainly involve passive monitoring, tracking predefined incidents and reviewing recorded footage after occurrence of an untoward incident. A significant challenge with these systems is the massive volume of data they generate, overwhelming human operators who must sift through it all. That is, these systems typically rely on human monitoring, which can be time-consuming and prone to human error. For instance, the reported average of video cameras being monitored by each security team staff member is between 25 and 50 on a daily basis. According to a blog from 3S Security Systems, quoted by Security and IoT magazine asmag.com, research in the US has suggested that staff watching video systems can experience fatigue “in as little as 12 minutes, overlooking up to 45% of activity in the camera scenes. After 22 minutes, that increases up to 95% of overlooked activity. This results in many untoward incidents going unnoticed or not being acted upon in a timely manner, which can lead to serious consequences. Additionally, existing systems often lack intelligent analytics capabilities, making it difficult to detect and predict untoward incidents based on patterns or anomalies in the data. Hence, they also have limited capabilities for untoward incident detection and management.
[0004] Another major issue with the existing surveillance systems is their reliance on centralized data storage and processing. This can lead to bottlenecks and a single point of failure, where a failure in one part of the system can bring down the entire system. Furthermore, centralized storage makes it more difficult to secure the data, as there is a greater risk of unauthorized access or data tampering.
[0005] Another challenge is the difficulty in managing and maintaining surveillance systems, especially in large-scale implementations. Deploying and configuring the necessary hardware and software components can be a complex process, and system updates and upgrades can be time-consuming and disruptive. This makes it difficult to keep up with the latest technologies and security measures, leaving the system vulnerable to new threats and exploits.
[0006] Finally, existing surveillance systems often lack interoperability, making it difficult to integrate with other systems and share data. This makes it more difficult for law enforcement and emergency responders to coordinate their efforts and respond to the untoward incidents in a timely and effective manner.
[0007] In recent days, advanced video analytics systems have existed in the market. Some of these systems use advanced algorithms to analyze video footage, but they may not be able to detect certain types of untoward incidents or provide real time alerts. They may also require a large amount of computing power and storage capacity. Some of these systems use machine learning algorithms to analyze video footage, however, they may be limited by the quality of the data used to train the model and require regular updates and retraining.
[0008] Accordingly, there remains a need to address the aforementioned technical problems in existing surveillance systems in detecting untoward incidents effectively in real-time with improved privacy and security and reducing response time for such detected untoward incidents.
SUMMARY
[0009] In view of the foregoing, an embodiment herein provides a system for detecting an untoward incident pre-emptively in a surveillance area and enabling rapid response for the untoward incident in real-time using edge artificial intelligence (AI) with user defined rules. The system includes one or more edge devices that includes at least one of a camera, a microphone, or a sensor, an edge processor, and one or more deep learning models in the edge processor. The deep learning model associated with each edge device is configured to predict a specific type of incident based on a firstuser-defined rule. The system further includes a surveillance control server that is configured to obtain a mapping information to map strategically the one or more edge devices in the surveillance area based on a seconduser-defined rule. The mapping information is obtained using user input by (i) defining virtually one or more zones in the surveillance area, and (ii) determining a number of edge devices that is to be assigned with each zone to comprehensively cover the surveillance area based on an allocation technique. The user input includes coordinates information of the surveillance area and information of the one or more edge devices including at least one of coverage range, field of view or resolution of the camera. The mapping information includes allocation information of the one or more edge devices 102A-N to each zone. Each edge device is assigned with each zone based on the mapping information and is configured to (i) obtain, using at least one of the camera, the microphone, or the sensor, input data by monitoring the corresponding zone; (ii) extract, by the edge processor, a one or more frames from the input data associated with the corresponding zone using a frame extraction technique; and (iii) predict the specific type of incident in the corresponding zone by analyzing, using the deep learning model, the one or more frames associated with the corresponding zone. The surveillance control server is configured to (a) receive the specific type of incident in the corresponding zone from the deep learning model associated with each edge device; (b) determine, using an ensemble model, a final prediction by aggregating the specific type of incident from the deep learning model associated with each edge device; (c) detect an occurrence of the untoward incident in the corresponding zone based on the final prediction; and (d) activate, upon detecting the occurrence of the untoward incident in the corresponding zone, a signalling unit to notify a one or more users in the surveillance area. The signalling unit provides at least one of audio cues or video cues to notify the one or more users. Thereby, enabling the one or more users to respond rapidly to the untoward incident in the one or more zones of the surveillance area.
[0010] In some embodiments, the seconduser-defined rule for mapping the one or more edge devices includes zones of interest criteria based on security priorities, critical areas, critical infrastructure, usage patterns, or specific surveillance objectives.
[0011] In some embodiments, the surveillance control server performs the allocation techniquethat involves (i) assigning a score to each zone based on the seconduser-defined rule; (ii) determining zones that are within a surveillance range of the one or more edge devices by calculating coverage area of the one or more edge devices based on the information of the one or more edge devices including at least one of the coverage range, the field of view or the resolution of the camera, location and orientation of the one or more edge devices; (iii) defining an optimization objective based on the score to each zone, the coverage area of the one or more edge devices, and a combination of factors comprising maximizing coverage of high-priority zones, minimizing overlapping coverage, and ensuring efficient device utilization; (allocating, using an allocation strategy, the one or more edge devices to the one or more zones based on the optimization objective; (v) applying a fitness function on the allocation strategy to evaluate how well the current allocation of the one or more edge devices aligns with the optimization objective; and (vi) iterating a process of (iv) and (v) until obtaining the mapping information that is aligned with the seconduser defined rule.
[0012] In some embodiments, the surveillance control server enables the one or more users to define the firstuser-defined rule to predict the specific type of incident associated with a specific zone using a user interface. In some embodiments, the firstuser-defined rule is associated with detecting at least one of specific individuals, specific objects, and specific types of behaviour, zone-specific criteria, threshold-based parameters, or environmental factors.
[0013] In some embodiments, the one or more frames are analyzed to predict the specific type of incident by (a) pre-processing, using at least one pre-processing technique, the one or more frames to obtain a one or more pre-processed frames associated with the corresponding zone; (b) extracting, using a feature extraction method, one or more features from the one or more pre-processed frames associated with the corresponding zone; and inputting the one or more features to the deep learning model to predict the specific type of incident in the corresponding zone.In some embodiments, the one or more features are selected based on the firstuser-defined rule.
[0014] In some embodiments, the one or more deep learning modelsaretrained by associating historic features of labelled frames with corresponding historic incident labels. The historic features are selected based on the specific type of incident to be detected according to the firstuser-defined rule.
[0015] In some embodiments, the signalling unit is configured to provide untoward incident information. The untoward incident information includes at least one of a location of the untoward incident or a type of the untoward incident that is detected.
[0016] In some embodiments, the surveillance control server is configured to organize the input data that is detected with the occurrence of the untoward incident from each edge device in the one or more zones with a time stamp and store the input data with the time stamp on a distributed ledger for easy access and retrieval by authorized users.
[0017] In one aspect, a method of detecting an untoward incident pre-emptively in a surveillance area and enabling rapid response for the untoward incident in real-timeusing edge artificial intelligence (AI) with user defined rulesis provided. The method includes (i) providing one or more edge devices that includes at least one of a camera, a microphone, or a sensor, an edge processor, and one or more deep learning modelsin the edge processor. The deep learning model associated with each edge device is configured to predict a specific type of incident based on a firstuser-defined rule; (ii) obtaining, by a surveillance control server, a mapping information to map strategically the one or more edge devices in the surveillance area based on a seconduser-defined rule. The mapping information is obtained using user input by (a) defining virtually one or more zones in the surveillance area, and (b) determining a number of edge devices that is to be assigned with each zone to comprehensively cover the surveillance area based on an allocation technique. The user input includes coordinates information of the surveillance area and information of the one or more edge devices including at least one of coverage range, field of view or resolution of the camera. The mapping information includes allocation information of the one or more edge devices to each zone. Each edge device is assigned with each zone based on the mapping information; (iii) obtaining, by each edge device that is assigned with each zone, input data by monitoring the corresponding zone using at least one of the camera, the microphone, or the sensor; extracting, by each edge device, a one or more frames from the input data associated with the corresponding zone using a frame extraction technique with the edge processor; (iv) predicting, by each edge device, the specific type of incident in the corresponding zone by analyzing the one or more frames associated with the corresponding zone using the deep learning model; (v) receiving, by the surveillance control server, the specific type of incident in the corresponding zone from the deep learning model associated with each edge device; (vi) determining, by the surveillance control server, a final prediction by aggregating the specific type of incident from the deep learning model associated with each edge device using an ensemble model; (vii) detecting, by the surveillance control server, an occurrence of the untoward incident in the corresponding zone based on the final prediction; and(viii) activating, by the surveillance control server, a signalling unit to notify a one or more users in the surveillance area upon detecting the occurrence of the untoward incident in the corresponding zone. The signalling unit provides at least one of audio cues or video cues to notify the one or more users. Thereby, enabling the one or more users to respond rapidly to the untoward incident in the one or more zones of the surveillance area.
[0018] In some embodiments, the method includes organizing, by the surveillance control server, the input data that is detected with the occurrence of the untoward incident from each edge device in the one or more zones with a time stamp and storing the input data with the time stamp on a distributed ledger for easy access and retrieval by authorized users.
[0019] The system provides a powerful solution to detect various untoward incidents in a proactive manner by leveraging input data from different cameras in combination with intelligent zoning, concurrent deep learning models at the one or more edge devices and user-defined rules. By strategically zoning the cameras, the system logically assigns specific areas of coverage to each camera. This allows for comprehensive surveillance and monitoring of different zones, ensuring that no area is left unmonitored.
[0020] The system can detect untoward incidents in near real-time by analyzing input data on the one or more edge devices with the user-defined rules, allowing for a rapid response to prevent the untoward incidents from happening, thereby, significantly reducing a response time. Further, as the system allows the one or more users to define one or more rules for incident detection, i.e., the firstuser-defined rule based on specific needs, the system can be adapted to changing conditions or environments, thereby providing flexibility. Moreover, with the firstuser-defined rule, the accuracy of the system can be improved by allowing the one or more users to fine-tune the incident detection capabilities.
[0021] In the system, defining zones based on the seconduser-defined rule allows the system to focus on specific areas of interest, for example, areas with high foot traffic or sensitive areas that need to be secured. This can further improve the accuracy of incident detection by allowing the system to only detect incidents that occur within a specific area. Further, the system can be customized to the specific needs of the one or more users, for example, different rules for different areas, or different levels of security for different zones. As the system can be customized to target the areas of highest risk, the resources can be utilized more efficiently.
[0022] Moreover, as the deep learning model is running on the one or more edge devices, the system reduces a need for a high-bandwidth connection to a central server or cloud-based analytics. This can be particularly useful in remote locations or in situations where internet connectivity is limited. Moreover, as the system processes the input data locally in the one or more edge devices without sending it to the cloud or central server, the privacy of the data being collected is improved.
[0023] Further, as the input data that is detected with the untoward incident is organized and stored in the distributed ledger of a blockchain, the system makes it easier for the one or more users to review the input data (for example, footage and data related to specific incidents) quickly. This can be useful for incident investigations and for identifying patterns or trends. As the input data is stored in the blockchain, the input data is protected against tampering or unauthorized modification. Also, with the system, it is possible to recover intentionally deleted input data as the input data is stored using blockchain technology. Hence, the system provides immutable and tamper-proof records. With the blockchain technology, the system can efficiently store and manage large amounts of data, thereby reducing storage costs and resource utilization.
[0024] Further, as the signalling unit provides audio and visual cues, the attention of the one or more users is quickly taken, thereby allowing the one or more users to respond more quickly to incidents. Hence, the response time is further reduced. The audio and visual cues can make the system more accessible to users, especially users with visual impairments. Audio and visual cues can improve the overall user experience by providing a more intuitive way to interact with the system.
[0025] As, the system is designed to handle a high-volume of input data in real-time, the system can be useful for large venues and crowded areas. Moreover, the system can be integrated with other security solutions such as access control, intrusion detection, and fire alarms to create a comprehensive security system.
[0026] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.
BRIEF DESCRIPTION OF THE DRAWINGS
[0027] The embodiments herein will be better understood from the following detailed description with reference to the drawings, in which:
[0028] FIG. 1 illustrates a block diagram of a system 100 for detecting an untoward incident pre-emptively in a surveillance area using edge artificial intelligence (AI) with user-defined rules and enabling rapid response for the untoward incident in real-time according to some embodiments herein;
[0029] FIG.2 is a block diagram that illustrates a surveillance control server of FIG. 1, according to some embodiments herein;
[0030] FIG.3 is a block diagram that illustrates one or more edge devices of FIG. 1, according to some embodiments herein;
[0031] FIG. 4 illustrates an exemplary view of one or more zones of an office building that are allocated with one or more cameras by a mapping module of FIG. 2 for surveillance, according to some embodiments herein;
[0032] FIG. 5 illustrates an exemplary view of a surveillance system that is implemented for traffic management, according to some embodiments herein;
[0033] FIGS. 6A-6B illustrate a method for detecting an untoward incident pre-emptively in a surveillance area using edge artificial intelligence (AI) with user-defined rules and enabling rapid response for the untoward incident in real-time using a system 100 of FIG. 1 according to some embodiments herein.; and
[0034] FIG. 7illustrates a representative hardware environment for practicing the embodiments herein.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
[0035] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
[0036] As mentioned, there remains a need for detecting untoward incidents effectively in real-time with improved privacy and security and reducing response time for such detected untoward incidents. Various embodiments disclosed herein providea system and method for pre-emptive detection of untoward incidents in a surveillancearea and rapid response in real-time using edge-computing with one or more deep learning models. Referring now to the drawings, and more particularly to FIGS. 1 through7, where similar reference characters denote corresponding features consistently throughout the figures, preferred embodiments are shown.
[0037] FIG. 1 illustrates a block diagram of a system 100 for detecting an untoward incident pre-emptively in a surveillance area using edge artificial intelligence (AI) with user-defined rulesand enabling rapid response for the untoward incident in real-time according to some embodiments herein.The system 100 includes one or more edge devices 102A-N, a surveillance control server 104, one or more user devices 106A-N associated with one or more users, and a signalling unit 108 that are connected with each other using a data communication network 110.The data communication network 110 may be one or more of a wired network, a wireless network based on at least one of a 2G protocol, a 3G protocol, a 4G protocol, a 5G protocol, and a narrowband internet of things protocol (NBIoT), a combination of the wired network and the wireless network or the Internet. The system 100 may be implemented in the surveillance area to be monitored. The surveillance area may be any area of interest that is to be monitored. The surveillance area may be at least one of indoor space (for example, any residential or commercial buildings, vehicles, etc.) and outdoor space (for example, public spaces, transportation hubs, traffic management locations, etc.).
[0038] The one or more edge devices102A-N includesat least one of a camera, a microphone, a sensor, or any other devices that are utilized for capturing input data in the context of surveillance activities within the surveillance area. The input data may include at least one of audio data, sensor data or visual data. The one or more edge devices 102A-N further includes an edge processor, and one or more deep learning models112A-Nin the edge processor. The edge processor may perform computing tasks directly at the "edge" of a network, closer to the input data source or the point of input data generation using AI technology. AI technology may be a deep learning model.The deep learning model 112A associated with each edge device 102A is configured to predict a specific type of incident based on a firstuser-defined rule. The firstuser-defined rule may be associated with detecting at least one of specific individuals (for example, human or animal intrusion), specific objects (for example, suspicious items), specific types of behaviour (for example, unusual crowd behaviour, emergency situations like medical incidents, accidents, or falls), zone-specific criteria (network breaches, perimeter breaches, asset theft or vandalism), threshold-based parameters (head count estimation, traffic violations, fire detection), or environmental changes (like gas leaks, chemical spills, or changes in air quality).
[0039] The surveillance control server 104 is communicatively connected with the one or more edge devices 102A-N and includes a memory and a processor in communication with the memory. The processor of the surveillance control server 104is configured to obtain a mapping information to map strategically the one or more edge devices 102A-N in the surveillance area based on a seconduser-defined rule. The processor may obtain the mapping information using user input by (i) defining virtually one or more zones in the surveillance area, and (ii) determining a number of edge devices that is to be assigned with each zone to comprehensively cover the surveillance area based on an allocation technique. The user input includes coordinates information of the surveillance area and information of the one or more edge devices 102A-N including at least one of coverage range, field of view, or resolution of the camera. The mapping information includes allocation information of the one or more edge devices 102A-N to each zone. The seconduser-defined rule for mapping the one or more edge devices 102A-N includes zones of interest criteria based on security priorities, critical areas, critical infrastructure, usage patterns, or specific surveillance objectives. The one or more edge devices 102A-N may be assigned with each zone based on the mapping information. The surveillance control server 104 include, but are not limited to, a mobile device, a smartphone, a smart watch, a notebook, a Global Positioning System (GPS) device, a tablet, a desktop computer, a laptop, or any network enabled device.
[0040] Each edge device 102A that is assigned with each zone is configured to (i) obtain the input data by monitoring the corresponding zone using at least one of the camera, the microphone, or the sensor; (ii) extract one or more frames from the input data associated with the corresponding zone by the edge processor; and (iii) predict the specific type of incident in the corresponding zone from the one or more frames. The edge processor may use a frame extraction technique to extract the one or more frames. The edge processor further predicts the specific type of incident by analyzing the one or more frames associated with the corresponding zone using the deep learning model 112A.In some embodiments, the edge processor in each edge device 102A analyzes the one or more frames to predict the specific type of incident by (i) pre-processing the one or more frames to obtain one or more pre-processed frames associated with the corresponding zone, (ii) extracting one or more features from the one or more pre-processed frames associated with the corresponding zone based on the firstuser-defined rule; and (iii) inputting the one or more features to the deep learning model 112A to predict the specific type of incident in the corresponding zone. The edge processor may use at least one pre-processing technique to process the one or more frames. The edge processor may use a feature extraction method to extract the one or more features from the one or more pre-processed frames. The one or more features may be selected based on the firstuser-defined rule (i.e. the specific type of incident to be detected). In some embodiments, the one or more deep learning models112A-N aretrained by associating historic features of labelled frames with corresponding historic incident labels based on the firstuser-defined rule by the surveillance control server 104. That is, the historic features may be selected for training based on the specific type of incident to be detected according to the first user-defined rule.All deep learning models 112A-N in the one or more edge devices 102A-N concurrently performthe prediction of the specific type of incident in the corresponding zone.
[0041] In some embodiments, the one or more deep learning models 112A-N may be configured in (i) a cloud server, (ii) the one or more edge devices 102A-N, or (iii) both. The one or more deep learning models 112A-N may process the input data in the cloud server completely to predict the specific type of incident. In a hybrid implementation approach, some deep learning models may be configured in the cloud server and some deep learning models may be configured in the one or more edge devices 102A-N to process the input data for predicting the specific type of incident.
[0042] The surveillance control server 104 with the processor is further configured to receive the specific type of incident in the corresponding zone from the deep learning model 112A associated with each edge device 102A through the data communication network 110. The surveillance control server 104 further determines a final prediction by aggregating the specific type of incident from the deep learning model 112A associated with each edge device 102A. The surveillance control server 104 may use an ensemble model to determine the final prediction. The ensemble model may be a machine learning technique. The surveillance control server 104 is configured to detect an occurrence of the untoward incident in the corresponding zone based on the final prediction.
[0043] Upon detecting the occurrence of the untoward incident in the corresponding zone, the surveillance control server 104 activates the signalling unit 108 to notify one or more users in the surveillance area. The signalling unit may provide at least one of audio cues or video cues to the one or more users. Thereby, enabling the one or more users to respond rapidly to the untoward incident in the one or more zones of the surveillance area.The signalling unit 108 may be a video wall. The signalling unit 108 may be any device or means that provides the at least one of audio cues or video cues. The at least one of audio cues or video cues may be customized to the specific needs of the one or more users (or user preference). For example, different sounds may be used for different types of incidents or different colors may be used for different levels of severity.In some embodiments, the signalling unit 108 is configured to provide untoward incident information in addition to the audio cues or the video cues. The untoward incident information may include at least one of a location of the untoward incident or a type of the untoward incident that is detected.
[0044] In some embodiments, the surveillance control server 104 sends a notification to the one or more user devices 106A-N associated with the one or more users to notify the one or more users, upon detecting the occurrence of the untoward incident in the corresponding zone.
[0045] The surveillance control server 104 may provide a user interface for the one or more users to interact with the surveillance control server 104. The user interface may be a mobile application or a web application. The user interface through which the notification regarding the occurrence of the untoward incident is notified to the one or more users. The notification may be notified as push notifications, phone calls, SMS notifications, email notifications, or sound and vibration alerts. The one or more user devices 106A-N include but are not limited to, a mobile device, a smartphone, a smart watch, a notebook, a Global Positioning System (GPS) device, a tablet, a desktop computer, a laptop or any network enabled device.
[0046] In some embodiments, the user interface also enables the one or more users to define the first user-defined rule and the second user-defined rule; and to view the input data.
[0047] In some embodiments, the surveillance control server 104 is configured to organize the input data that is detected with the occurrence of the untoward incident from each edge device 102A in the one or more zones with a time stamp and store the input data with the time stamp on a distributed ledger for easy access and retrieval by authorized users.
[0048] FIG.2 is a block diagram that illustrates a surveillance control server 104 of FIG. 1, according to some embodiments herein. The surveillance control server 104 includes a memory 200 including a database 201, a processor 202 in communication with the memory 200, a receiving module 204, a mapping module 206, a training module 208, an aggregating module 210, an ensemble model 210A, an untoward incident detection module 212, an activating module 214, a notification module 216, and an organizing module 218.
[0049] The memory 200 stores the database 201 and a set of modules of the surveillance control server 104. The memory further stores a first user-defined rule and a second user-defined rule in the database 201. The first user-defined rule and the second user-defined rule may be received by the receiving module 204 from one or more users through one or moreuser devices 106A-N. In some embodiments, the surveillance control server 104 provides a user interface (not shown) that enables one or more users to define the first user-defined rule and the second user-defined rule using one or more user devices 106A-N.
[0050] The first user-defined rule includes criteria, conditions, and parameters that one or more deep learning models 112A-N associated with the one or more edge devices 102A-N use to classify and predict specific types of incidents. By applying the criteria, conditions, and parameters from the first user-defined rule to configure the one or more deep learning models 112A-N, the system ensures that predictions correspond to the specific type of incident intended for detection.
[0051] The seconduser-defined rule includes zones of interest criteria. The zones of interest criteria may refer to specific factors or conditions that are used to determine which areas within a surveillance area should be given special attention or priority. The seconduser-defined rule (or the zones of interest criteria) may help in defining one or more zones (or regions) that are of particular interest for surveillance, thereby facilitating the allocation and positioning of one or more edge devices 102A-N to ensure effective coverage. The zones of interest criteria may be based on security priorities, critical areas, critical infrastructure, usage patterns, or specific surveillance objectives.
[0052] The processor 202 executes the set of modules in the memory 200 for detecting an untoward incident pre-emptively in the surveillance area. The receiving module 204 receives user input from the one or more users through the one or more user devices 106A-N. The user input includes coordinates information of the surveillance area; and information about the available edge devices including coverage range of the camera, field of view of the camera, resolution of the camera, and other technical specifications. The user input may be inputted using the user interface.
[0053] The mapping module 206 is configured to map strategically the one or more edge devices 102A-N in the surveillance area based on the seconduser-defined rule. The mapping module 206 obtains mapping information using the user input. The mapping information is an allocation plan that includes allocation information of the one or more edge devices 102A-N to each zone. The mapping module 206 obtains the mapping information by (i) defining virtually one or more zones in the surveillance area based on the coordinates information of the surveillance area, and (ii) determining a number of edge devices that is to be assigned with each zone to comprehensively cover the surveillance area. The mapping module 206 may use an allocation technique to obtain the mapping information.
[0054] The allocation technique may (i) perform zone scoring by assigning a score or weight to each zone based on its alignment with theseconduser-defined rule; (ii) calculate coverage area of the one or more edge devices 102A-N to determine which zones are within the effective surveillance range of the one or more edge devices 102A-N by considering the information about the available edge devices and the location and orientation of the edge devices; (iii) define an optimization objective based on the combination of factors such as maximizing coverage of high-priority zones, minimizing overlapping coverage, and ensuring efficient device utilization; and (iv) employ an allocation strategy to allocate the one or more edge devices 102A-N to one or more zones based on the defined optimization objective. In some embodiments, the optimization objective may be defined by assigning numerical values or scores that represent the importance of achieving the goal. For instance, the goal of covering critical areas might be assigned a high score, while minimizing overlap could be assigned a lower score.
[0055] The allocation techniquefurther applies a fitness function on the allocation strategy to evaluate how well the current allocation of the one or more edge devices 102A-N aligns with the defined optimization objective. The allocation technique may use an iterative process to explore different allocation possibilities to improve the current allocation strategy. On the iterative process, the allocation technique may adjust the placement of the one or more edge devices 102A-N and recalculate the fitness function to assess how well the new allocation strategy meets the optimization objective. The algorithm continues iterating until it reaches a point where further improvements are minimal or until a predefined stopping criterion is met. The final output of the algorithm is the mapping information (i.e. the allocation plan) that aligns with the seconduser-defined rule and optimizes the allocation of the one or more edge devices 102A-N to each zone accordingly.
[0056] The training module 208is configured to train the one or more deep learning models 112A-N in the one or more edge devices 102A-N to predict a specific type of incident based on thefirstuser-defined rule. In some exemplary embodiments, the firstuser-defined rule is associated with detecting at least one of specific individuals (for example, human or animal intrusion), specific objects (for example, suspicious items), specific types of behaviour (for example, unusual crowd behaviour, emergency situations like medical incidents, accidents, or falls), zone-specific criteria (network breaches, perimeter breaches, asset theft or vandalism), threshold-based parameters (head count estimation, traffic violations, fire detection), or environmental changes (like gas leaks, chemical spills, or changes in air quality).
[0057] The training module 208 selects appropriate deep learning model architecture based on the nature of the incident type to be predicted.Further, the training module 208 collects historical datarelevant to the specific type of incident to be predictedbased on the firstuser-defined rule. The historical data may include image or video data, audio data or sensor data. The training module 208 extracts historic frames from the collected historical video data. The training module 208 may obtain the historic frames from the historic video data by breaking down the continuous stream of historic video data into individual images (frames). The historic video data may be decoded into a sequence of frames. During decoding, each historic frame may be extracted one by one from the historic video data. Each historic frame may be represented as image (with pixel values representing colors and intensity levels) captured at a specific time instant. The historic frames may be processed for at least one of resizing to a specific resolution or aspect ratio using interpolation techniques; color space conversion; noise reduction and filtering; contrast and brightness adjustment; or pixel value normalization.The training module 208 obtains historic labelled frames by performing annotation on the historic frames. In some embodiments, the training module 208 processes the historic audio data and the historic sensor data and obtains historic labelled audio data and historic labelled sensor data by annotation.
[0058] The training module 208 extractshistoric features (training data) relevant to the specific type of incident to be predicted from the at least one of historic labelled frames, historic labelled audio data, or historic labelled sensor datausing a feature extraction method. The historic features may include relevant patterns, attributes, or characteristics associated with the specific type of incident. The feature extraction method may be at least one of convolutional neural networks (CNNs), histogram of oriented gradients (HOG), optical flow, local binary patterns (LBP), motion history images (MHI), mel-frequency cepstral coefficients (MFCCs), spectral analysis techniques like short-time Fourier transform (STFT) or constant-Q transform (CQT), wavelet transform or any other related techniques. The feature extraction method may be selected based on the specific type of incident to be predicted (or the firstuser-defined rule). The training module 208 trains the one or more deep learning models112A-Nby associating the historic features of the labelled frames or data with corresponding historic incident labels.
[0059] The one or more edge devices 102A-N may be assigned with each zone based on the mapping information and the specific type of incident may be predicted concurrently in each zone from input data (real-time data of video or audio or sensor) with the one or more deep learning models112A-Nin the one or more edge devices 102A-N.
[0060] The receiving module 204 receives the specific type of incident that is predicted in each zone from the deep learning model 112A associated with each edge device 102A through the data communication network 110.
[0061] The aggregating module 210 aggregates the specific type of incidentsfrom the deep learning model 112Aassociated with each edge device 102A in each zone to determine a final prediction. The aggregating module 210 may use the ensemble model 210A to determine the final prediction. The ensemble model 210A may aggregate predictions from all the deep learning models 112A-N in each zone. The ensemble model 210A may perform aggregation using at least one technique of voting, averaging, weighted averaging, stacking or boosting.The final prediction may be a probability score, confidence value, or categorical label.The untoward incident detection module 212 detects an occurrence of the untoward incident in each zone based on the final prediction.The untoward incident detection module 212 may apply a decision logic to the final prediction to make a decision. The decision logic may include a set of rules, conditions, and criteria to decide whether the untoward incident is happening. The decision logic may be pre-defined and aligned with the firstuser-defined rule and the nature of the untoward incident being detected.
[0062] The activating module 214 activates a signalling unit 108 upon detecting the occurrence of the untoward incident in the one or more zones to notify one or more users in the surveillance area. The activating module 214 may control the signalling unit 108 to provide at least one of audio cues or video cues while notifying the one or more users. The activating module 214 is further configured to control the signalling unit 108 to provide untoward incident information in addition to the at least one of audio cues or the video cues. The untoward incident information may include at least one of a location of the untoward incident or a type of the untoward incident that is detected.
[0063] The notification module 216 sends a notification to the one or more user device 106A-N associated with the one or more users to notify the one or more users, upon detecting the occurrence of the untoward incident in the one or more zones. The notification may be notified as push notifications, phone call, SMS notifications, email notifications, or sound and vibration alert.
[0064] The organizing module 218 organizes the input data that is detected with the occurrence of the untoward incident from each edge device 102A in the one or more zones with a time stamp. The organizing module 218 further stores the input data with the time stamp on a distributed ledger for easy access and retrieval by authorized users.
[0065] FIG.3 is a block diagram that illustrates one or more edge devices 102A-N of FIG. 1, according to some embodiments herein.Each edge device 102A is assigned with one or more zones in a surveillance area based on mapping information. Each edge device 102A includes a device memory 300, an edge processor 302 in communication with the device memory 300, an input data capturing unit 304 including at least one of a camera 304A, a microphone 304B, or a sensor 304C, a frame extraction module 306, a pre-processing module 308, a feature extraction module 310, a specific type of incident prediction module 312, and one or more deep learning models 112A-N.The device memory 300 stores a set of modules of the edge processor 302. The edge processor 302 executes the set of modules in the device memory 300 for predicting a specific type of incident in the one or more zones of the surveillance area.
[0066] The input data capturing unit 304 monitors the corresponding zonein real-time and obtains input data using the at least one of the camera 304A, the microphone 304B, or the sensor 304C. The input data may include at least one of audio data, sensor data or visual data.
[0067] The frame extraction module 306 extracts one or more frames from the input data associated with the corresponding zone using a frame extraction technique.The frame extraction module 308 may obtain the one or more frames from the visual data by breaking down the continuous stream of the visual data into individual images. The frame extraction module 308 may decode the visual data into a sequence of frames. During decoding, each frame may be extracted one by one from the visual data. Each frame may be represented as an image with pixel values representing colors and intensity levels, captured at a specific time instant.
[0068] The pre-processing module 308 processes the one or more frames using at least one pre-processing technique to obtain one or more pre-processed frames. The one or more frames may be processed for at least one of resizing to a specific resolution or aspect ratio using interpolation techniques; color space conversion; noise reduction and filtering; contrast and brightness adjustment; or pixel value normalization.
[0069] The feature extraction module 310 extracts one or more features from the at least one of one or more pre-processed frames, audio data, orsensor data based on a firstuser-defined rule. The feature extraction module 310 may apply a feature extraction method to extract one or more features. The one or more features may be relevant to a specific type of incident to be predicted. The one or morefeatures may include relevant patterns, attributes, or characteristics associated with the specific type of incident. The feature extraction method may be at least one of convolutional neural networks (CNNs), histogram of oriented gradients (HOG), optical flow, local binary patterns (LBP), motion history images (MHI), mel-frequency cepstral coefficients (MFCCs), spectral analysis techniques like short-time Fourier transform (STFT) or constant-Q transform (CQT), wavelet transform or any other related techniques. The feature extraction method may be selected based on the specific type of incident to be predicted (or the firstuser-defined rule).
[0070] The specific type of incident prediction module 312 predicts the specific type of incident in the corresponding zone by analyzing the one or more frames associated with the corresponding zone. The specific type of incident prediction module 314 may use the one or more deep learning models 112A-N to predict the specific type of incident. The specific type of incident prediction module 314 may input the one or more features that are extracted from the one or more frames associated with the corresponding zone to the one or more deep learning models 112A-N to predict the specific type of incident in the corresponding zone. The one or more deep learning models 112A-N may be trained by the surveillance control server 104 by associating historic features of labelled frames with corresponding historic incident labels. The historic features may be selected based on the specific type of incident to be detected and aligned with the firstuser-defined rule.
[0071] In some embodiments, each specific type of incident prediction module 314 in the one or more edge devices 102A-N concurrently performs the prediction of the specific type of incident in the corresponding zone using the one or more deep learning models 112A-N. The predictions from the one or more edge devices 102A-N enable the surveillance control server 104 to detect an occurrence of an untoward incident in the corresponding zone.
[0072] FIG. 4 illustrates an exemplary view of one or more zones of an office building 402 that are allocated with one or more cameras 404A-C by a mapping module 206 of FIG. 2 for surveillance according to some embodiments herein. In some exemplary embodiments, the mapping module 206 obtains the mapping information for allocating the one or more cameras, for example camera A 404A, camera B 404B, camera C 404C in the office building 402 for surveillance. The mapping module 206 receives a second user defined rule that includes high priority areas that are to be given high attention such as server room 406B, main entrance 406A, and parking garage 406C. The mapping module 206 performs zoning based the second user defined rule. Zoning is a logical partition, where the mapping module 206 receivescoordinates information of the office building 402 and creates virtual perimeters using coordinates information of the office building402 to outline different areas on a digital map to define zone 1: main entrance 406A, zone 2: server room 406B, zone 3: parking garage 406C, zone 4: lobby 406D, zone 5: reception area 406E, and zone 6: conference room 406F. The mapping module 206 assigns scores for each zone (zones 1-6) based on the seconduser-defined rule as follows: main entrance 406A: 9; server room 406B: 8; parking garage 406C: 7; lobby 406D 5; reception area 406E: 4; and conference room 406F: 5.
[0073] In some exemplary embodiments, three cameras with different specifications are used such as camera A 404A that covers a 30-meter range with a 60-degree field of view (FOV) and is positioned at the main entrance 406A, facing outward; camera B 404B that covers a 20-meter range with a 90-degree FOV and is placed in a corner, facing diagonally across the server room 406B; and camera C 404C that covers a 15-meter range with a 120-degree FOV and is located centrally in the parking garage 406C.The mapping module 206 calculatesthe coverage area of the cameras A-C 404A-C using trigonometry, for example, as: a coverage area of the camera A: 471.24 square meters; a coverage area of the camera B: 314.16 square meters, and a coverage area of the camera C: 235.62 square meters.
[0074] The mapping module 206 further obtains mapping information to allocate the cameras A-C 404A-C to the zones 1-6 based on a combination of factors, including the importance of each zone (as indicated by scores) and the coverage capabilities (i.e. the coverage area) of each camera. Accordingly, the mapping module 206 allocates camera A 404A to a first high priority zone 1: the main entrance; camera B 404Bto a second priority zone 2: the server room 406B; and the camera C 404Cto next priority zone 3: parking garage 406C. Further, the camera A 404A can also be configured to cover zone 4: lobby and the camera B 404B can be configured to cover zone 5: reception area 406E and zone 6: conference room 406F as they have high coverage area.
[0075] In some embodiments, a single zone may be allocated with more than one camera. For example, zone 1: main entrance may be allocated with camera A 404A, and camera B 404B; and zone 2: server room may be allocated with camera B 404B, and camera C 404C based on the second user-defined rule.
[0076] FIG. 5 illustrates an exemplary view of a surveillance system 500 that is implemented for traffic management, according to some embodiments herein.In some exemplary embodiments, it is a scenario of vehicles or vehicle users needingto follow traffic rules for traffic management. To monitor such a scenario, the surveillance system 500 may be implemented. For example, a zone 1 508 is defined and is assigned with three edge devices,first edge device 502, second edge device 504, and third edge device 506, where each edge device is configured to predict a specific type of incident concurrently. A first deep learning model 502A, a second deep learning model 504A, and a third deep learning model 506A are running concurrently in three edge devices based on user-defined rules. Thefirst deep learning model 502A in the first edge device 502 is configured to predict whether there is a vehicular movement or not in the zone 1 508, Thesecond deep learning model 504Ain thesecond edge device 504 is configured to predict whether the vehicles are moving in allowed speed, and thethird deep learning model 506A in thethird edge device 506 is configured to predict whether the vehicles are following lane discipline or not. The binary predictions (Yes or No) from thefirst edge device 502, the second edge device 504, and the third edge device 506are received and are used combinedby the surveillance control server 104 to predict whether the users ofthe vehicles follow the traffic rules or not.
[0077] For example, the surveillance control server 104 receives the following predictions from the first deep learning model 502A, the second deep learning model 504A, and the third deep learning model 506A. The first deep learning model 502A predicts that there is a vehicular movement (Yes); the second deep learning model 504A predicts that a vehicle is not moving at an allowed speed (No); and the third deep learning model 506A predicts thatthe vehicle is not following lane discipline (No). The surveillance control server 104applies a voting method and counts the votes of predictions from the first deep learning model 502A, the second deep learning model 504A, and the third deep learning model 506Ato predict final prediction as follows: one (1) vote for ‘Yes’ and two (2) votes for ‘No’. Since, ‘No’ has the majority of votes, the final prediction by thesurveillance control server 104 is “No”, which means there is a traffic rule violation (i.e. an occurrence of an untoward incident). On predicting the traffic rule violation, thesurveillance control server 104 notifies one or more users to respond immediately against the traffic violation of the users of the vehicles.
[0078] In some exemplary embodiments, the zone 1 508 may be assigned with a single edge device that may include the first deep learning model 502A, the second deep learning model 504A and the third deep learning model 504Athat may work concurrently to predict (a) whether there is a vehicular movement, (b) whether the vehicles are moving in allowed speed, and (c) whether the vehicles are following lane discipline respectively for predicting the traffic rule violation.
[0079] In some exemplary embodiments, the specific type of incident may be any event occurred in the surveillance area. The specific type of incident may be varied for different zones. Examples for the specific type of incidents include headcount estimation, noise decibel level monitoring, and intruder detection. For instance, a headcount estimation model can analyze video feeds to determine the number of people present in a given area. This can be valuable for crowd management, ensuring compliance with occupancy limits, or identifying unusual crowd behaviour. Noise decibel level monitoring model can detect abnormal sound patterns, alerting security personnel to potential disturbances or dangerous situations. These models might run only in cameras atthe entry points. Furthermore, the intruder detection model identifies unauthorized individuals entering restricted zones by analyzing the video feed. This model may run only in cameras in the high security zones. In some exemplary embodiments, night vision cameras play a crucial role in incident detection, as night vision cameras provide enhanced visibility in low-light or dark conditions. Leveraging the capabilities of the night vision cameras, the system can effectively monitor areas that are typically more susceptible to criminal activities or incidents during nighttime hours.
[0080] FIGS. 6A-6B illustrate a method for detecting an untoward incident pre-emptively in a surveillance area using edge artificial intelligence (AI) with user-defined rules and enabling rapid response for the untoward incident in real-time using a system 100 of FIG. 1 according to some embodiments herein.At step 602, the method includes providing one or more edge devices 102A-N that includes at least one camera, microphone, or sensor, an edge processor, and one or more deep learning models 112A-N in the edge processor. The deep learning model 112A associated with each edge device 102A is configured to predict a specific type of incident based on a firstuser-defined rule.
[0081] At step 604, the method includes obtaining, by a surveillance control server 104, a mapping information to map strategically one or more edge devices 102A-N in the surveillance area based on a seconduser-defined rule. The mapping information may be obtained using user input by (i) defining virtually one or more zones in the surveillance area, and (ii) determining a number of edge devices that is to be assigned with each zone to comprehensively cover the surveillance area based on an allocation technique. The mapping information includes allocation information of the one or more edge devices 102A-N to each zone.The one or more edge devices 102A-N may be assigned with each zone based on the mapping information.
[0082] At step 606, the method includes obtaining, by each edge device 102A that is assigned with each zone, input data by monitoring the corresponding zone using at least one of a camera, a microphone, or a sensor.At step 608, the method includes extracting, by each edge device 102A, one or more frames from the input data associated with the corresponding zone usingan edge processor.At step 610, the method includes predicting, by each edge device 102A, the specific type of incident in the corresponding zone by analyzing the one or more frames associated with the corresponding zone using the deep learning model 112A based on the first user-defined rule.
[0083] At step 612, the method includes receiving, by the surveillance control server 104, the specific type of incident in the corresponding zone from the deep learning model 112A associated with each edge device 102A.At step 614, the method includes determining, by the surveillance control server 104, a final prediction by aggregating the specific type of incident from the deep learning model 112A associated with each edge device 102A using an ensemble model.At step 616, the method includes detecting, by the surveillance control server 104, an occurrence of the untoward incident in the corresponding zone based on the final prediction.
[0084] At step 618, the method includes activating, by the surveillance control server 104, a signalling unit to notify one or more users in the surveillance area upon detecting the occurrence of the untoward incident in the corresponding zone based on user preference. The signalling unit may be a video wall that provides at least one of audio cues or video cues to notify the one or more users. Thereby, enabling the one or more users to respond rapidly to the untoward incident in the one or more zones of the surveillance area.
[0085] FIG.7illustrates a representative hardware environment for practicing the embodiments herein. This schematic drawing illustrates a hardware configuration of an information handling/computer system in accordance with the embodiments herein. The system comprises at least one processor or central processing unit (CPU) 10. The CPUs 10 are interconnected via system bus 12 to various devices such as a random access memory (RAM) 14, read-only memory (ROM) 16, and an input/output (I/O) adapter 18. The I/O adapter 18 can connect to peripheral devices, such as disk units 11 and tape drives 13, or other program storage devices that are readable by the system. The system can read the inventive instructions on the program storage devices and follow these instructions to execute the methodology of the embodiments herein.
[0086] The system further includes a subject interface adapter 19 that connects a predetermined board 15, mouse 17, speaker 24, microphone 22, and/or other subject interface devices such as a touch screen device (not shown) or a remote control to the bus 12 to gather subject input. Additionally, a communication adapter 20 connects the bus 12 to a data processing network 25, and a display adapter 21 connects the bus 12 to a display device 23 which may be embodied as an output device such as a monitor, printer, or transmitter, for example.
[0087] The embodiments herein can take the form of, an entirely hardware embodiment, an entirely software embodiment or an embodiment including both hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. Furthermore, the embodiments herein can take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
[0088] The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk - read only memory (CD-ROM), compact disk - read/write (CD-R/W) and DVD.
[0089] A data processing system suitable for storing and/or executing program code will include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.
[0090] Input/output (I/O) devices (including but not limited to predetermined boards, displays, pointing devices, remote controls, etc.) can be coupled to the system either directly or through intervening I/O controllers. Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.
[0091] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation.Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the scope of appended claims. , Claims:I/We claim:
1. A system (100) for detecting an untoward incident pre-emptively in a surveillance area and enabling rapid response for the untoward incident in real-time using edge artificial intelligence (AI) with user-defined rules, wherein the system (100) comprising,
a plurality of edge devices (102A-N) that comprises at least one of a camera (304A), a microphone (304B), or a sensor (304C), an edge processor (302), and one or more deep learning models(112A-N) in the edge processor (302), wherein the at least one deep learning model (112A) associated with each edge device (102A) is configured to predict a specific type of incident based on a firstuser-defined rule; and
a surveillance control server (104) that is configured to obtain a mapping information to map strategically the plurality of edge devices (102A-N) in the surveillance area based on a seconduser-defined rule, wherein the mapping information is obtained using user input by (i) defining virtually one or more zones in the surveillance area, and (ii) determining a number of edge devices that is to be assigned with each zone to comprehensively cover the surveillance area based on an allocation technique, wherein the user input comprises coordinates information of the surveillance area and information of the plurality of edge devices (102A-N) including at least one of coverage range, field of view or resolution of the camera (304A), wherein the mapping information comprises allocation information of the plurality of edge devices (102A-N) to each zone, wherein each edge device (102A) is assigned with each zone based on the mapping information and is configured to:
obtain, using at least one of the camera (304A), the microphone (304B), or the sensor (304C), input data by monitoring the corresponding zone;
extract, by the edge processor (302), a plurality of frames from the input data associated with the corresponding zone using a frame extraction technique; and
predict the specific type of incident in the corresponding zone by analyzing, using the at least one deep learning model (112A), the plurality of frames associated with the corresponding zone; wherein the surveillance control server (104) is configured to:
receive the specific type of incident in the corresponding zone from the at least one deep learning model (112A) associated with each edge device (102A);
determine, using an ensemble model, a final prediction by aggregating the specific type of incident from the at least one deep learning model (112A) associated with each edge device (102A);
detect an occurrence of the untoward incident in the corresponding zone based on the final prediction; and
activate, upon detecting the occurrence of the untoward incident in the corresponding zone, a signalling unit (108) to notify a plurality of users in the surveillance area, wherein the signalling unit (108) provides at least one of audio cues or video cues to the plurality of users, thereby enabling the plurality of users to respond rapidly to the untoward incident in the one or more zones of the surveillance area.

2. The system (100) as claimed in claim 1, wherein the seconduser-defined rule for mapping the plurality of edge devices (102A-N) comprises zones of interest criteria based on security priorities, critical areas, critical infrastructure, usage patterns, or specific surveillance objectives.

3. The system (100) as claimed in claim 1, wherein the surveillance control server (104) performs the allocation techniquethat involves
(i) assigning a score to each zone based on the seconduser-defined rule;
(ii)determining zones that are within a surveillance range of the plurality of edge devices (102A-N) by calculating coverage area of the plurality of edge devices (102A-N) based on the information of the plurality of edge devices (102A-N) including at least one of the coverage range, the field of view or the resolution of the camera (304A), location and orientation of the plurality of edge devices (102A-N);
(iii) defining an optimization objective based on the score to each zone, the coverage area of the plurality of edge devices (102A-N),anda combination of factors comprising maximizing coverage of high-priority zones, minimizing overlapping coverage, and ensuring efficient device utilization;
(iv) allocating, usingan allocation strategy, the plurality of edge devices (102A-N) to the one or more zones based on the optimization objective;
(v) applying a fitness function on the allocation strategy to evaluate how well the current allocation of the plurality of edge devices (102A-N) aligns with the optimization objective; and
(vi) iterating a process of (iv) and (v) until obtaining the mapping information that is aligned with the seconduser defined rule.

4. The system (100) as claimed in claim 1, wherein the surveillance control server (104) enables the plurality of users to define the firstuser-defined rule to predict the specific type of incident associated with a specific zone using a user interface, wherein the firstuser-defined rule is associated with detecting at least one of specific individuals, specific objects, specific types of behaviour, zone-specific criteria, threshold-based parameters, or environmental factors.
5. The system (100) as claimed in claim 1, wherein the plurality of frames are analyzed to predict the specific type of incident by
pre-processing, using at least one pre-processing technique, the plurality of frames to obtain a plurality of pre-processed frames associated with the corresponding zone;
extracting, using a feature extraction method, one or more features from the plurality of pre-processed frames associated with the corresponding zone, wherein the one or more features are selected based on the firstuser-defined rule; and
inputting the one or more features to the at least one deep learning model (112A) to predict the specific type of incident in the corresponding zone.

6. The system (100) as claimed in claim 5, wherein the one or more deep learning models(112A-N) aretrained by associating historic features of labelled frames with corresponding historic incident labels, wherein the historic features are selected based on the specific type of incident to be detected according to the firstuser-defined rule.

7. The system (100) as claimed in claim 1, wherein the signalling unit (108) is configured to provide untoward incident information, wherein the untoward incident information comprises at least one of a location of the untoward incident or a type of the untoward incident that is detected.

8. The system (100) as claimed in claim 1, wherein the surveillance control server (104) is configured to organize the input data that is detected with the occurrence of the untoward incident from each edge device (102A) in the one or more zones with a time stamp and store the input data with the time stamp on a distributed ledger for easy access and retrieval by authorized users.
9. A method of detecting an untoward incident pre-emptively in a surveillance area and enabling rapid response for the untoward incident in real-timeusing edgeartificial intelligence (AI) with user defined rules, wherein the method comprising,
providing a plurality of edge devices (102A-N) that comprises at least one of a camera (304A), a microphone (304B), or a sensor (304C), an edge processor (302), and one or more deep learning models (112A-N) in the edge processor (302), wherein the at least one deep learning model (112A) associated with each edge device (102A) is configured to predict a specific type of incident based on a firstuser-defined rule;
obtaining, by a surveillance control server (104), a mapping information to map strategically the plurality of edge devices in (102A-N) in the surveillance area based on a seconduser-defined rule, wherein the mapping information is obtained using user input by (i) defining virtually one or more zones in the surveillance area, and (ii) determining a number of edge devices that is to be assigned with each zone to comprehensively cover the surveillance area based on an allocation technique, wherein the user input comprises coordinates information of the surveillance area and information of the plurality of edge devices (102A-N) including at least one of coverage range, field of view or resolution of the camera (304A), wherein the mapping information comprises allocation information of the plurality of edge devices (102A-N) to each zone, wherein each edge device (102A) is assigned with each zone based on the mapping information;
obtaining, by each edge device (102A) that is assigned with each zone, input data by monitoring the corresponding zone using at least one of the camera (304A), the microphone (304B), or the sensor (304C);
extracting, by each edge device (102A), a plurality of frames from the input data associated with the corresponding zone using a frame extraction technique with the edge processor (302);
predicting, by each edge device (102A), the specific type of incident in the corresponding zone by analyzing the plurality of frames associated with the corresponding zone using at least one deep learning model (112A);
receiving, by the surveillance control server (104), the specific type of incident in the corresponding zone from the at least one deep learning model (112A) associated with each edge device (102A);
determining, by the surveillance control server (104), a final prediction by aggregating the specific type of incident from the at least one deep learning model (112A) associated with each edge device (102A) using an ensemble model;
detecting, by the surveillance control server (104), an occurrence of the untoward incident in the corresponding zone based on the final prediction; and
activating, by the surveillance control server (104), a signalling unit (108) to notify a plurality of users in the surveillance area upon detecting the occurrence of the untoward incident in the corresponding zone, wherein the signalling unit (108) provides at least one of audio cues or video cues to notify the plurality of users, thereby enabling the plurality of users to respond rapidly to the untoward incident in the one or more zones of the surveillance area.

10. The method as claimed in claim 9, wherein the method comprising organizing, by the surveillance control server (104), the input data that is detected with the occurrence of the untoward incident from each edge device (102A) in the one or more zones with a time stamp and storing the input data with the time stamp on a distributed ledger for easy access and retrieval by authorized users.

Dated this 20th September 2023

Arjun Karthik Bala
(IN/PA 1021) Agent for Applicant