Patent information
IP India · live recordAbstract
The disclosed method (600) and system (102) for managing one or more policies for anomaly detection in real-time data. The system (102) is configured for receiving a policy creation request from a user interface (206), wherein the policy creation request includes one or more user-defined parameters. The system (102) is configured for creating a policy based on the user-defined parameters received in the policy creation request. The system (102) is configured for validating the created policy and storing the created policy in a distributed data lake (210) upon the validation. The system (102) is configured for scheduling the created policy retrieved from distributed data lake (210) for execution at a predefined schedule. The system (102) is configured for determining dynamic policy thresholds for each data profile associated with the created policy. The system (102) is configured for executing the created policy based on the determined dynamic policy thresholds for anomaly detection. FIGURE 4
Applicants
1Inventors
24Claims
17Specification
Show full textCollapse
FORM 2 THE PATENTS ACT, 1970 (39 of 1970) THE PATENTS RULES, 2003 COMPLETE SPECIFICATION (See section 10; rule 13) TITLE OF THE INVENTION SYSTEM AND METHOD TO MANAGE ONE OR MORE POLICIES FOR ANOMALY DETECTION IN REAL-TIME DATA APPLICANT JIO PLATFORMS LIMITED of Office-101, Saffron, Nr. Centre Point, Panchwati 5 Rasta, Ambawadi, Ahmedabad - 380006, Gujarat, India; Nationality : India The following specification particularly describes the invention and the manner in which it is to be performed
RESERVATION OF RIGHTS
[001] A portion of the disclosure of this patent document contains material, which is subject to intellectual property rights such as, but are not limited to, copyright, design, trademark, Integrated Circuit (IC) layout design, and/or trade 5 dress protection, belonging to Jio Platforms Limited (JPL) or its affiliates (hereinafter referred as owner). The owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all rights whatsoever. All rights to such intellectual property are fully 10 reserved by the owner. TECHNICAL FIELD
[002] The present disclosure relates to wireless cellular communications, and specifically to a system and a method for managing one or more policies for anomaly detection in real-time data. 15 DEFINITION
[003] As used in the present disclosure, the following terms are generally intended to have the meaning as set forth below, except to the extent that the context in which they are used to indicate otherwise.
[004] “User Profiles” refers to a collection of information associated with 20 a particular user. A user profile can be defined as the explicit digital representation of the identity of the user, with respect to different parameters such as policies or rules to monitor the particular use case and thresholds associated with respective policies. The user profile is dependent on user and the use case. For a given user, the profile can be created based on what kind of data that user visualizes. 25 [005] “Use cases” refer to methodologies and processes used in network deployment, mobility, software development, product design, and other fields to describe how a system can be used to achieve specific goals or tasks. For example, 2
the different use cases may be in network usage, autonomous vehicles, fixed wireless technology etc. In an example, the use cases may be associated with monitoring minutes of usage (MOU), attempted calls, answered calls for a pre¬ defined duration and aggregating the plurality of profiles based on each circle and 5 each quarter. There can be various key performance indicators (KPIs) which are linked/attached to different use cases.
[006] “Policy” refers to a statement of intent and is implemented as a procedure or protocol. Policy Control Function in 5G networks is a key component that enables efficient policy control and management, facilitating network behavior 10 control, network slicing, UE activities, and communication with other 5G core network functions. The policy is a combination of one or more use cases, and one or more rules. The policy is applied on data.
[007] “Rules” refer to clauses or definitions which are defined for these KPIs. For example, answer-seizure ratio (ASR) as a KPI should be between 40-50. 15 This is a rule. The answer-seizure ratio (ASR) is a measurement of network quality and call success rates in telecommunication. It is the percentage of answered telephone calls with respect to the total call volume.
[008] The expression ‘KPI (Key Performance Indicator)’ used hereinafter in the specification refers to a measurement and a benchmark to achieve optimal 20 network performance goals. To support these goals, measuring actual performance against the KPI goals helps the network team make decisions to improve and sustain network performance.
[009] The expression ‘MOU (Minutes of Usage)’ used hereinafter in the specification refers to the number of minutes a service, such as a cellular plan, is 25 used over a period of time.
[0010] The expression “dynamic policy threshold” refer to a minimum or maximum value which determine when an anomaly is triggered and specify a severity level based on the deviation from a predefined value. The minimum or 3
maximum value are adjusted or updated in real time. The dynamic thresholds can be adjusted dynamically using historical data or statistical methods such as mean, deviation and regression. A machine learning model is used to determine and update the dynamic threshold values based on factors like time and geography. The 5 thresholds are continuously adjusted in near real time based on a policy. For example, thresholds in a policy can be defined as normal-(quality of service) QoS and low-QoS, usage threshold, time period threshold etc. These threshold value when updated in real time based on the policy are known as dynamic thresholds.
[0011] These definitions are in addition to those expressed in the art. 10 BACKGROUND
[0012] The following description of related art is intended to provide background information pertaining to the field of the disclosure. This section may include certain aspects of the art that may be related to various features of the present disclosure. However, it should be appreciated that this section be used only 15 to enhance the understanding of the reader with respect to the present disclosure, and not as admissions of prior art.
[0013] In today's rapidly evolving digital landscape, organizations heavily rely on real-time data streams to make critical decisions and ensure the smooth functioning of their systems. However, detecting anomalies in these data streams is 20 a complex and time-consuming process, especially when dealing with large volumes of data from diverse sources. Conventional anomaly detection systems often struggle to keep pace with the dynamic nature of real-time data, leading to several challenges and limitations.
[0014] One of the primary disadvantages of existing solutions is their 25 reliance on static policies and thresholds for anomaly detection. These predefined rules are often inflexible and require manual updates whenever data patterns change, resulting in a reactive rather than proactive approach to anomaly detection. This manual intervention not only increases the workload on data analysts and 4
system administrators but also introduces the risk of human error and delays in identifying critical anomalies.
[0015] Another significant issue faced by organizations is the lack of adaptability in current anomaly detection systems. As data patterns evolve over 5 time, static thresholds become less effective in capturing new types of anomalies, leading to an increased number of false positives or false negatives. This can result in unnecessary alerts that consume valuable resources or, worse, missing critical anomalies that can have severe consequences for the organization.
[0016] Furthermore, the absence of advanced machine learning techniques 10 in many anomaly detection solutions limits their ability to learn from historical data and improve their accuracy over time. This lack of learning capability makes it difficult for these systems to identify complex patterns and subtle anomalies that may be indicative of potential issues or security breaches.
[0017] Moreover, the sheer volume and velocity of real-time data streams 15 pose significant challenges for traditional anomaly detection approaches. Processing and analyzing massive amounts of data in real-time requires robust and scalable infrastructure, which can be costly and complex to implement. This scalability issue often forces organizations to make trade-offs between the speed of anomaly detection and the depth of analysis performed. 20 [0018] Lastly, the lack of user-friendly interfaces and customization options in existing anomaly detection systems can hinder their adoption and effectiveness. Users may find it challenging to define and manage policies, thresholds, and alerts based on their specific requirements, leading to a one-size-fits-all approach that fails to address the unique needs of different organizations and use cases. 25 [0019] There is, therefore, a need in the art to provide an improved mechanism for managing the policies for the anomaly detection in the real time data using Machine Learning (ML) models. SUMMARY 5
[0020] In an embodiment, a method for managing one or more policies for anomaly detection in real-time data is disclosed. The method comprises receiving, by a load balancer module, a policy creation request from a user interface which includes one or more user-defined parameters for configuring multiple profiles 5 associated with different use cases. The method further comprises creating, by a workflow engine, a policy based on the one or more user-defined parameters received in the policy creation request and validating, by the workflow engine, the created policy. The created policy is stored in a distributed data lake upon the validation. The method also comprises scheduling, by sending a policy execution 10 request to the workflow engine, execution of the created policy retrieved from distributed data lake at a predefined schedule. Additionally, the method comprises determining, by an AI/ML engine, dynamic policy thresholds for each data profile associated with the created policy. Finally, the method comprises executing, by the workflow engine, the created policy based on the determined dynamic policy 15 thresholds according to the predefined schedule for the anomaly detection.
[0021] In an embodiment, the one or more user-defined parameters include thresholds for polices, scheduling frequency, and configurations for the multiple profiles to monitor the performance metrics for the different use cases.
[0022] In an embodiment, the performance metrics include minutes of usage 20 (MOU), attempted calls, and answered calls for a predefined duration.
[0023] In an embodiment, the load balancer module is configured to receive the policy creation request from the user interface and transmit the policy creation request to the workflow engine to authenticate the received policy creation request.
[0024] In an embodiment, the scheduling request sent from the workflow 25 engine is configured for maintaining a scheduling frequency indicating how often the created policy should be executed, and a set of recipient addresses for notifications related to policy execution status and breach alerts.
[0025] In an embodiment, the scheduler is configured to parse the 6
scheduling request received from the workflow engine and generate a response based on the parsed scheduling request. The response may include a schedule for execution of the created policy.
[0026] In an embodiment, the scheduler is configured to send a success 5 notification to the workflow engine when the policy is scheduled successfully.
[0027] In an embodiment, the workflow engine is configured to send a policy creation failure notification to a user via the user interface when a validation of the policy creation request is failed.
[0028] In an embodiment, a process of dynamically updating the dynamic 10 policy thresholds for each profile using the AI/ML engine comprises steps of retrieving historical data for each profile from the distributed data lake, preprocessing the retrieved historical data by handling missing values, outliers, and data formatting. A ML model is trained using the pre-processed historical data to learn normal behavior patterns for each profile. The dynamic policy thresholds are 15 computed based on the trained ML model using one or more parameters from seasonality, trend, data distribution, time, and geography. The computed dynamic policy thresholds in the distributed data lake are stored for use during the execution of the created policy. For example, the computed dynamic threshold may include an updated value of QoS, updated value of usage, or updated value of resource 20 utilization etc.
[0029] In an embodiment, a system for managing one or more policies for anomaly detection in real-time data is disclosed. The system comprises a memory and one or more processor(s) configured to execute a set of instructions stored in the memory for performing a set of steps. The set of steps comprise receiving, by a 25 load balancer module, a policy creation request from a user interface. The request includes one or more user-defined parameters for configuring multiple profiles associated with different use cases. The set of steps comprise creating, by a workflow engine, a policy based on the one or more user-defined parameters received in the request. The set of steps comprise validating, by the workflow 7
engine, the created policy and storing the created policy in a distributed data lake upon the validation. The set of steps comprise scheduling, by a scheduler by sending a policy execution request to the workflow engine, the created policy for execution at a predefined schedule. The set of steps comprise determining, by an AI/ML 5 engine, dynamic policy thresholds for each data profile associated with the created policy. Finally, the set of steps comprise executing, by the workflow engine, the created policy based on the determined dynamic policy thresholds according to the predefined schedule for the anomaly detection.
[0030] In an embodiment, the one or more user-defined parameters include 10 policy thresholds, scheduling frequency, and configurations for the multiple profiles to monitor the performance metrics for the different use cases.
[0031] In an embodiment, the performance metrics include minutes of usage (MOU), attempted calls, and answered calls for a predefined duration.
[0032] In an embodiment, the load balancer module is configured to receive 15 the policy creation request from the user interface and transmit the policy creation request to the workflow engine to authenticate the received policy creation request.
[0033] In an embodiment, the scheduling request from the workflow engine is configured for maintaining a scheduling frequency indicating how often the created policy should be executed, and a set of recipient addresses for notifications 20 related to policy execution status and breach alerts.
[0034] In an embodiment, the scheduler is configured to parse the scheduling request received from the workflow engine and create a response based on the parsed scheduling request.
[0035] In an embodiment, the scheduler is configured to send a success 25 notification via the user interface to a user when the policy is scheduled successfully.
[0036] In an embodiment, the workflow engine is configured to send a 8
policy creation failure notification to a user via the user interface when a validation of the policy creation request fails.
[0037] In an embodiment, a process of dynamically updating the dynamic policy thresholds for each data profile using the AI/ML engine comprises retrieving 5 historical data for each data profile from the distributed data lake, preprocessing the retrieved historical data by handling missing values, outliers, and data formatting, training a ML model using the pre-processed historical data to learn normal behavior patterns for each data profile, computing the dynamic policy thresholds based on the trained ML model using one or more parameters from seasonality, 10 trend, data distribution, time, and geography, and storing the computed dynamic policy thresholds in the distributed data lake for use during the execution of the created policy. The policy management uses minimum and maximum threshold values coming from machine learning algorithms depending upon time and geography. For example, people from Mumbai having different threshold value 15 compared to Uttar Pradesh because Uttar Pradesh people wake up early and start making a call while Mumbai people wake up late So, MOU (Minute of Usage) will be higher for Uttar Pradesh people at early morning.
[0038] In an embodiment, a non-transitory computer-readable medium having stored thereon computer-executable instructions for causing a computer to 20 perform a method for anomaly detection in real-time data using a machine learning (ML) model is disclosed. The method further comprises creating, by a workflow engine, a policy based on the one or more user-defined parameters received in the policy creation request and validating, by the workflow engine, the created policy. The created policy is stored in a distributed data lake upon the validation. The 25 method also comprises scheduling, by sending a policy execution request to the workflow engine, execution of the created policy retrieved from distributed data lake at a predefined schedule. Additionally, the method comprises determining, by an AI/ML engine, dynamic policy thresholds for each data profile associated with the created policy. Finally, the method comprises executing, by the workflow 30 engine, the created policy based on the determined dynamic policy thresholds 9
according to the predefined schedule for the anomaly detection.
[0039] In an embodiment, a User Equipment (UE) for managing one or more policies for anomaly detection in real-time data is disclosed. The UE comprises a memory, one or more processors configured to execute a set of 5 instructions stored in the memory to perform the method steps as stated above. OBJECTS OF THE PRESENT DISCLOSURE
[0040] It is an object of the present disclosure to provide a method and a system for managing policies for anomaly detection in real-time data.
[0041] It is another object of the present disclosure to receive a policy 10 creation request from a user. The policy creation request includes one or more user-defined parameters for configuring multiple profiles associated with different use cases and performance metrics.
[0042] It is yet another object of the present disclosure to create a policy based on the one or more user-defined parameters received in the policy creation 15 request, validate the created policy, and store the validated policy for future use.
[0043] It is an object of the present disclosure to schedule the created policy for execution at predefined intervals based on a scheduling request.
[0044] It is another object of the present disclosure to determine dynamic policy thresholds for each data profile associated with the created policy using an 20 AI/ML engine.
[0045] It is yet another object of the present disclosure to execute the created policy based on the determined dynamic policy thresholds according to the predefined schedule for anomaly detection.
[0046] It is an object of the present disclosure to dynamically update the 25 policy thresholds for each profile using the AI/ML engine by utilizing historical data, preprocessing techniques, machine learning model training, and threshold 10
computation based on the trained model. BRIEF DESCRIPTION OF DRAWINGS
[0047] The accompanying drawings, which are incorporated herein, and constitute a part of this disclosure, illustrate exemplary embodiments of the 5 disclosed methods and systems in which like reference numerals refer to the same parts throughout the different drawings. Components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Some drawings may indicate the components using block diagrams and may not represent the internal circuitry of each 10 component. It will be appreciated by those skilled in the art that disclosure of such drawings includes the disclosure of electrical components, electronic components or circuitry commonly used to implement such components.
[0048] The diagrams are for illustration only, which thus is not a limitation of the present disclosure, and wherein: 15 [0049] FIG. 1 illustrates an exemplary architecture of a system for managing one or more policies for anomaly detection in real-time data, in accordance with embodiments of the present disclosure.
[0050] FIG. 2 illustrates an exemplary architecture of the system, in accordance with embodiments of the present disclosure. 20 [0051] FIG. 3 illustrates a process flow showing a series of steps for managing one or more policies for anomaly detection in real-time data, in accordance with an embodiment of the disclosure.
[0052] FIG. 4 illustrates an exemplary flow representation of the components for managing one or more policies for anomaly detection in real-time 25 data, in accordance with an embodiment of the disclosure. 11
[0053] FIG. 5 illustrates an exemplary block diagram of a computer system in which or with which embodiments of the present disclosure may be implemented.
[0054] FIG. 6 illustrates a method for managing one or more policies for 5 anomaly detection in real-time data, in accordance with an embodiment of the disclosure.
[0055] The foregoing shall be more apparent from the following more detailed description of the disclosure. LIST OF REFERENCE NUMERALS 10 100 – Network Architecture 102 –System 104 –Network 106 – Centralized Server 108-1, 108-2…108-N – User Equipments 15 110-1, 110-2…110-N – Users 202 – One or more processor(s) 204 – Memory 206 – A Plurality of Interfaces 208 – Processing Engine 20 210 – Distributed Data Lake 212 – Load Balancer Module 214 – Workflow Engine 216 – Scheduler 218 – Database 25 220 – Artificial Intelligence/Machine Learning (AI/ML) engine 222 – Other Module(s) 510 – External Storage Device 520 – Bus 530 – Main Memory 30 540 – Read Only Memory 12
550 – Mass Storage Device 560 – Communication Port 570 – Processor 600 – Method 5 602, 604, 606, 608, 610, and 612 – steps of Method 600 DETAILED DESCRIPTION
[0056] In the following description, for the purposes of explanation, various specific details are set forth in order to provide a thorough understanding of embodiments of the present disclosure. It will be apparent, however, that 10 embodiments of the present disclosure may be practiced without these specific details. Several features described hereafter can each be used independently of one another or with any combination of other features. An individual feature may not address any of the problems discussed above or might address only some of the problems discussed above. Some of the problems discussed above might not be 15 fully addressed by any of the features described herein. Example embodiments of the present disclosure are described below, as illustrated in various drawings in which like reference numerals refer to the same parts throughout the different drawings.
[0057] The ensuing description provides exemplary embodiments only, and 20 is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the ensuing description of the exemplary embodiments will provide those skilled in the art with an enabling description for implementing an exemplary embodiment. It should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope 25 of the disclosure as set forth.
[0058] Specific details are given in the following description to provide a thorough understanding of the embodiments. However, it will be understood by one of ordinary skill in the art that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other 13
components may be shown as components in block diagram form in order not to obscure the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments. 5 [0059] Also, it is noted that individual embodiments may be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. 10 A process is terminated when its operations are completed but could have additional steps not included in a figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination can correspond to a return of the function to the calling function or the main function. 15 [0060] The word “exemplary” and/or “demonstrative” is used herein to mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited by such examples. In addition, any aspect or design described herein as “exemplary” and/or “demonstrative” is not necessarily to be construed as preferred or advantageous over other aspects or 20 designs, nor is it meant to preclude equivalent exemplary structures and techniques known to those of ordinary skill in the art. Furthermore, to the extent that the terms “includes,” “has,” “contains,” and other similar words are used in either the detailed description or the claims, such terms are intended to be inclusive like the term “comprising” as an open transition word without precluding any additional or other 25 elements.
[0061] Reference throughout this specification to “one embodiment” or “an embodiment” or “an instance” or “one instance” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, the appearances of the 30 phrases “in one embodiment” or “in an embodiment” in various places throughout 14
this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0062] The terminology used herein is to describe particular embodiments 5 only and is not intended to be limiting the disclosure. As used herein, the singular forms “a”, “an”, and “the” are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or 10 components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term “and/or” includes any combinations of one or more of the associated listed items. It should be noted that the terms “mobile device”, “user equipment”, “user device”, “communication device”, “device” and similar terms 15 are used interchangeably for the purpose of describing the invention. These terms are not intended to limit the scope of the invention or imply any specific functionality or limitations on the described embodiments. The use of these terms is solely for convenience and clarity of description. The invention is not limited to any particular type of device or equipment, and it should be understood that other 20 equivalent terms or variations thereof may be used interchangeably without departing from the scope of the invention as defined herein.
[0063] As used herein, an “electronic device”, or “portable electronic device”, or “user device” or “communication device” or “user equipment” or “device” refers to any electrical, electronic, electromechanical, and computing 25 device. The user device is capable of receiving and/or transmitting one or parameters, performing function/s, communicating with other user devices, and transmitting data to the other user devices. The user equipment may have a processor, a display, a memory, a battery, and an input-means such as a hard keypad and/or a soft keypad. The user equipment may be capable of operating on any radio 30 access technology including but not limited to IP-enabled communication, Zig Bee, 15
Bluetooth, Bluetooth Low Energy, Near Field Communication, Z-Wave, Wi-Fi, Wi-Fi direct, etc. For instance, the user equipment may include, but not limited to, a mobile phone, smartphone, virtual reality (VR) devices, augmented reality (AR) devices, laptop, a general-purpose computer, desktop, personal digital assistant, 5 tablet computer, mainframe computer, or any other device as may be obvious to a person skilled in the art for implementation of the features of the present disclosure.
[0064] Further, the user device may also comprise a “processor” or “processing unit” includes processing unit, wherein processor refers to any logic circuitry for processing instructions. The processor may be a general-purpose 10 processor, a special purpose processor, a conventional processor, a digital signal processor, a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits, Field Programmable Gate Array circuits, any other type of integrated circuits, etc. The processor may perform signal coding data processing, 15 input/output processing, and/or any other functionality that enables the working of the system according to the present disclosure. More specifically, the processor is a hardware processor.
[0065] As portable electronic devices and wireless technologies continue to improve and grow in popularity, the advancing wireless technologies for data 20 transfer are also expected to evolve and replace the older generations of technologies. In the field of wireless data communications, the dynamic advancement of various generations of cellular technology are also seen. The development, in this respect, has been incremental in the order of second generation (2G), third generation (3G), fourth generation (4G), and now fifth generation (5G), 25 and more such generations are expected to continue in the forthcoming time.
[0066] While considerable emphasis has been placed herein on the components and component parts of the preferred embodiments, it will be appreciated that many embodiments can be made and that many changes can be made in the preferred embodiments without departing from the principles of the 30 disclosure. These and other changes in the preferred embodiment as well as other 16
embodiments of the disclosure will be apparent to those skilled in the art from the disclosure herein, whereby it is to be distinctly understood that the foregoing descriptive matter is to be interpreted merely as illustrative of the disclosure and not as a limitation. 5 [0067] The various embodiments throughout the disclosure will be explained in more detail with reference to FIG. 1- FIG. 6.
[0068] FIG. 1 illustrates an exemplary architecture (100) of a system (102) for anomaly detection in real-time data using a machine learning (ML) model, in accordance with embodiments of the present disclosure. 10 [0069] Referring to FIG. 1 the network architecture 100 is implemented for anomaly detection in real-time data using a machine learning (ML) model, is illustrated. In an embodiment, the system (102) is connected to a network 104, which is further connected to at least one computing devices 108-1, 108-2, … 108-N (collectively referred as computing device 108, herein) associated with one or 15 more users 110-1, 110-2, … 110-N (collectively referred as user (110), herein). The computing device 108 may be a smartphone, a personal computer, a laptop, a tablet, a wristwatch, or any custom-built computing device integrated within a modern diagnostic machine that can connect to a network as an IoT (Internet of Things) device. In an embodiment, the computing device 108 may also be referred to as 20 User Equipment (UE) or user device. Accordingly, the terms “computing device” and “User Equipment” may be used interchangeably throughout the disclosure. In an aspect, the user (110) is a network operator or a field engineer. Further, the network 104 can be configured with a centralized server 106 that stores compiled data. 25 [0070] In an embodiment, the system (102) may receive at least one input data from the user (110) via the at least one computing devices (108). In an aspect, the user (110) may be configured to initiate the process of managing one or more policies for anomaly detection in real-time data, through an application interface of a mobile application installed in the computing devices (108). The mobile 17
application may be configured to communicate with the network analysis server. In some examples, the mobile application may be a software or a mobile application from an application distribution platform. In an embodiment, the computing device 108 may transmit the at least one captured data packet over a point-to-point or point- 5 to-multipoint communication channel or network (104) to the system (102). In an embodiment, the computing device (108) may involve collection, analysis, and sharing of data received from the system (102) via the network (104).
[0071] In an exemplary embodiment, the network 104 may include, but not be limited to, at least a portion of one or more networks having one or more nodes 10 that transmit, receive, forward, generate, buffer, store, route, switch, process, or a combination thereof, etc. one or more messages, packets, signals, waves, voltage or current levels, some combination thereof, or so forth. In an exemplary embodiment, the network 104 may include, but not be limited to, a wireless network, a wired network, an internet, an intranet, a public network, a private network, a packet- 15 switched network, a circuit-switched network, an ad hoc network, an infrastructure network, a public-switched telephone network (PSTN), a cable network, a cellular network, a satellite network, a fiber optic network, or some combination thereof.
[0072] Although FIG. 1 shows exemplary components of the network architecture 100 for managing one or more policies for anomaly detection in real- 20 time data, in other embodiments, the network architecture 100 may include fewer components, different components, differently arranged components, or additional functional components than depicted in FIG. 1. Additionally, or alternatively, one or more components of the network architecture 100 may perform functions described as being performed by one or more other components of the network 25 architecture 100.
[0073] FIG. 2 with reference to FIG. 1, illustrates an exemplary architecture (200) of the system (102) for managing one or more policies for anomaly detection in real-time data, in accordance with an embodiment of the present disclosure. 18
[0074] The system (102) includes one or more processor(s) (202), a memory (204), a processing engine (208), a database (218), and an interface(s) (206). In an exemplary embodiment, the processing engine (208) may include one or more engines selected from any of a load balancer module (212), a workflow engine 5 (214), a scheduler (216), an artificial intelligence/machine learning (AI/ML) engine (220), and other modules (222) having functions that may include but are not limited to receiving data, processing data, testing, storage, and peripheral functions, such as wireless communication unit for remote operation, audio unit for alerts and the like. 10 [0075] The one or more processor(s) (202) is configured to initiate the processor of managing one or more policies for anomaly detection in real-time data through the application interface of the user equipment (UE) (108). In an embodiment, the application interface is configured to transmit one or more instructions to the one or more processor(s) (202). Further, the system (102) 15 comprises a distributed data lake (210). The distributed data lake (210) is configured to store historical data for each data profile.
[0076] In an embodiment, the one or more processor(s) 202 may be implemented as one or more microprocessors, microcomputers, microcontrollers, edge or fog microcontrollers, digital signal processors, central processing units, 20 logic circuitries, and/or any devices that process data based on operational instructions. Among other capabilities, the one or more processor(s) 202 may be configured to fetch and execute computer-readable instructions stored in the memory 204 of the system 102. The memory 204 may be configured to store one or more computer-readable instructions or routines in a non-transitory computer 25 readable storage medium, which may be fetched and executed to create or share data packets over a network service. The memory 204 may comprise any non-transitory storage device including, for example, volatile memory such as random access memory (RAM), or non-volatile memory such as Erasable Programmable read-only memory (EPROM), flash memory, and the like. 19
[0077] The interface(s) (206) is included within the system (102) to serve as a medium for data exchange, configured to facilitates user interaction with the mobile application for anomaly detection in real-time data using a machine learning (ML) model. The interface(s) (206) may be composed of interfaces for data input 5 and output devices, storage devices, and the like, providing a communication pathway for the various components of the system (102).
[0078] The interface(s) 206 may comprise a variety of interfaces, for example, interfaces for data input and output devices, referred to as I/O devices, storage devices, and the like. The interface(s) 206 may facilitate communication 10 to/from the system (102). The interface(s) 206 may also provide a communication pathway for one or more components of the system (102). Examples of such components include but are not limited to, the processing unit/engine(s) (208), the database 218, and the distributed data lake (210).
[0079] In an embodiment, the processing unit/engine(s) (208) may be 15 implemented as a combination of hardware and programming (for example, programmable instructions) to implement one or more functionalities of the processing engine(s) (208). In examples described herein, such combinations of hardware and programming may be implemented in several different ways. For example, the programming for the processing engine(s) (208) may be processor- 20 executable instructions stored on a non-transitory machine-readable storage medium and the hardware for the processing engine(s) (208) may comprise a processing resource (for example, one or more processors), to execute such instructions. In the present examples, the machine-readable storage medium may store instructions that, when executed by the processing resource, implement the 25 processing engine(s) (208). In such examples, the system (102) may comprise the machine-readable storage medium storing the instructions and the processing resource to execute the instructions, or the machine-readable storage medium may be separate but accessible to the system (102) and the processing resource. In other examples, the processing engine(s) (208) may be implemented by electronic 30 circuitry. 20
[0080] In an embodiment, the database (218) is configured for serving as a centralized repository for storing and retrieving various operational data. The database (218) is designed to interact seamlessly with other components of the system (102), such as the load balancer module (212), the workflow engine (214), 5 the scheduler (216), AI/ML engine (220) to support the system's functionality effectively. The database (218) may store data that may be either stored or generated as a result of functionalities implemented by any of the components of the one or more processor(s) (202) or the processing engines (208). In an embodiment, the database (218) may be separate from the system (102). 10 [0081] In an embodiment, the load balancer module (212) acts as the entry point for policy creation requests from the user interface (206). The load balancer module (212) receives the policy creation requests, sends the received policy creation requests for authentication to the workflow engine for validating user credentials and further processing. 15 [0082] In an embodiment, the workflow engine (214) is responsible for authenticating the policy creation requests and creating policies based on the one or more user-defined parameters received from the load balancer module (212). The one or more user-defined parameters are included in the policy creation/schedule request. The one or more user-defined parameters include policy thresholds, 20 scheduling frequency, and configurations for the multiple profiles to monitor the performance metrics for the different use cases. The threshold may represent a minimum and maximum value below and above an anomaly is detected. The scheduling frequency represents a time interval after which the policy is executed. For example, the scheduling frequency may be set hourly, daily or a specified time 25 interval. The policy configurations may include but not limited to configuring user access rights and setting user authentication rules. The workflow engine (214) validates the created policies and stores the created policies in the distributed data lake (210) upon the validation. The workflow engine (214) also handles sending policy creation failure notifications to the user via the user interface (206) when the 30 validation of the created policy fails. 21
[0083] In an embodiment, the scheduler (216) plays a vital role in scheduling the execution of the created policies. The scheduler (216) parses scheduling requests received from the workflow engine (214), creates a response based on scheduling frequency and policy included in the scheduling request. The 5 scheduler (216) monitors the execution of the created policies by the workflow engine (214). When the created policies are successfully scheduled and executed, the scheduler (216) sends success notifications to the user via the user interface (206).
[0084] In an embodiment, the AI/ML engine (220) is the core component 10 responsible for determining and dynamically updating the dynamic policy thresholds for each data profile associated with the created policies. The data profile is a user specific profile configured by the user to track the different use cases. A single profile can be configured to track a single or multiple use cases. The AI/ML engine (220) retrieves historical data for each data profile configured by the user to 15 track different user cases from the distributed data lake (210), preprocesses the retrieved historical data by handling missing values, outliers, and data formatting, and trains a ML model using the pre-processed historical data to learn normal behavior patterns for each profile. The profile may include user defined parameters such as thresholds, scheduling frequency, and configurations. The threshold may 20 represent a minimum and maximum value below and above an anomaly is detected. The scheduling frequency represents a time interval after which the policy is executed. For example, the scheduling frequency may be set hourly, daily or a specified time interval. The policy configurations may include but not limited to configuring user access rights and setting user authentication rules. 25 [0085] The AI/ML engine (220) computes the dynamic policy thresholds based on the trained ML model using one or more parameters from seasonality, trend, time, and geography. The dynamic policy threshold refer to a minimum or maximum value which determine when an anomaly is triggered and specify the severity level based on the deviation from a predefined value. The dynamic 30 thresholds can be adjusted dynamically using historical data or statistical methods 22
such as mean, deviation and regression. A machine learning model is used to determine and update the dynamic threshold values based on factors like time and geography. . For example, thresholds in a policy can be defined as normal-(quality of service) QoS, low-QoS, usage threshold, and time period threshold etc. The 5 thresholds can be updated in real time as the policy is executed. For example, the dynamic threshold computed by the AI/ML engine may be different in morning and night when the load on the network is different. In another example, time may be specific days of the month, or a season such as winter, rainy season where the policy threshold needs to be adjusted accordingly. In another example, the dynamic 10 threshold also depends on geography such as high populated area and low populated area. The geography may also include specific longitude and latitude value. The trend may include historical information of the dynamic thresholds at specific places and specific time periods.
[0086] FIG. 3 illustrates a process flow (300) showing a series of steps for 15 managing one or more policies for anomaly detection in real-time data, in accordance with embodiments of the present disclosure.
[0087] Initially, the user interface (206) serves as a primary point of interaction between the users (110) and the system (102). The user interface (206) enables users (110) to input policy creation requests at 302, which include one or 20 more user-defined parameters for configuring multiple profiles associated with different use cases and performance metrics. For example, the different use cases may be in network usage, autonomous vehicles, fixed wireless technology etc. There can be various key performance indicators (KPIs) which are linked/attached to different use cases. In an example, the performance metrics may include 25 monitoring minutes of usage (MOU), attempted calls, answered calls for a pre-defined duration, resource utilization, latency and efficiency etc.
[0088] The user interface (206) is further configured to enable users to define thresholds, scheduling frequency, and configurations for the multiple profiles to monitor the performance metrics for the different use cases. The 30 threshold may represent a minimum and maximum value below and above an 23
anomaly is detected. The scheduling frequency represents a time interval after which the policy is executed. For example, the scheduling frequency may be set hourly, daily or a specified time interval. The policy configurations may include but not limited to configuring user access rights, resource allocation strategy, and 5 setting user authentication rules.
[0089] The load balancer module (212) receives the policy creation request from the user interface (206) at step 304 and sends the received policy creation request to the workflow engine for validating user credentials. Once authenticated, the workflow engine (214) transfers the policy schedule request to the scheduler. 10 [0090] The workflow engine (214) plays a crucial role in creating and managing policies. The workflow engine (214) creates a policy based on the one or more user-defined parameters received in the policy creation request. The workflow engine (214) then validates the created policy at 308. If the validation fails, the workflow engine (214) sends a policy creation failure notification to the user via 15 the user interface (206). Once validated, the workflow engine (214) stores the created policy in the distributed data lake (210) at step 310.
[0091] The scheduler (216) is responsible for scheduling the execution of the created policies based on a scheduling request received from the workflow engine (214) at step 312. The scheduling request from the workflow engine (214) 20 is configured for maintaining a scheduling frequency, indicating how often the created policy should be executed, and a set of recipient addresses for notifications related to policy execution status and breach alerts. The scheduler (216) parses the scheduling request received from the workflow engine (214), creates a response based on the scheduling frequency and policy configurations for creation of the 25 policy. The policy configurations may include but not limited to configuring user access rights, resource allocation strategy, and setting user authentication rules.
[0092] When the created policy is scheduled successfully at step 314, the scheduler (216) sends a success notification to the workflow engine. The workflow engine sends a response based on the response from the scheduler indicating that 24
the policy is created successfully at step 316. The response is forwarded to the user interface (206) to the user at step 318 via the load balancer module. In case the validation is failed, the workflow engine sends a failed to create policy message to the load balancer at step 320 which in turn is provided to the user at step 322. 5 [0093] The AI/ML engine (220) is at the core of the system's anomaly detection capabilities. The workflow engine receives policy execution request from the scheduler at 326. The AI/ML engine (220) determines dynamic policy thresholds for each data profile associated with the created policy at step 328. For this purpose, the AI/ML engine (220) is configured for retrieving historical data for 10 each profile from the distributed data lake (210), preprocessing the retrieved historical data by handling missing values, outliers, and data formatting. Further, the AI/ML engine (220) is configured for training a ML model using the pre-processed historical data to learn normal behavior patterns for each profile. The AI/ML engine (220) then computes the dynamic policy thresholds based on the 15 trained ML model using one or more parameters from seasonality, trend, data distribution, time, and geography at step 328. The computed dynamic policy thresholds are stored in the distributed data lake (210) for use during the execution of the created policy.
[0094] During the execution of the created policy, the workflow engine 20 (214) executes the created policy based on the determined dynamic policy thresholds according to the predefined schedule for the anomaly detection at step 330. The workflow engine (214) receives real-time data streams for each data profile from data sources and applies the trained ML model to detect anomalies. The detected anomalies are identified as a policy breach. The workflow engine 25 (214) saves the detected anomalies and then sends a notification to the user via the user interface (206) at step 334 according to a predefined schedule.
[0095] The distributed data lake (210) serves as a centralized repository for storing the created policies, historical data, and other relevant information. The distributed data lake (210) ensures data durability, scalability, and accessibility for 30 the various components of the system (102). 25
[0096] In summary, the system (102) for managing one or more policies for anomaly detection in real-time data provides a comprehensive and automated approach to detect anomalies and take proactive measures. The user interface (206) allows users to create policies and define one or more user-defined parameters for 5 configuring multiple profiles associated with different use cases and performance metrics. The load balancer module (212) transfers the policy creation requests to the workflow engine (214), which creates, validates, and stores the policies. The scheduler (216) then handles the scheduling of the policies. The AI/ML engine (220) determines the dynamic policy thresholds for each data profile associated 10 with the created policy using ML models. The workflow engine (214) executes the created policy based on the determined dynamic policy thresholds according to the predefined schedule for the anomaly detection. The distributed data lake (210) serves as a centralized storage for the created policies and historical data. Overall, the system (102) enables proactive anomaly detection and simplifies the 15 configuration process for users.
[0097] FIG. 4 illustrates an exemplary flow representation of the components for managing one or more policies for anomaly detection in real-time data, in accordance with an embodiment of the disclosure.
[0098] As depicted in FIG. 4, the user (110) interacts with the system (102) 20 through the user interface (206), which is bi-directionally connected to the load balancer module (212). The user interface (206) allows users (110) to create policies by providing one or more user-defined parameters for configuring multiple profiles associated with different use cases and performance metrics. The one or more user-defined parameters include thresholds, scheduling frequency, and configurations 25 for the multiple profiles to monitor the performance metrics for the different use cases. The performance metrics may include minutes of usage (MOU), attempted calls, and answered calls for a previous hour. The user interface (206) provides a user-friendly and intuitive means for users to input their requirements and receive notifications. 26
[0099] The load balancer module (212) acts as an intermediary between the user interface (206) and the workflow engine (214). When a user submits a policy creation request through the user interface (206), the load balancer module (212) receives the policy creation request and transfers the received policy creation 5 request to the workflow engine for validating user credentials. Once the policy creation request is authenticated, the workflow engine (214) transfers the authenticated policy creation request to the scheduler for scheduling the policy. The communication between the load balancer module (212) and the workflow engine (214) is facilitated via a HyperText Transfer Protocol (HTTP) request, ensuring 10 secure and efficient data transfer.
[00100] The workflow engine (214) plays a central role in the policy management process. The workflow engine (214) receives the policy creation request from the load balancer module (212) and creates a policy based on the one or more user-defined parameters. The workflow engine (214) validates the created 15 policy. If the validation fails, the workflow engine (214) sends a policy creation failure notification to the user via the user interface (206). If the validation is successful, the workflow engine (214) stores the created policy in the distributed data lake (210). The distributed data lake (210) serves as a centralized repository for storing the created policies, historical data, and other relevant information. The 20 workflow engine (214) and the distributed data lake (210) are connected via a transmission control protocol (TCP) connection, ensuring reliable and efficient data transfer.
[00101] After storing the created policy, the workflow engine (214) may send a scheduling request to the scheduler (216). The scheduling request from the 25 workflow engine (214) is configured for maintaining the scheduling frequency, indicating how often the created policy should be executed, and a set of recipient addresses for notifications related to policy execution status and breach alerts. The scheduler (216) is responsible for scheduling the execution of the created policies based on the scheduling request received from the workflow engine (214). The 30 scheduler (216) parses the scheduling request received from the workflow engine 27
(214), creates a response based on the scheduling frequency and policy. The scheduler (216) and the distributed data lake (210) are also connected via a TCP connection, allowing seamless data exchange.
[00102] The AI/ML engine (220) is a critical component of the system, 5 responsible for determining the dynamic policy thresholds for each data profile associated with the created policy. The AI/ML engine (220) is connected to the workflow engine (214), enabling close collaboration and data flow between the two components. The AI/ML engine (220) retrieves historical data for each profile from the distributed data lake (210), preprocesses the retrieved historical data by 10 handling missing values, outliers, and data formatting, and trains a ML model using the pre-processed historical data to learn normal behavior patterns for each profile. The pre-processed historical data may correspond to network data captured from network resources including a wireless network, a wired network, an internet, an intranet, a public network, a private network, a packet-switched network, a circuit- 15 switched network, an ad hoc network, an infrastructure network, a Public-Switched Telephone Network (PSTN), a cable network, a cellular network, a satellite network, a fiber optic network, or some combination thereof. The AI/ML engine (220) then computes the dynamic policy thresholds based on the trained ML model using one or more parameters from seasonality, trend, data distribution, time, and 20 geography. The computed dynamic policy thresholds are stored in the distributed data lake (210) for use during the execution of the created policy. The workflow engine (214) executes the created policy based on the determined dynamic policy thresholds according to the predefined schedule for the anomaly detection.
[00103] The disclosed system and method leverage the power of trained ML 25 models to automatically determine the dynamic policy thresholds and detect anomalies, eliminating the need for manual threshold setting and monitoring. This approach saves time and effort while ensuring accurate identification of potential issues. The ability to track anomalies in real-time enables users to promptly respond to any potential issues, mitigating their impact and minimizing downtime. 30 Moreover, the dynamic determination of thresholds and policies based on historical
data eliminates the need for manual changes to the policy each time the data pattern changes. The AI/ML engine (220), working in conjunction with the workflow engine (214) and the scheduler (216), identifies the tolerance levels and thresholds of network failures at each layer from historical data and applies anomaly detection 5 to detect anomalies in real-time data.
[00104] FIG. 5 illustrates an exemplary computer system 500 in which or with which embodiments of the present disclosure may be implemented.
[00105] As shown in FIG. 5, the computer system 500 may include an external storage device 510, a bus 520, a main memory 530, a read-only memory 10 540, a mass storage device 550, communication port(s) 560, and a processor 570. A person skilled in the art will appreciate that the computer system 500 may include more than one processor and communication ports. The processor 570 may include various modules associated with embodiments of the present disclosure. The communication port(s) 560 may be any of an RS-232 port for use with a modem- 15 based dialup connection, a 10/100 Ethernet port, a Gigabit or 10 Gigabit port using copper or fiber, a serial port, a parallel port, or other existing or future ports. The communication port(s) 560 may be chosen depending on a network, such a Local Area Network (LAN), Wide Area Network (WAN), or any network to which the computer system 500 connects. The main memory 530 may be random access 20 memory (RAM), or any other dynamic storage device commonly known in the art. The read-only memory 540 may be any static storage device(s) including, but not limited to, a Programmable Read Only Memory (PROM) chips for storing static information e.g., start-up or basic input/output system (BIOS) instructions for the processor 570. The mass storage device 550 may be any current or future mass 25 storage solution, which may be used to store information and/or instructions.
[00106] The bus 520 communicatively couples the processor 570 with the other memory, storage, and communication blocks. The bus 520 can be, e.g. a Peripheral Component Interconnect (PCI) / PCI Extended (PCI-X) bus, Small Computer System Interface (SCSI), universal serial bus (USB), or the like, for 30 connecting expansion cards, drives, and other subsystems as well as other buses, such a front side bus (FSB), which connects the processor 570 to the computer system 500.
[00107] Optionally, operator and administrative interfaces, e.g. a display, keyboard, and a cursor control device, may also be coupled to the bus 520 to support 5 direct operator interaction with the computer system 500. Other operator and administrative interfaces may be provided through network connections connected through the communication port(s) 560. In no way should the aforementioned exemplary computer system 500 limit the scope of the present disclosure.
[00108] FIG. 6 illustrates an exemplary flow diagram of a method (600) for 10 managing one or more policies for anomaly detection in real-time data.
[00109] At step 602, the method may begin with the load balancer module (212) receiving a policy creation request from the user interface (206). This policy creation request may include one or more user-defined parameters for configuring multiple profiles associated with different use cases and performance metrics. The 15 one or more user-defined parameters may comprise thresholds, scheduling frequency, and configurations for the multiple profiles to monitor the performance metrics for the different use cases. For example, the performance metrics may include minutes of usage (MOU), attempted calls, and answered calls for a predefined duration. By allowing users to define multiple profiles and monitor 20 various performance metrics, the system may provide flexibility and adaptability to diverse use cases and requirements.
[00110] Upon receiving the policy creation request, the load balancer module (212) may transfer the received policy creation request for validating user credentials. This authentication process may ensure that only authorized users can 25 create policies, enhancing security. Once authenticated, the workflow engine (214) may send a policy schedule request to the scheduler (216) for further processing.
[00111] At step 604, the workflow engine (214) may then create a policy based on the one or more user-defined parameters received in the policy creation request. The workflow engine (214) may then validate the created policy. This validation process may involve checking for user defined parameters and data consistency. If the validation fails, the workflow engine (214) may send a policy creation failure notification to a user via the user interface (206). This feedback may 5 help users to identify and rectify any issues in their policy creation requests.
[00112] However, if the policy is successfully validated, the workflow engine (214) may store the created policy in the distributed data lake (210) at step 606. The distributed data lake (210) may serve as a centralized repository for storing the created policies, historical data, and other relevant information. Storing the 10 created policies in the distributed data lake (210) may facilitate efficient retrieval and processing of policies during the anomaly detection process.
[00113] At step 608, after storing the created policy, the workflow engine (214) may send a scheduling request to the scheduler (216). The scheduling request from the workflow engine (214) may include the scheduling frequency indicating 15 how often the created policy should be executed and a set of recipient addresses for notifications related to policy execution status and breach alerts. The scheduler (216) may parse the scheduling request received from the workflow engine (214) and create a response based on the scheduling frequency and other user defined parameters. The response may be responsible for triggering the execution of the 20 created policy at the specified intervals.
[00114] The scheduler (216) may also monitor the execution of the scheduled policy by the workflow engine (214). When the created policy is scheduled successfully, the scheduler (216) may send a success notification via the user interface (206) to the user. This notification may prompt the user to take action if 25 necessary, facilitating proactive management of network resources.
[00115] At step 610, during the execution of the created policy, the workflow engine (214) may rely on the AI/ML engine (220) which determine dynamic policy thresholds for each data profile associated with the created policy. The AI/ML engine (220) may employ a machine learning model trained on historical data to learn normal behavior patterns for each profile. By analyzing historical data, the machine learning model may establish a baseline for normal behavior.
[00116] The process of dynamically updating the dynamic policy thresholds may involve several steps. Firstly, the AI/ML engine (220) may retrieve historical 5 data for each profile from the distributed data lake (210). This historical data may undergo preprocessing to handle missing values, outliers, and data formatting inconsistencies. The pre-processed historical data may then be used to train the ML model, enabling it to learn the normal behavior patterns for each profile. Once trained, the ML model may compute the dynamic policy thresholds based on 10 various parameters such as seasonality, trend, data distribution, time, and geography. These parameters may contribute to the dynamic nature of the thresholds. The computed dynamic policy thresholds may be stored in the distributed data lake (210) for use during the execution of the created policy.
[00117] At step 612, with the dynamic policy thresholds determined, the 15 workflow engine (214) may proceed to execute the created policy based on the determined dynamic policy thresholds according to the predefined schedule for the anomaly detection. Executing the created policy may involve receiving real-time data streams for each data profile from data sources. By analyzing the real-time data streams, the ML model may detect anomalies that deviate from the expected 20 behavior.
[00118] The distributed data lake (210) may serve as a centralized repository for storing the created policies, historical data, and other relevant information. The distributed data lake (210) may ensure data durability, scalability, and accessibility for the various components of the system. The use of a distributed data lake (210) 25 as a centralized repository for storing the created policies, historical data, and other relevant information may enhance scalability and efficiency. The distributed architecture may allow for the processing of large volumes of real-time data streams. Additionally, the distributed data lake (210) may ensure data durability and fault tolerance, protecting against data loss and failures.
[00119] In an embodiment, a non-transitory computer-readable medium having stored thereon computer-executable instructions for causing a computer to perform the method (600) for anomaly detection in real-time data using a machine learning (ML) model is disclosed. The method comprises receiving, by a load 5 balancer module (212), a policy creation/schedule request from a user interface (206), wherein the request includes one or more user-defined parameters for configuring multiple profiles associated with different use cases and performance metrics. The method further involves creating, by a workflow engine (214), a policy based on the one or more user-defined parameters received in the request, 10 validating, by the workflow engine (214), the created policy and storing the created policy in a distributed data lake (210) upon the validation of the policy. The method also includes scheduling, by a scheduler (216) based on a scheduling request from the workflow engine (214), the created policy for execution at a predefined schedule. Additionally, the method involves determining, by an AI/ML engine 15 (220), dynamic policy thresholds for each data profile associated with the created policy and executing, by the workflow engine (214), the created policy based on the determined dynamic policy thresholds according to the predefined schedule for the anomaly detection.
[00120] In an embodiment, a User Equipment (UE) (108) for managing one 20 or more policies for anomaly detection in real-time data is disclosed. The UE (108) comprises a memory and one or more processors configured to execute a set of instructions stored in the memory to perform the method (600) as described above.
[00121] The present disclosure provides technical advancement related to anomaly detection in real-time data. This advancement addresses the limitations of 25 existing solutions by introducing a comprehensive policy management system that leverages user-defined parameters, dynamic policy thresholds, and machine learning techniques. The disclosure involves a load balancer module for forwarding policy creation requests, a workflow engine for creating, validating, and storing policies, a scheduler for scheduling policy execution, and an AI/ML engine for 30 determining dynamic policy thresholds. By implementing a distributed data lake architecture, the disclosed invention enhances scalability, data durability, and fault tolerance, resulting in efficient processing of large volumes of real-time data streams and protection against data loss and failures.
[00122] While the foregoing describes various embodiments of the 5 invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof. The scope of the invention is determined by the claims that follow. The invention is not limited to the described embodiments, versions or examples, which are included to enable a person having ordinary skill in the art to make and use the invention when combined with information and 10 knowledge available to the person having ordinary skill in the art. ADVANTAGES OF THE PRESENT DISCLOSURE
[00123] The present disclosure provides a system and method to manage one or more policies for anomaly detection in real-time data.
[00124] The present disclosure enables receiving a policy creation request 15 from a user interface, wherein the policy creation request includes one or more user-defined parameters for configuring multiple profiles associated with different use cases and performance metrics.
[00125] The present disclosure provides advantage related to creating a policy based on the one or more user-defined parameters received in the policy 20 creation request, validating the created policy, and storing the created policy in a distributed data lake upon the validation.
[00126] The present disclosure enables scheduling the created policy retrieved from the distributed data lake for execution at a predefined schedule based on a scheduling request. 25 [00127] The present disclosure provides advantage related to determining dynamic policy thresholds for each data profile associated with the created policy using an AI/ML engine.
[00128] The present disclosure provides a system and a method to execute the created policy based on the determined dynamic policy thresholds according to the predefined schedule for anomaly detection.
[00129] The present disclosure provides a system and a method to utilize 5 historical data, preprocessing techniques, machine learning model training, and threshold computation based on the trained model to dynamically update the policy thresholds for each profile using the AI/ML engine.
[00130] The present disclosure provides advantage related to automatically determining the dynamic policy thresholds and detecting anomalies using trained 10 ML models, eliminating the need for manual threshold setting and monitoring, thus saving time and effort while ensuring accurate identification of potential issues.
[00131] The present disclosure provides technical advancement related to tracking anomalies in real-time, enabling users to promptly respond to any potential issues, mitigating their impact and minimizing downtime. 15 [00132] The present disclosure provides technical advancement related to dynamically determining thresholds and policies based on historical data, eliminating the need for manual changes to the policy each time the data pattern changes.
WE CLAIM: 1. A method for managing one or more policies for anomaly detection in real¬ time data, the method comprising: 5 receiving (602), by a load balancer module (212), a policy creation request from a user interface (206), wherein the policy creation request includes one or more user-defined parameters for configuring multiple profiles associated with different use cases; creating (604), by a workflow engine (214), a policy based on the one 10 or more user-defined parameters received in the policy creation request, validating (606), by the workflow engine (214), the created policy and storing the created policy in a distributed data lake (210) upon the validation; scheduling (608), by a scheduler (216), execution of the created policy retrieved from the distributed data lake (210) at a predefined schedule; 15 determining (610), by an Artificial Intelligence/Machine Learning (AI/ML) engine (220), dynamic policy thresholds for each profile associated with the created policy; and executing (612), by the workflow engine (214), the created policy based on the determined dynamic policy thresholds according to the predefined 20 schedule for the anomaly detection. 2. The method (600) as claimed in claim 1, wherein the one or more user-defined parameters include thresholds, scheduling frequency for execution of the created policy, and configurations for the multiple profiles to monitor performance metrics for the different use cases. 25 36
3. The method (600) as claimed in claim 2, wherein the performance metrics include minutes of usage (MOU), attempted calls, and answered calls for a predefined duration. 5 4. The method (600) as claimed in claim 1, wherein the load balancer module (212) is configured to: receive the policy creation request from the user interface (206); and transmit the policy creation request to the workflow engine to authenticate the received policy creation request. 10 5. The method (600) as claimed in claim 1, wherein a scheduling request sent from the workflow engine (214) to the scheduler (216) is configured for maintaining: a scheduling frequency indicating how often the created policy should 15 be executed; and a set of recipient addresses for notifications related to policy execution status and breach alerts. 6. The method (600) as claimed in claim 5, wherein the scheduler is configured to: 20 parse the scheduling request received from the workflow engine (214); and generate a response based on the parsed scheduling request. 7. The method (600) as claimed in claim 6, wherein the scheduler is configured 25 to: send a success notification to the workflow engine when the policy is scheduled successfully. 8. The method (600) as claimed in claim 1, wherein the workflow engine (214) 30 is configured to: 37
send a policy creation failure notification to a user via the user interface (206) when a validation of the policy creation request fails. 9. The method (600) as claimed in claim 1, wherein a process of dynamically 5 updating the dynamic policy thresholds for each profile using the AI/ML engine (220) comprises steps of: retrieving historical data for each profile from the distributed data lake (210); preprocessing the retrieved historical data by handling missing 10 values, outliers, and data formatting; training a ML model using the pre-processed historical data to learn normal behavior patterns for each profile; computing the dynamic policy thresholds based on the trained ML model using one or more parameters from seasonality, trend, data 15 distribution, time, and geography; and storing the computed dynamic policy thresholds in the distributed data lake (210) for use during the execution of the created policy. 10. A system (102) for managing one or more policies for anomaly detection in real-time data, the system comprising: 20 a memory (204); and one or more processor(s) (206) configured to execute a set of instructions stored in the memory (204) for: receiving, by a load balancer module (212), a policy creation request from a user interface (206), wherein the policy creation 25 request includes one or more user-defined parameters for configuring multiple profiles associated with different use cases;
creating, by a workflow engine (214), a policy based on the one or more user-defined parameters received in the request; validating, by the workflow engine (214), the created policy and storing the created policy in a distributed data lake (210) upon the 5 validation; scheduling, by a scheduler (216) execution of the created policy at a predefined schedule; determining, by an Artificial Intelligence/Machine Learning (AI/ML) engine (220), dynamic policy thresholds for each data profile 10 associated with the created policy; and executing, by the workflow engine (214), the created policy based on the determined dynamic policy thresholds according to the predefined schedule for the anomaly detection. 11. 15 The system (102) as claimed in claim 10, wherein the one or more user-defined parameters include thresholds, scheduling frequency, and configurations for the multiple profiles to monitor the performance metrics for the different use cases. 12. 20 13. 25 The system (102) as claimed in claim 10, wherein the performance metrics include minutes of usage (MOU), attempted calls, and answered calls for a predefined duration. The system (102) as claimed in claim 10, wherein the load balancer module (212) is configured to: receive the policy creation request from the user interface (206); and transmit the policy creation request to the workflow engine to authenticate the received policy creation request. 39
14. The system (102) as claimed in claim 11, wherein a scheduling request from the workflow engine (214) is configured for maintaining: a scheduling frequency indicating how often the created policy should be executed, and 5 a set of recipient addresses for notifications related to policy execution status and breach alerts. 15. The system (102) as claimed in claim 14, wherein the scheduler is configured to: parse the scheduling request received from the workflow engine 10 (214); and create a response based on the parsed scheduling request. 16. The system (102) as claimed in claim 10, wherein the scheduler is configured to: 15 send a success notification to the workflow engine when the policy is scheduled successfully. 17. The system (102) as claimed in claim 10, wherein the workflow engine (214) is configured to: 20 send a policy creation failure notification to a user via the user interface (206) when a validation of the policy creation request fails. 18. The system (102) as claimed in claim 10, wherein a process of dynamically updating the dynamic policy thresholds for each data profile using the 25 AI/ML engine (220) comprises steps of: retrieving historical data for each data profile from the distributed data lake (210),
preprocessing the retrieved historical data by handling missing values, outliers, and data formatting, training a ML model using the pre-processed historical data to learn normal behavior patterns for each data profile, 5 computing the dynamic policy thresholds based on the trained ML model using one or more parameters from seasonality, trend, data distribution, time, and geography; and storing the computed dynamic policy thresholds in the distributed data lake (210) for use during the execution of the created policy. 10 19. A User Equipment (UE) (108) for managing one or more policies for anomaly detection in real-time data, the UE (108) comprising: a memory (204); one or more processors (202) configured to execute a set of instructions stored in the memory (204) to perform a method (600) as 15 claimed in claim 1.