Machine Learning Based Abstraction Layer For Device Performance Prediction In A Resource Constrained Environment
Abstract:
Device performance evaluation in resource constrained environments, such as Internet of Things (IoT), conventionally requires domain expert to select feature or performance parameters to be used for the device performance evaluation. The conventional approach is time consuming, cost inefficient and challenging when number of features to be analyzed is very high. Embodiments herein provide a machine learning based abstraction layer for device performance prediction in a resource constrained environment, such as IoT system. The method disclosed utilizes a first classifier in conjunction with a clustering technique to automatically identify relevant features for device performance evaluation of a device being monitored. Further, the method utilizes a second classifier in conjunction with an ensemble of regression models to predict state of the device as a success or a failure for a time instance of observation by analyzing the real time test data using the second classifier and the ensemble of regression models.
Get Free WhatsApp Updates!
Notices, Deadlines & Correspondence
Nirmal Building,
9th Floor,
Nariman Point,
Mumbai - 400021,
Maharashtra, India
Inventors
1. ROYCHOUDHURY, Abhishek
Tata Consultancy Services Limited, Block -1B, Eco Space, Plot No. IIF/12 (Old No. AA-II/BLK 3. I.T) Street 59 M. WIDE (R.O.W.) Road, New Town, Rajarhat, P.S. Rajarhat, Dist - N. 24 Parganas, Kolkata - 700160, West Bengal, India
2. DUTTA, Payal
Tata Consultancy Services Limited, Block -1B, Eco Space, Plot No. IIF/12 (Old No. AA-II/BLK 3. I.T) Street 59 M. WIDE (R.O.W.) Road, New Town, Rajarhat, P.S. Rajarhat, Dist - N. 24 Parganas, Kolkata - 700160, West Bengal, India
3. DAS, Abhisek
Tata Consultancy Services Limited, Block -1B, Eco Space, Plot No. IIF/12 (Old No. AA-II/BLK 3. I.T) Street 59 M. WIDE (R.O.W.) Road, New Town, Rajarhat, P.S. Rajarhat, Dist - N. 24 Parganas, Kolkata - 700160, West Bengal, India
4. CHATTOPADHYAY, Tanushyam
Tata Consultancy Services Limited, Block -1B, Eco Space, Plot No. IIF/12 (Old No. AA-II/BLK 3. I.T) Street 59 M. WIDE (R.O.W.) Road, New Town, Rajarhat, P.S. Rajarhat, Dist - N. 24 Parganas, Kolkata - 700160, West Bengal, India
5. MISRA, Prateep
Tata Consultancy Services Limited, Block -1B, Eco Space, Plot No. IIF/12 (Old No. AA-II/BLK 3. I.T) Street 59 M. WIDE (R.O.W.) Road, New Town, Rajarhat, P.S. Rajarhat, Dist - N. 24 Parganas, Kolkata - 700160, West Bengal, India
6. CHATTOPADHYAY, Dhiman
Tata Consultancy Services Limited, Block -1B, Eco Space, Plot No. IIF/12 (Old No. AA-II/BLK 3. I.T) Street 59 M. WIDE (R.O.W.) Road, New Town, Rajarhat, P.S. Rajarhat, Dist - N. 24 Parganas, Kolkata - 700160, West Bengal, India
7. NASKAR, Soumitra
Tata Consultancy Services Limited, Block -1B, Eco Space, Plot No. IIF/12 (Old No. AA-II/BLK 3. I.T) Street 59 M. WIDE (R.O.W.) Road, New Town, Rajarhat, P.S. Rajarhat, Dist - N. 24 Parganas, Kolkata - 700160, West Bengal, India
Specification
Claims:
1. A processor implemented method for device performance prediction in a resource constrained environment using a Machine Learning ( ML) based abstraction layer, the method comprising:
obtaining (202) data corresponding to a plurality of performance parameters of a device when the device is subjected to a load test in the resource constrained environment, wherein the data obtained is used for training a plurality of ML models of the ML based abstraction layer -, wherein the data obtained for first time from the device is annotated for the ML models that comprise a first classifier, a second classifier, and an ensemble of regression models;
assigning (204), using the first classifier, weight to each performance parameter among the plurality of performance parameters, based on relevance of the each performance parameter in the device performance prediction of the device to generate a plurality of weighted performance parameters;
selecting (206) a set of features from the plurality of weighted performance parameters by identifying relevant device specific features from the plurality of features based on the assigned weight of the each performance parameter;
constructing the second classifier utilizing the set of features and a training data set derived from the obtained data, wherein the second classifier classifies a state of the device as a failure or a success for a time instance identified for observation by analyzing a test data set recorded in real time corresponding to the plurality of performance parameters of the device; and
constructing the ensemble of regression models utilizing the set of features and the training data set derived from the obtained data, wherein the ensemble of regression models:
predicts values of unobtained performance parameters corresponding to the set of features, which are missing in the test data set due to resource constraints; and
provides the predicted values of the missing performance parameters to the second classifier during analysis of the test data to classify the state of device.
2. The method as claimed in claim 1, wherein selecting the set of features from the plurality of weighted performance parameters comprises:
clustering the plurality of weighted performance parameters generated by the first classifier into a plurality of clusters using an unsupervised clustering mechanism, wherein each cluster among the plurality of clusters is assigned a cluster weight;
selecting a cluster among the plurality of clusters that carries maximum cluster weight, wherein the selected cluster provides a unique combination of performance parameters comprising the device specific performance parameters, wherein the unique combination of performance parameters is used for predicting the state of the device at the time instance of observation when the device is performing under load condition; and
identifying each performance parameter, from the unique combination of performance parameters of the selected cluster, as a feature among the set of features for constructing the second classifier and each regression model.
3. The method as claimed in claim 1, wherein the values of the unobtained performance parameters predicted by the ensemble of regression models are derived based on a preset criteria and values of the unobtained performance parameters predicted by each regression model from the ensemble of regression models.
4. The method as claimed in 1, wherein the second classifier enables classification of the state of device using at least one experimental value corresponding to at least one performance parameter, among the weighted performance parameters selected as the set of features, even though the at least one experimental value is practically not implementable in the resource constrained environment due to the resource constraints.
5. A system (100) for device performance prediction in a resource constrained environment using a Machine Learning (ML) based abstraction layer (110), the system (100) comprising:
a memory (102) storing instructions;
one or more Input/Output (I/O) interfaces (106);
and one or more processors (104) coupled to the memory (102) via the one or more I/O interfaces (106), wherein the one or more processors (104) is configured by the instructions to:
obtain data corresponding to a plurality of performance parameters of a device when the device is subjected to a load test in the resource constrained environment, wherein the data obtained is used for training a plurality of ML models of the ML based abstraction layer (110) for the resource constrained environment, wherein the data obtained for first time from the device is annotated for the ML models that comprise a first classifier (113), a second classifier (114) and an ensemble of regression models (116);
assign, using the first classifier (112), weight to each performance parameter among the plurality of performance parameters based on relevance of the each performance parameter in the device performance prediction of the device to generate a plurality of weighted performance parameters;
select a set of features from the plurality of weighted performance parameters by identifying relevant device specific features from the plurality of features based on the assigned weight of the each performance parameter;
construct the second classifier (114) utilizing the set of features and a training data set derived from the data, wherein the second classifier classifies a state of the device as a failure or a success for a time instance identified for observation by analyzing a test data set recorded in real time corresponding to the plurality of performance parameters of the device; and
construct the ensemble of regression models (116) utilizing the set of features and the training data set derived from the obtained data, wherein the constructed ensemble of regression models:
predicts values of unobtained performance parameters, corresponding to the set of features, which are missing in the test data set due to resource constraints; and
provides the predicted values of the missing performance parameters to the second classifier during analysis of test data to classify the state of device.
6. The system (100) as claimed in claim 5, wherein the processor (104) is configured to select the set of features from the plurality of weighted performance parameters by:
clustering the plurality of weighted performance parameters generated by the first classifier into a plurality of clusters using an unsupervised clustering mechanism, wherein each cluster among the plurality of clusters is assigned a cluster weight;
selecting a cluster among the plurality of clusters that carries maximum cluster weight, wherein the selected cluster provides a unique combination of performance parameters comprising the device specific performance parameters for predicting the state of the device at the time instance of observation when the device is performing under load condition; and
identifying each performance parameter, from the unique combination of performance parameters of the selected cluster, as a feature among the set of features for constructing the second classifier and each regression model.
7. The system (100) as claimed in claim 5, wherein the processor (104) is configured to derive values of the unobtained performance parameters using the ensemble of regression models (116) by combining, based on a preset criteria, values of the unobtained performance parameters predicted by each regression model from the ensemble of regression models (116).
8. The system (100) as claimed in 5, wherein the processor (104) implementing the second classifier (114) is configured to classify the state of device using at least one experimental value corresponding to at least one performance parameters among the weighted performance parameters selected as the set of features, even though the at least one experimental value is practically not implementable in the resource constrained environment due to resource constraints.
, Description:FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003
COMPLETE SPECIFICATION
(See Section 10 and Rule 13)
Title of invention:
MACHINE LEARNING BASED ABSTRACTION LAYER FOR DEVICE PERFORMANCE PREDICTION IN A RESOURCE CONSTRAINED ENVIRONMENT
Applicant
Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th floor,
Nariman point, Mumbai 400021,
Maharashtra, India
The following specification particularly describes the invention and the manner in which it is to be performed.
TECHNICAL FIELD
[001] The disclosure herein generally relates to device performance predictions, and, more particularly to, Machine Learning (ML) based method and system for predicting device performance in a resource constrained environment.
BACKGROUND
[002] Machine Learning (ML) in system performance evaluation is a domain research and development. Internet of Things (IoT) environment is commonly used for system performance monitoring. ML has been combined with IoT for enhanced analysis, and for performance monitoring of end systems. However, the focus remains in bringing ML based processing for enhanced monitoring and performance analysis of one or more end systems.
[003] With IoT environments playing major role in remotely monitoring and evaluating end system performance, it is critical to ensure IoT system performance by monitoring all devices/entities of the IoT environment. This enables to maintain the downtime of entities in the IoT environment to minimum.
[004] Manual intervention plays a major role is exiting methods for monitoring and prediction of device performance of devices/ entities functioning in the IoT environment, which is a resource constrained environment. Here, a domain expert is expected to select feature or performance parameters to be used for device performance evaluation of the entity to be monitored. Thus, the traditional approach solely depends on the knowledge of the domain expert. Moreover, prediction of failure at an instance of observation will be a challenge for the observer as this requires analysis of the performance parameters, which is time consuming. In addition, when features to be selected or the performance parameters associated with the device increases, the domain expert based analysis is further challenging. This poses a risk of higher downtime of the IoT environment.
SUMMARY
[005] Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems. For example, in one aspect, there is provided a processor implemented method for device performance prediction in a resource constrained environment using a Machine Learning (ML) based abstraction layer. The method comprising, obtaining data corresponding to a plurality of performance parameters of a device when the device is subjected to a load test in the resource constrained environment. The data obtained is used for training a plurality of ML models of the ML based abstraction layer for the constrained environment, wherein the data obtained for first time from the device is annotated for the ML models that comprise a first classifier, a second classifier and an ensemble of regression models. Further method comprises assigning, using the first classifier, weight to each performance parameter among the plurality of performance parameters based on relevance of the each performance parameter in the device performance prediction of the device to generate a plurality of weighted performance parameters. Further comprises selecting a set of features from the plurality of weighted performance parameters by identifying relevant device specific features from the plurality of features based on the assigned weight of the each performance parameter. Further comprises constructing the second classifier utilizing the set of features and a training data set derived from the obtained data, wherein the second classifier classifies a state of the device as a failure or a success for a time instance identified for observation by analyzing a test data set recorded in real time corresponding to the plurality of performance parameters of the device. Furthermore, comprises constructing the ensemble of regression models utilizing the set of features and the training data set derived from the obtained data, wherein the ensemble of regression models predicts values of unobtained performance parameters, corresponding to the set of features, which are missing in the test data set due to resource constraints and provides the predicted values of the missing performance parameters to the second classifier during analysis of the test data to classify the state of device.
[006] In another aspect, there is provided a system for device performance prediction in a resource constrained environment using a Machine Learning ( ML) based abstraction layer, the system comprising a memory storing instructions; one or more Input/Output (I/O) interfaces; and one or more processors coupled to the memory via the one or more I/O interfaces, wherein the processor is configured by the instructions to obtain data corresponding to a plurality of performance parameters of a device when the device is subjected to a load test in the resource constrained environment. The data obtained is used for training a plurality of ML models of the ML based abstraction layer for the constrained environment, wherein the data obtained for first time from the device is annotated for the ML models that comprise a first classifier, a second classifier and an ensemble of regression models. Further configured to assign, using the first classifier, weight to each performance parameter among the plurality of performance parameters based on relevance of the each performance parameter in the device performance prediction of the device to generate a plurality of weighted performance parameters. Further configured to select a set of features from the plurality of weighted performance parameters by identifying relevant device specific features from the plurality of features based on the assigned weight of the each performance parameter. Further configured to construct the second classifier utilizing the set of features and a training data set derived from the obtained data, wherein the second classifier classifies a state of the device as a failure or a success for a time instance identified for observation by analyzing a test data set recorded in real time corresponding to the plurality of performance parameters of the device. Furthermore configured to construct the ensemble of regression models utilizing the set of features and the training data set derived from the obtained data, wherein the ensemble of regression models predicts values of unobtained performance parameters, corresponding to the set of features, which are missing in the test data set due to resource constraints and provides the predicted values of the missing performance parameters to the second classifier during analysis of test data to classify the state of device.
[007] In yet another aspect, there are provided one or more non-transitory machine readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause: obtaining data corresponding to a plurality of performance parameters of a device when the device is subjected to a load test in a resource constrained environment. The data obtained is used for training a plurality of ML models of the ML based abstraction layer for the resource constrained environment, wherein the data obtained for first time from the device is annotated for the ML models that comprise a first classifier, a second classifier and an ensemble of regression models. Further comprises assigning, using the first classifier, weight to each performance parameter among the plurality of performance parameters based on relevance of the each performance parameter in the device performance prediction of the device to generate a plurality of weighted performance parameters. Further comprises selecting a set of features from the plurality of weighted performance parameters by identifying relevant device specific features from the plurality of features based on the assigned weight of the each performance parameter. Further comprises constructing the second classifier utilizing the set of features and a training data set derived from the obtained data, wherein the second classifier classifies a state of the device as a failure or a success for a time instance identified for observation by analyzing a test data set recorded in real time corresponding to the plurality of performance parameters of the device. Furthermore, comprises constructing the ensemble of regression models utilizing the set of features and the training data set derived from the obtained data, wherein the ensemble of regression models predicts values of unobtained performance parameters, corresponding to the set of features, which are missing in the test data set due to resource constraints and provides the predicted values of the missing performance parameters to the second classifier during analysis of the test data to classify the state of device.
[008] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
BRIEF DESCRIPTION OF THE DRAWINGS
[009] The accompanying drawings, which are incorporated in and constitute a component of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles:
[010] FIG. 1 illustrates an exemplary block diagram of a system for predicting device performance in a resource constrained environment using Machine Learning (ML) based abstraction layer, in accordance with an embodiment of the present disclosure.
[011] FIG. 2A and FIG. 2B illustrate an exemplary flow diagram of a method for predicting the device performance in the resource constrained environment using Machine Learning (ML) based abstraction layer and the system of FIG. 1, in accordance with an embodiment of the present disclosure.
[012] FIG. 3 is a graphical representation illustrating device specific datasets used by the system of FIG. 1, in accordance with an embodiment of the present disclosure.
DETAILED DESCRIPTION OF EMBODIMENTS
[013] Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the spirit and scope of the disclosed embodiments. It is intended that the following detailed description be considered as exemplary only, with the true scope and spirit being indicated by the following claims.
[014] As mentioned, Internet of Things (IoT) system, alternatively referred as IoT environment, is a typical example of resource constrained environment. The IoT system deals with huge number of devices or entities out of which most of the devices are edge devices. IoT generates high volume of machine generated data. The data processing is done either at a cloud end where service is hosted or at an edge end where some subset of processing is done to minimize the data traffic between the devices and the cloud to remove latency. These devices must not fail during sudden abnormally high volume of input data traffic. One major challenge in device reliability evaluation or device performance evaluation of devices in the IoT system arises as elastically provisioning of resources is difficult for the constrained devices, typically the edge devices. On the other hand, lightweight alternative of full machine virtualization in IoT environment is containerization, where applications can run on any suitable physical machine without any dependencies. Containerization of software is normal practice nowadays and is used in edge services too. However, with varying load it is important to track the resource utilization of the container as it can detect (indicate?) possible failures. Prior knowledge of a device’s resource usage under different varying load can provide an insight into model behavior of the device under a proposed load and possible gaps between demand of resources and instantaneous resource availability. The resource unavailability can lead to degraded performance or denial performance for the edge services and the limitation of resources cripple the performance of the device and the IoT environment as whole. There is no concrete study to establish the detection of failures of the devices or entities of the IoT system with relation to device load in a constrained environment.
[015] State of the art analysis in this domain depicts that most of the works have been done for facilitating dynamic scaling of physical infrastructure using heuristic algorithms. Existing solutions are able to build a performance model and predict performance metrics after scaling, but unable to control the load when it has surpassed the resource capacity or predict device failures in constrained environment. Some existing ML based solutions offer infrastructure scaling approach, which mostly rely on reinforcement learning to figure out the optimized RDBMS scaling action that requires a huge state space for training the system models. Moreover, the existing solutions have hardly executed any experiments in a micro service architecture under containerized constrained environment. Some existing ML based methods attempted on deploying IoT based analytic solutions but not on evaluation of device performance. In previous studies, for multi-tier architecture dynamic resource allocation has been done by taking resource from other healthy tiers to facilitate the bottleneck tier. Often, the above existing approach leads to performance degradation of the healthy one to remove the bottleneck of another service.
[016] Embodiments of the present disclosure provide a method and system for predicting device failure in the resource constrained environment using Machine Learning (ML) based abstraction layer. The ML is applied to provide insight from an available performance test data to detect relationship between device’s resource crunch and associated performance parameters and further classifies the (device?) behavior in similar constrained scenarios. The method disclosed herein provides automation in device performance prediction, providing instant insight on device failure under constrained environment using ML. The ML based approach used minimizes manual intervention in selection of performance parameters and reduces overall cost of device performance monitoring.
[017] Embodiments herein provide ML based abstraction layer for device performance prediction in a resource constrained environment, such as the IoT system. The method disclosed utilizes a first classifier in conjunction with a clustering technique to automatically identify the relevant features for device performance evaluation of a device being monitored. Further, the method utilizes a second classifier in conjunction with an ensemble of regression models to predict state of the device as a success or a failure for a time instance of observation by analyzing the real time test data using the second classifier and the ensemble of regression models.
[018] Internet of Things (IoT) being one example of resource constrained environment, embodiments herein describe a method and system for predicting device performance in constrained environment using Machine Learning (ML) based abstraction layer, with the IoT environment or IoT system as an example. However, it is to be understood that the method and system disclosed is applicable in any resource constraint environment with or without minimal modifications.
[019] Referring now to the drawings, and more particularly to FIGS. 1 through 3, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and/or method.
[020] FIG. 1 illustrates an exemplary block diagram of a system 100 for predicting device performance in the resource constrained environment using a Machine Learning (ML) based abstraction layer 110, in accordance with an embodiment of the present disclosure.
[021] In an embodiment, the system 100 includes one or more processors 104, communication interface device(s) or input/output (I/O) interface(s) 106, and one or more data storage devices or memory 102 operatively coupled to the one or more processors 104. The one or more processors 104 may be one or more software processing modules and/or hardware processors. In an embodiment, the hardware processors can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor(s) is configured to fetch and execute computer-readable instructions stored in the memory. In an embodiment, the system 100 can be implemented in a variety of computing systems, such as laptop computers, notebooks, hand-held devices, workstations, mainframe computers, servers, a network cloud and the like.
[022] The I/O interface device(s) 106 can include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like and can facilitate multiple communications within a wide variety of networks N/W and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. In an embodiment, the I/O interface device(s) can include one or more ports for connecting a number of devices to one another or to another server. The I/O interface 106, through the ports is configured to receive input such as data corresponding to a plurality of performance parameters of the device, wherein the device is monitored for performance evaluation to predict device failure. The data includes both a training dataset and a test data.
[023] The memory 102 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. In an embodiment a plurality of modules 108 can be stored in the memory 102, wherein the modules 108 may comprise the ML based abstraction layer 110, an application layer 118 which form layered architecture for the device failure prediction provided by system 100. The ML based abstraction layer 110 includes a first classifier 112, a second classifier 114 and an ensemble of regression models 116. The application layer 118 includes gateway layers 120, logic layers 122 and a data layer 124.
[024] The ML based abstraction layer 110 analyzes device metrics or device performance parameters to predict the device failure at a time instance. The gateway layer 118 runs a lightweight protocol stack and ensures secure communication. The logic layer 120 provides geo-spatial computation on sensor data flowing into the device being monitored, and data housekeeping of local data store and syncing local data store with a cloud. Further, data layer 122 stores the sensor data being captured.
[025] The modules 108, when executed by the processors (s) 104 are configured to perform device performance prediction in the resource constrained environment using the ML based abstraction layer 110. The functions of the modules 108 are explained in conjunction with a method 200 of FIG. 2A and 2B. The memory 102 may further comprise information pertaining to input(s)/output(s) of each step performed by the modules 108 of the system 100 and methods of the present disclosure.
[026] FIG. 2A and FIG. 2B illustrate an exemplary flow diagram of the method 200 for predicting device performance in the resource constrained environment using the ML based abstraction layer 110 of the system of FIG. 1, in accordance with an embodiment of the present disclosure. In an embodiment, the system 100 comprises one or more data storage devices or the memory 102 operatively coupled to the one or more processors 104 and is configured to store instructions for execution of steps of the method 200 by the one or more processors (alternatively referred as processor(s)) 104 in conjunction with various modules of the modules 108. The steps of the method 200 of the present disclosure will now be explained with reference to the components or blocks of the system 100 as depicted in FIG. 1 and the steps of flow diagram as depicted in FIG. 2. Although process steps, method steps, techniques or the like may be described in a sequential order, such processes, methods and techniques may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes described herein may be performed in any order practical. Further, some steps may be performed simultaneously.
[027] For a device whose performance is to be monitored to predict device state as one of a failure or a success, a plurality of performance parameters of the device being monitored are computed using known tools and techniques. The tools capture the plurality of performance parameters such as number of incoming threads, CPU percentage, memory usage in megabytes (MB), ratio of memory usage and the limit of memory, memory usage in percentage, network input, and network output and block input in MB. In conventional methods, all these parameters are observed and selected by the domain expert from field of performance evaluation, while the method 200 disclosed herein automates the feature selection process, elimination manual intervention. The feature selection is intelligently and automatically performed by the modules of the ML based abstraction layer 110.
[028] Thus, with one or more exiting tools, once the computed performance parameters are available, then at step 202 of the method 200, the modules 108 when executed by the processors (s) 104 are configured to obtain data corresponding to a plurality of performance parameters of the device when the device is subjected to a load test in the constrained environment. Data obtained for example devices is explained in conjunction with the experimental results of FIG. 3, wherein example devices of the IoT system being monitored include a cloud (cloud server) and an edge device. The data, so obtained is used for training a plurality of ML models of the ML based abstraction layer 110 for the constrained environment. The data obtained or captured for first time from the device is annotated for the ML models that comprise a first classifier, a second classifier and an ensemble of regression models.
[029] The data obtained, for example can be obtained using VMstat™ or DockerStat™ and manually annotated with success or failure based on the meter output which is used to benchmark the constrained environment. The obtained data set is highly unbalanced to non-faulty data. This gives rise to the problem of biasness during training phase. So the input data is converted to a balanced one by using a down-sampling method, further explained in conjunction with FIG. 3 based on experimental results. Thus data obtained from multiple sensors containing parameters of performance is converted to the balanced data, where the cardinality of the members of both the faulty and non-faulty classes are almost equal.
[030] At step 204 of the method 200, the first classifier 112, when executed by the processors (s) 104 is configured to assign weight to each performance parameter among the plurality of performance parameters based on relevance of each performance parameter in the device performance prediction of the device. This generates a plurality of weighted performance parameters at the output of the first classifier. The first classifier, can be implemented using known classifiers such as Support Vector Machine (SVM), Random forest and the like, which are trained using the training data obtained from the data associated with the performance parameters.
[031] At step 206 of the method 200, the modules 108, when executed by the processors (s) 104 are configured to obtain a set of features from the plurality of weighted performance parameters by identifying relevant device specific features from the plurality of features. The set of features are obtained based on assigned weight of each performance parameter. Obtaining or deriving the set of features from the plurality of weighted performance parameters for building or constructing the second classifier and each regression model among the ensemble of regression models comprises clustering the plurality of weighted performance parameters generated by the first classifier into a plurality of clusters using an unsupervised clustering mechanism. Each cluster among the plurality of clusters is assigned a cluster weight. Once the cluster weights are assigned, a cluster among the plurality of clusters is selected, such that the selected cluster carries maximum cluster weight. The selected cluster comprises a unique combination of performance parameters from the plurality of parameters that are device specific performance parameters for predicting the state of the device at the time instance of observation when the device is performing under load conditions. Further, each performance parameter, from the unique combination of performance parameters of the selected cluster, is identified as a feature among the set of features for constructing the second classifier 114 and each regression model.
[032] At step 208 of the method 200, the modules 108, when executed by the processors (s) 104 are configured to construct the second classifier utilizing the set of features and a training data set derived from the obtained data. The construction of the second classifier 114 includes running a HPO and obtaining best classifier, as the second classifier. Further, the identified best classifier is trained using the set of selected features + ground truth (training data corresponding to input and output of the classifier). The trained second classifier 114 provides a model file at output depicting the hyper-plane of classification. The classifier selection is further explained in conjunction with the experimental results of the FIG. 3. Thus constructed second classifier 114 classifies the state of the device as failure or success for a time instance identified for observation. The second classifier 114 predicts the state by analyzing a test data set recorded in real time corresponding to the plurality of performance parameters of the device.
[033] In another scenario, an experimental value of a performance parameter is available, for example say memory size. Further, considering the experimental value, the device state needs to be predicted. Practically adding the memory with the size as provided by the experimental value may not be possible due to system implementation/ infrastructure constraints. Thus, with traditional process such scenarios can never be tested. However, herein, the method disclosed enables to predict device state, wherein the experimental value of the memory can be provided as input feature to the second classifier 114 and prediction the device state can be observed. Thus, the second classifier 114 provides a simulation environment.
[034] At step 210 of the method 200, the modules 108, when executed by the processors (s) 104 are configured to construct the ensemble of regression models 116 utilizing the set of features and the training data set derived from the obtained data. The constructed ensemble of regression models 116 predicts values of unobtained performance parameters, corresponding to the set of features derived or obtained in the above step. The values of performance parameters may not be obtained or are missing in the test data set due to resource constraints (unavailability of the respective resource while capturing test data). Thus, the ensemble of regression models (116) provides the predicted values of the missing performance parameters to the second classifier during analysis of the test data to classify the state of device.
[035] For example, suppose in real time, for test data, out of the set of features say (m) required by the second classifier 114 only n performance parameters, where n
Documents
Orders
Section
Controller
Decision Date
Application Documents
#
Name
Date
1
201921010959-IntimationOfGrant07-12-2023.pdf
2023-12-07
1
201921010959-STATEMENT OF UNDERTAKING (FORM 3) [20-03-2019(online)].pdf
2019-03-20
2
201921010959-PatentCertificate07-12-2023.pdf
2023-12-07
2
201921010959-REQUEST FOR EXAMINATION (FORM-18) [20-03-2019(online)].pdf
2019-03-20
3
201921010959-PETITION UNDER RULE 137 [21-11-2023(online)].pdf