FIELD OF INVENTION

[0001] The present disclosure relates to a wireless communication system, and more specifically relates to a machine learning (ML) orchestrator architecture and method for using thereof for enabling real-time capabilities at edge nodes in the wireless communication system. This specification is based on the Indian patent application with application number 202011040411 filed on 18th September 2020
BACKGROUND OF INVENTION
[0002] A machine learning (ML) orchestrator platform manages dataflow and workflow automation required for machine learning requirements for a fifth generation (5G) architecture. In the 5G architecture, separate processing is required for data preparation, model creation, and deployment in the machine learning orchestrator platform that leads to high latency and the requirement of separate resources. In other words, the machine learning orchestrator platform includes multiple separate modules to perform data preparation, model creation and deployment, and each module requires separate network or infrastructure resources that results in consuming a lot of resources.
[0003] Thus, it is desired to address the above-mentioned disadvantages or other shortcomings or at least provide a useful alternative.
OBJECT OF INVENTION
[0004] The principal object of the invention herein is to provide a ML orchestrator architecture and method for using the ML orchestrator architecture for enabling real time capabilities at edge

nodes in a wireless communication system without consuming much resource.
SUMMARY
[0005] Accordingly, the present invention discloses a machine learning (ML) orchestrator architecture that enables real time capabilities at edge nodes in a wireless communication system without consuming much resources. The ML orchestrator architecture allows optimized use of infrastructure resources for data preparation, model creation and deployment processes leading to swift completion of these processes that enable real time capabilities at the edge nodes.
[0006] In the ML orchestrator architecture, the method can be used for possessing a capability of data pipeline creation, machine learning model formulation and deployment across multiple edge nodes in the wireless communication system. The data pipeline creation consists of data cleaning, data ingestion, data analysis, data transformation, data validation, and data splitting.
[0007] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.
BRIEF DESCRIPTION OF FIGURES

[0008] The method and system are illustrated in the accompanying drawings, throughout which like reference letters indicate corresponding parts in the various figures. The embodiments herein will be better understood from the following description with reference to the drawings, in which:
[0009] FIG. 1 illustrates a high-level overview of a model training using a machine learning (ML) orchestrator platform and ML training hosts, according to an example of the present disclosure;
[0010] FIG. 2 is an example illustration in which a data preparation layer, a model creation layer and a roll out/deployment layer are processed together in a machine learning (ML) orchestrator environment;
[0011] FIG. 3 is an example illustration in which cleansing of the data is explained;
[0012] FIG. 4 is an example illustration in which the data splitting used for training and testing is explained;
[0013] FIG. 5 is an example illustration in which a first version of the ML model is built depending on a number of iterations, validation and problem statement;
[0014] FIG. 6 is an example illustration in which a model validation is explained;
[0015] FIG. 7 is an example illustration in which the model creation is explained; and
[0016] FIG. 8 is an example illustration in which main four stages for building the ML model is explained.

DETAILED DESCRIPTION OF INVENTION
[0017] In the following detailed description of embodiments of the invention, numerous specific details are set forth in order to provide a thorough understanding of the embodiment of invention. However, it will be obvious to a person skilled in the art that the embodiments of the invention may be practiced with or without these specific details. In other instances, well known methods, procedures, and components have not been described in detail so as not to unnecessarily obscure aspects of the embodiments of the invention.
[0018] Furthermore, it will be clear that the invention is not limited to these embodiments only. Numerous modifications, changes, variations, substitutions, and equivalents will be apparent to those skilled in the art, without parting from the scope of the invention.
[0019] The methods and processes described herein may have fewer or additional steps or states and the steps or states may be performed in a different order. Not all steps or states need to be reached. The methods and processes described herein may be embodied in, and fully or partially automated via, software code modules executed by one or more general purpose computers. The code modules may be stored in any type of computer-readable medium or other computer storage device. Some or all of the methods may alternatively be embodied in whole or in part in specialized computer hardware.
[0020] The results of the disclosed methods may be stored in any type of computer data repository, such as relational databases and flat file systems that use volatile and/or non-volatile memory (e.g., magnetic disk storage, optical storage, EEPROM and/or solid-state RAM).

[0021] The various illustrative logical blocks, modules, routines, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. The described functionality can be implemented in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosure.
[0022] Moreover, the various illustrative logical blocks and modules described in connection with the embodiments disclosed herein can be implemented or performed by a machine, such as a general purpose processor device, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components or any combination thereof designed to perform the functions described herein. A general-purpose processor device can be a microprocessor, but in the alternative, the processor device can be a controller, microcontroller, or state machine, combinations of the same, or the like. A processor device can include electrical circuitry configured to process computer-executable instructions. In another embodiment, a processor device includes an FPGA or other programmable device that performs logic operations without processing computer-executable instructions. A processor device can

also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Although described herein primarily with respect to digital technology, a processor device may also include primarily analog components. A computing environment can include any type of computer system, including, but not limited to, a computer system based on a microprocessor, a mainframe computer, a digital signal processor, a portable computing device, a device controller, or a computational engine within an appliance, to name a few.
[0023] The elements of a method, process, routine, or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor device, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of a non-transitory computer-readable storage medium. An exemplary storage medium can be coupled to the processor device such that the processor device can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor device. The processor device and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor device and the storage medium can reside as discrete components in a user terminal.
[0024] Conditional language used herein, such as, among others, "can," "may," "might," "may," "e.g.," and the like, unless specifically stated otherwise, or otherwise understood within the

context as used, is generally intended to convey that certain embodiment include, while other embodiments do not include, certain features, elements and/or steps. Thus, such conditional language is not generally intended to imply that features, elements and/or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without other input or prompting, whether these features, elements and/or steps are included or are to be performed in any particular embodiment. The terms "comprising," "including," "having," and the like are synonymous and are used inclusively, in an open-ended fashion, and do not exclude additional elements, features, acts, operations, and so forth. Also, the term "or" is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term "or" means one, some, or all of the elements in the list.
[0025] Disjunctive language such as the phrase "at least one of X, Y, Z," unless specifically stated otherwise, is otherwise understood with the context as used in general to present that an item, term, etc., may be either X, Y, or Z, or any combination thereof (e.g., X, Y, and/or Z). Thus, such disjunctive language is not generally intended to, and should not, imply that certain embodiments require at least one of X, at least one of Y, or at least one of Z to each be present.
[0026] The accompanying drawings are used to help easily understand various technical features and it should be understood that the embodiments presented herein are not limited by the accompanying drawings. As such, the present disclosure should be construed to extend to any alterations, equivalents and substitutes in addition to those which are particularly set out in the accompanying

drawings. Although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are generally only used to distinguish one element from another.
[0027] Accordingly, the present invention discloses a machine learning (ML) orchestrator architecture that enables real time capabilities at edge nodes in a wireless communication system. The ML orchestrator architecture allows optimized use of infrastructure resources for data preparation, model creation and deployment processes leading to swift completion of these processes that enables real time capabilities at the edge nodes in the wireless communication system.
[0028] Unlike conventional methods and platforms, the proposed machine learning orchestrator platform brings data preparation, model creation and deployment in a single module that is deployed across multiple edge nodes in the wireless communication system. The ML orchestrator platform allows optimized use of infrastructure resources for data preparation, model creation and deployment processes leading to swift completion of these processes that enable real time capabilities at the edge nodes in the wireless communication system. The infrastructure resources include resources such as compute, storage and network resources, particularly virtual machines, containers, network bandwidth, QoS (quality of service) alignment, storage etc.
[0029] Referring now to the drawings, and more particularly to FIGS. 1 through 8, there are shown preferred embodiments.
[0030] FIG. 1 illustrates a high-level overview of a model training using a machine learning (ML) orchestrator platform and ML training hosts. The ML training hosts can be, for example, but not

limited to acumos and Kubeflow. In the ML orchestrator platform, the operations basically start with a data lake. The data lake collects various raw data (e.g., performance management, configuration management etc.,) from network nodes. Further, the raw data needs to be prepared, for passing on to the ML training hosts, for data preparation. The data preparation has various stages and components to accomplish it (the various stages and components are explained in detail in conjunction with FIG. 2). After the data preparation, a training data set can move on to the ML training hosts, at which a model can be trained using various ML/deep learning (DL) techniques. The model training comprises different stages (the various stages and components are explained in detail in conjunction with FIG. 2). Once the model is trained, an ML model catalogue is deployed on to the ML orchestrator platform, so that end users can use any kind of ML models they require and can perform the actions accordingly.
[0031] FIG. 2 is an example illustration in which a data preparation layer, a model creation layer and a roll out/deployment layer are processed together in a machine learning (ML) orchestrator environment, according to an example of the present disclosure.
[0032] The data preparation layer consists of following stages such as data cleaning stage, data ingestion stage, data analysis stage, data transformation stage, data validation stage, and data splitting stage.
[0033] Data Preparation layer: The data preparation is an important aspect of ML. At times, the system does not have control over the type of data. The system receives the data which may or may not be arranged/structured.

[0034] Structured Data: The structured data are highly organized following predefined format to set some specific structure and hence they are addressable for analysis. Example of structured data is a data stored in a relational database where the data has been formatted into predefined format such as credit card numbers or address.
[0035] Unstructured Data: The unstructured data are not organized and do not follow predefined format and consist of internal structure but are not structured using a predefined format. The unstructured data may be textual, non-textual, human or machine generated unstructured data. Examples of such data are emails, social media data etc.
[0036] If the data is structured, then the data can be forwarded for further operation but if the data is unstructured then the data needs to be made structured and uniform before analysis because the unstructured data is not suitable for analysis. The data preparation is more important stage before using the data to create any machine learning model. Data preparation is a technique to collect raw data from various sources, cleaning the raw data and then transforming the raw data to the compatible structure or format and then digging out the insights from it. For achieving the better results from any ML model or DL model, the format of the data needs to be in the proper format.
[0037] Further, the data preparation is necessary because of the presence of the unstructured format of real-world data, where the real-world data may consist of missing values, outliers, noise data, inconsistent data etc. Further, there could be many reasons for all these kinds of data such as data is not being collected continuously, a

mistake in data entry, containing mistakes in codes or names i.e., violation of data constraints and much more.
[0038] Therefore, there are various ways to deal with such situations while preparing data. If these preparations are not done properly, it can cause huge inconsistency in the data hence, it is better to get rid of all these inconsistencies, redundancy in the early stage of building the ML model or DL model. Therefore, it is important to understand, summarize, and analyze the contents of a dataset, to investigate a specific question or to prepare a more advanced modeling. Thus, to handle the raw data, the data preparation layer is used.
[0039] Data cleaning stage: The data cleaning techniques or data cleansing techniques are not only an essential part of the data science process but it's also the most time-consuming part. Without well-structured data, the system can't have a good machine learning model. A well-cleaned data only can-do justice to the model, otherwise a model can be trained but it will not give accurate predictions. Hence exploring and cleaning of the data in building machine learning models is the first prior task as it gives foundation to the model. Further, without the data cleaning process, the model training will not be as efficient as the user of the system want them to be.
[0040] If the user of the system has performed the cleansing of the data well, the user of the system can get desired results even with a very simple algorithm, which can prove very beneficial at times. Different kinds of data require different kinds of cleaning. However, there is also a standard approach to achieve this part as shown in the FIG. 3.

a. Removing Unwanted observations: This step involves the removal of
unwanted observations from the datasets such as duplicate or redundant
observation which occurs mostly at the time of data collection or when
the user of the system combines datasets from multiple sources. Such
observations are any type of data that is of no use and can be removed
directly.
b. Fixing structural errors: The structural error in the datasets refers to
those errors which occur while measuring, transferring the data or
similar kinds of situations such errors include typographical errors,
mislabeled classes etc.
c. Filtering unwanted outliers: The dataset might contain some values
which seems completely extreme from the rest of the values present in
the dataset. Such extreme values can cause more problems to the ML/DL
models. However, the user of the system should have a legitimate reason
to remove an outlier. Sometimes the removal of outliers improves model
performance, sometimes it doesn't.
d. Handling missing data: The concept of handling the missing values in
the dataset is vital in order to manage the data successfully.
Understanding the nature of missing data is crucial in determining what
solutions can be imputed to overcome the lack of data. The data can be
missing in these ways: Missing Completely at Random (MCAR),
Missing at Random (MAR), Not Missing at Random (NMAR) and these
missing values can be treated by various ways like mean, median, mode
imputation method, multivariate Imputation by chained equations
(MICE) etc.
[0041] Data ingestion stage: The data ingestion refers to the process of importing, transferring, loading and processing of both structured and unstructured data for later use or storage in databases. The destination is typically a data warehouse, data mart, database or

a document store. The data ingestion involves connectivity to different data sources, such as web logs, RDBMS, application-logs, streaming data etc. Since the data comes from different places, the data needs to be cleansed and transformed in a way that allows the user to analyze the data together from other sources. Otherwise, data would be like a bunch of puzzle pieces that don't fit together. One can ingest data in real time, in batches, or in a combination of the two (this is called lambda architecture).
[0042] A batch data is an efficient way of processing large volumes of data where a large set of transactions collect over time. The data is collected, entered, and processed and then batch results are produced with tools like Hadoop. Real-time data, on the other hand, demands continual input, process and output of data. The data must be processed in a small time or period (or near real-time). It is the process of moving data into big data systems as and when they arrive.
[0043] One can also ingest data using a lambda architecture.
This approach attempts to balance the benefits of batch and real-time
modes by using batch processing to provide comprehensive views of
batch data, while also using real-time processing to provide views of
time-sensitive data. Further, collecting data from multiple sources
and in different forms for business use cases showcases a variety of
challenges in data ingestion. These challenges can include: multiple
source ingestion, managing streaming/real-time data, Insecure, speed
of ingestion etc.
a. Multiple Source Ingestion: Organizations can generate huge amounts of
data during any given product or service life cycle. This is a challenge
when the user of the system has to manage and decide which data to
include in a data warehouse.

b. Insecure: Security is always an issue in case of moving data. Data is
often staged at various steps during ingestion, which makes it difficult
to meet compliance standards throughout the process.
c. Speed of ingestion: Different Data Sources transport data at different
speeds which draws attention on data ingestion process.
[0044] Data analysis stage: The data analysis is an overarching science that encompasses the complete management of data. The data analysis refers to the process of examining, transforming and arranging a given dataset in such a way that its individual parts can be studied and extract insights from it. The data analysis allows for the evaluation of data through analytical and logical reasoning to lead to an outcome or conclusion within a stipulated context. The approach the user take to data analysis mostly depends on the type of the availability of the data for analysis and its purpose of the analysis. The data analytics is broken down into four basic types.
[0045] Descriptive analytics: The main objective of descriptive analytics is to find out the reasons behind precious success or failure in the past. The descriptive analytics concentrate on historical data, providing the context that is vital for understanding information and numbers. The descriptive analytics describes the use of a range of historic data to draw comparisons. The vast majority of big data analytics used by organizations falls into the category of descriptive analytics.
[0046] Diagnostic analytics: It is a type of analytics that gets into the root cause analysis, data detection and investigation which focuses on determining the factors and focuses more on why something happened. This involves more diverse data inputs and a bit of hypothesizing. The diagnostic analytics gives in-depth insights

into a particular problem, at this stage, historical data can be measured against other data to answer the question of why something happened.
[0047] Predictive analytics: Analyzing past data patterns and trends can accurately inform a business about what could happen in the future. The predictive analytics helps predict the likelihood of a future outcome by using various statistical machine learning algorithms. It is a wide field of techniques that share a common goal of predicting future behavior. Choosing the right prediction modeling method is perhaps the most important step in the process, because predictive models are the driving force behind predictive analytics.
[0048] Prescriptive analytics: After using predictive analytics to determine what could happen based on data trends, the next step is to figure out what should be done moving forward.
[0049] Data Transformation stage: Data transformation makes data useful and transforming data will ensure maximum data quality, which is imperative to gain accurate analysis, leading to valuable insights that will eventually empower data-driven decisions. The data transformation is an information technology (IT) process where data scientists analyze, review and convert data from one type of format to another. It is a process that is essential to a business, especially when there is a need to integrate data from different databases, integrate data more efficiently or change it to be able to store it securely. The most common data transformations are converting raw data into a clean and usable form, converting data types, removing duplicate data, and enriching the data to benefit an organization. The transformed data is usable, accessible, and secure to benefit a variety of purposes. In addition, data transformation makes it possible for organizations to transform data from a storage

database to the cloud to keep information moving. Data element to element mapping can be complicated and requires complex transformations that require lots of rules. The user of the system needs to transform data to make it compatible with other data, move it to another location, join it with other data and add it to other data parts.
[0050] There are some common data transformations that can be performed like:
1. Removing unused and repeated columns: Removing those columns
which are not necessary can speed up the learning of training
models.
2. Change data types: The format of individual columns and rows will
impact analysis performed on a dataset. The user can't perform
mathematical calculations on a string (text formatted data) or vice-
versa, that's why it is important to change the data type into a
compatible format.
3. Remove String Formatting and Non-Alphanumeric Characters:
Removing characters like line breaks, carriage returns, white spaces
at the beginning and end of values, currency symbols, etc. this
approach helps an algorithm better digest the data.
4. Convert Categorical Data to Numerical: Many machine learning models require categorical data to be in a numerical format, requiring conversion of values such as yes or no to 1 or 0.
5. Convert Timestamps: Timestamps are in all types of formats. It is a good idea to define data/time format and convert all timestamps to the defined format.
[0051] The transforming data for analysis can be challenging based on the growing volume, variety, and velocity of big data, but it is worth it as businesses continue to use data and insights to innovate

and grow. This challenge will need to be overcome to unlock the potential of the data.
[0052] Data Validation stage: If the data isn't accurate from the start, the model results definitely won't be accurate either. That's why it's necessary to verify and validate the data before it is used. The data validation is an essential part of any data handling task. The data validation performs after training and before testing on the data. After training the model, it's important to validate the results to achieve satisfactory results. Once the validation is completed, the user can move to test the model to predict and evaluate the performance. While the data validation is a critical step in any data workflow, it's often skipped over. It may seem as if data validation is a step that slows down the pace of work, however, it is essential because it will help the user to create the best results possible.
[0053] Validating the accuracy, clarity, and details of the data is necessary to mitigate any project defects. Without validating the data, the user runs the risk of biasing decisions on data with imperfections that are not accurately representative of the situation at hand. Both the structure and content of the data files will dictate what exactly the user can do with the data. Using validation rules to cleanse the data before use helps to mitigate "garbage in = garbage out" scenarios. Ensuring the integrity of the data helps to ensure the legitimacy of your conclusions.
[0054] Different types of validation can be performed depending on destination constraints or objectives. Data validation is a form of data cleansing. There is no single validation method that works in all scenarios. It is important to understand if the user of the system are dealing with groups, time-indexed data, or if the user are

leaking data in the validation procedure. There are different data validation techniques available, some of them are mentioned below.
1. Train/test split,
2. k-Fold Cross-Validation,
3. Leave-one-out Cross-Validation,
4. Leave-one-group-out Cross-Validation,
5. Nested Cross-Validation,
6. Time-series Cross-Validation,
7. Wilcoxon signed-rank test,
8. McNemar'stest,
9. 5x2CV paired t-test, and
10. 5x2CV combined F test
[0055] Data Splitting stage: The data splitting has a significant impact on the performance of the final model. It is standard in ML to split data into training and test sets. The reason for this is very straightforward, if the user tries and evaluates the system on data, we have trained it on, we are doing something unrealistic.
[0056] The proposed method/platform uses the training data to basically train the proposed model. The training data is a complete set of feature variables/independent variable and target variable/dependent variable. So that the proposed model is able to learn the value of target variable on a particular set of feature variables. After applying the training data, it is the time to test how much the model has learned from the training data. Here, testing comes into the picture and lets us know how well the model has trained. In an example, the proposed model is tested by applying the feature variables to model in order to get the value of the target variable predicted by the proposed model. The user of the system

generally takes a small part of the whole data, which is generally 25% or 33% of the complete data set.
[0057] The whole point of the machine learning system is to be able to work with unseen data, if the user knows that the user of the system is going to see all possible values in training data, the user might as well just use some form of lookup. Therefore, it is important that the proposed model has to train with the unseen data, as in future it might face the different kinds of data. However, the conventional form of ML is generally divided in two splits i.e., TRAIN and TEST. But, when the user of the system looks deeper, it can be classified in four phases.
1. Training: Training is the phase where the user feeds the data to the model and make that model learn from that data.
2. Development (also known as validation or tuning): This is the phase where the user identifies how well the model is performing. Ideally, the test data should only be used once, and therefore it is important to make such a split where the user validates or tunes the data until the model is well trained.
3. Testing (aka Evaluation): Once, the model is well trained and is without over fitting and under fitting, the user can now feed the data to the trained model and get the predictions done.
4. Use: Use is the phase, where the user can finally use the model after all the experiments that have been done on the model for the accuracy, it is now ready to perform on streaming or batch data.
[0058] For supervised learning, the available labeled data is usually quite limited. This data must be split to be used in the first three phases. The fourth phase is usually not explicitly discussed in research, but of course the aim of real ML is to get the best possible performance in actual use, where the user of the system does not have

ground truth values. This should always be borne in mind when the user talks about evaluation and statistical methodology, the ultimate aim is to make sure that the performance in the use phase is as high as possible. Therefore, these four steps are utmost important. However, the general talk about the data splitting is training and testing as shown in the FIG. 4.
[0059] Model Creation layer: Building the ML model is like building any other product. The modeling in the machine learning is an iterative phase where a data scientist continually trains and tests machine learning models to discover the best one for the given task. There are different machine learning models for different problem statements and the choice of which one to use entirely depends on the problem statement and the dataset, the user of the system needs to test multiple algorithms so that the user of the system can have the best model according to the user requirement. Nevertheless, there are different ways to build a machine learning model, either the user can summarize the stages or can go in detail. However, the user can also follow the below mentioned steps to create the ML model.
a. Ideation: This step is the utmost important as it allows alignment on
the key problem to achieve some solution. Without the idea, the user
of the system can't agree on what model or algorithm the user should
go for.
b. Research: With the model idea in mind, it is important to validate
the model by studying the existing or upcoming ideas to get a better
result. In this way, the user can have a better idea of where the user
can land.
c. Define the Requirement: By creating the minimum requirements,
the user will learn what the customer really wants and accordingly
the user can go ahead.

d. Design multiple versions: the user can have many design versions
of ideas, and out of which the user can have the best version, so it is
important that the user must have different versions in order to have
a clear idea about the problem statement.
e. Prototyping: In this stage, the user creates the primary version of the
model for the initial phase.
f Testing: Testing lets us know how well this primary model performs,
and it iterates until the model gives satisfactory results, g. Productization: Stabilizing and scaling of models as well as data collection and processing to produce useful outputs in production environments is termed as productization It is a final product which goes into production. However, the model gets modified every time the user of the system does fine tuning.
[0060] Building the model: Building the ML model involves all the above-mentioned stages. The user may build a prototype machine learning model on the existing data before the user moves forward. The main idea behind building the prototype of the ML model is to understand the data and necessary preprocessing steps required before the model building process. Once the data is in good shape, the data science team can start working on the actual model. It involves a lot of experimentation and discovery — selection of best relevant features, testing of multiple algorithms etc. It is not always a straightforward execution task; it involves various steps to finally achieve the best suited model for the problem and therefore the timeline of getting to a production-ready model can be very unpredictable.
[0061] This is how the first version of the model is built, depending on the number of iterations, validation and problem statement as shown in the FIG. 5.

[0062] Training and Optimization: This is the most crucial step in building ML models, as training data for the ML is a key input to algorithms that helps models to memorize the information for future prediction. The training data is a backbone of an entire AI and ML proj ect without that it is not possible to train a machine that learns like a human does.
[0063] The structured and organized data is very important to act as an input for the model as the unstructured data collected from the different sources are usually in unorganized format, which is not useful for machines to ingest the useful information. But when such data is well cleansed and transformed in a compatible format, it becomes a well-organized data that can be used to train the AI or ML model.
[0064] Another most important role of training data for machine learning is classifying the data sets into various categories. When the algorithm learns what features are important in distinguishing between two classes, it helps model to recognize and classify the similar objects in future, thus training data is very important for such classification. And if it is not accurate it will badly affect the model results that can become the major reason behind the failure of AI projects.
[0065] Training data is the only source, the user can use as an input into algorithms, that will help the AI model to gain the useful information from the data. After successfully training, it is very important to validate and maintain the model and improve its performance by optimizing the model. For optimization and improving the performance of the model, there are many ways and some of them are mentioned below: a) Improve performance with data.

b) Improve performance with algorithms.
c) Improve performance with algorithm tuning.
d) Improve performance with ensembles.
[0066] Model validation: As shown in the FIG. 6, after developing the ML model, it is extremely important to check the accuracy of the model predictions and validate the same to ensure the precision of results given by the model and make it usable in real life application. For machine learning validation, the user can follow the technique depending on the model development methods as there are different types of methods to generate the ML model.
[0067] Further, choosing the right validation method is also very important to ensure the accuracy and biasness of the validation process. Without checking the accuracy of model output, relying on the results can be devastating.
[0068] But the selection of the right validation method has got all the importance for making sure the accuracy of the model. The user can choose between automated and manual (done by humans) model validation processes depending on the types of models the user are developing through machine learning or deep learning algorithms. Further, the user of the system can evaluate the predictions and validate the same using various techniques, out of which few ML model validation methods are mentioned below:
a) ML model validation by humans
b) Holdout set validation method
c) Cross-validation method for models
d) Leave-one-out cross-validation
e) Random subsampling validation
f) Teach and test method
g) Bootstrapping ML validation method

h) Running AI model simulations i) Overriding mechanism method
[0069] Training at scale: The ML at scale addresses two different scalability concerns. The first is training a model against large data sets that require the scale-out capabilities of a cluster to train, it means the user of the system needs to have a cluster which can handle such a huge dataset. The second centers on operationalizing the learned model so it can scale to meet the demands of the applications that consume it. Typically, this is accomplished by deploying the predictive capabilities as a web service that can then be scaled out.
[0070] To understand model training at scale, the user of the system needs to understand the difference between building model and building model at scale. If the user of the system wants to build a model for a proof of concept (POC), it is quite easy, the user of the system can write code on the local system and then run it whenever the user wants to. But if the user wants to be able to work at a larger scale, it is bit harder.
[0071] Training at scale means that the model works for huge amount of data. This is very different from a training model locally. However, Machine learning at scale has the benefit that it can produce powerful, predictive capabilities because better models typically result from more data. Once a model is trained, the model can be deployed as a stateless, highly performant scale-out web service.
[0072] When the model is ready to be deployed, it can be encapsulated as a web service and deployed in the cloud, to an edge device, or within an enterprise ML execution environment. This deployment process is referred to as operationalization. The user of the system can also go with the docker image, basically it depends on

the kind of execution the user wants. There are many challenges when training model at scale like:
a) The user needs a lot of data for training models.
b) For scenarios such as deep learning, not only a cluster that can provide a scale-out on CPUs is needed, but the cluster will need to consist of GPU-enabled nodes.
c) The model training phase must access the big data stores.
[0073] Creating and deploying the ML model is an iterative process. The data scientists train and validate models based on appropriate algorithms to find the optimal model for prediction. The optimal model is deployed into production, as a web service or some other encapsulated function.
[0074] Model: After successfully performing all the above processes, the ML model is now available. The ML model is prepared by combining the data with algorithms as shown in the FIG. 7. The ML model intends to determine the optimal patterns or trends in a dataset to achieve an assigned task. It results from learning algorithms applied on the dataset.
[0075] Below are main four stages for building the ML model (as shown in the FIG. 8):
Data Preparation: The goal of data preparation is to shape the data for future. If the formats of the data are different (if collected from the different sources), then the user have to transform the data and make them uniform so that it can have a certain structure and the user can conduct different analysis and build machine learning models. Feature Engineering: Most of the time, data is in the tabular form and each column is a feature, and most of the time these features may not produce the best results and hence the user need to alter, delete, and combine these features to create a new dataset so that it would be

more adaptable at training the model. This process is said to be Feature Engineering. There are many techniques to perform feature Engineering. One of them is feature selection, having irrelevant features in the data can increase the inefficiency of the model and hence the user needs to perform feature selection where the user selects those features which contribute most to the model. Data Modeling: Data Modeling is the process of creating a data model which basically organizes each and every element of data and how they are related to each other.
Performance measure: Evaluating any ML model is an important part of any ML lifecycle process. Evaluation metrics explains the performance of a model. The ML model may provide good results but when evaluated using a metric, it may produce poor results. By using these feedbacks, the user of the system can make improvements and can achieve a desirable accuracy. The system has different types of evaluation metrics are available for improving the performance of the model.
[0076] Roll out layer/Deployment layer:
[0077] Deploying: The deployment of the ML models is the process for making the models available in production environments, where they can provide predictions to other software services. The user just can't throw the model into production, the user of the system has to discover the various ways for deploying the proposed model depending on the specific use case.
[0078] Once the ML model is trained locally, and if the user of the system wants it to be scaled, the user of the system can go for the deployment of the ML model. Such trained models can be wrapped in a web service. This trained model is now ready to be utilized as a REST API based service using frameworks like flask.

Now at this point, the user of the system has a web service that runs locally. The present disclosure aims to run this service on cloud VM for the larger users.
[0079] In order to achieve this step, containerization plays a very vital role. The containerization of webservers allows us to avoid the trouble of running into environment related issues. The containerization helps to overcome such machine/OS related problems. The user of the system would be able to achieve code which would be independent of the machine/OS that runs it. If the containerized code works on one machine, it will definitely run on another machine as well irrespective of the characteristics of that machine/operating system (OS).
[0080] Now, the system has the containerized machine learning application running locally. Now it's time to scale this application for other users and it's not possible to utilize this service if the application is running in a local system. In order to scale it, the user of the system needs to build an architectural ecosystem which should be easily available to the end users, is automated and scalable for bigger data. Hence, it is important to host this application on the internet to meet such big requirements.
[0081] If applications are containerized, deployments on most platforms/infrastructure tend to be easier. Containerization also gives you the option to use a container orchestration platform (Kubernetes is now the standard) to rapidly scale the number of containers as demand shifts.
[0082] Serving model: Once the model is ready for deployment, now it is time to decide how the application will be used in the production. Basically, this depends on the problem statement and how much the end users are comfortable to use its service. In one

of the approaches, the user of the system treats the model as a dependency within the running application, and the proposed model acts as an embedded model within the active application running in the production environment. Here, the user of the system doesn't release such models independently as they are just a part of a bigger software. In the other approach, the user of the system deploys the machine learning model as a completely separate service, where the user of the system wraps the model in a service which can be deployed independently. In this way it helps to facilitate the models with the latest release.
[0083] This is a current area of development and various tools and vendors are working to simplify this task. And hence, it means there is no clear standard (open or proprietary) that can be considered a clear winner yet for choosing which option works better for serving the ML models in the production, and therefore the user will need to evaluate the right option as per the user requirement.
[0084] Monitoring and Logging: The ML model is deployed in the production, after deploying the ML model, it is being checked whether the ML model is working fine or not, is this going to give the correct output as it is giving in a first week or is the accuracy of my model decreased? Therefore, continuously monitoring and logging of deployed ML models is mandatory, so that there won't be any reduction in the efficiency of the ML model. This is a complex challenge, and hence it is an important part of deploying ML models. While monitoring and logging of the model, below mentioned challenges need to be taken care of:
a) Incoming data
b) Model Quality
c) System operation

[0085] The incoming data could be a problem as the format, structure of the incoming data might get changed. Similarly, the decreasing quality of the model can also contribute to the challenges in the production, it has to be checked upon as its capacity might get decreased with the kind of data it is receiving. However, system operation doesn't depend on the data or model, the problem lies in the production environment itself. These challenges have to be looked upon in order to determine if the ML model is performing as the user expects them to be.
[0086] Fine tuning and Improvements: The fine tuning have its own importance in enhancing the accuracy of the model by integrating data from an existing model. From the name itself, it is quite clear that tuning doesn't stop in good results, it craves for the better results. The objective of the fine tuning and improvements is to find the best model with the highest accuracy.
[0087] There are again different ways to achieve this, but the user of the system can go with the below mentioned approaches:
a) Algorithm Tuning: where discovering the best models is treated like a search problem through model parameter space, which means the user checks different algorithms for the same problem until the user gets the best one with the help of the feature engineering.
b) Ensembles: In this approach, different machine learning models are combined to make predictions.
c) Extreme Feature Engineering: In this approach, feature engineering is being performed, a lot of Exploratory Data Analysis (EDA) and from feature selection to hyper parameters all kinds of feature engineering is performed.

[0088] The embodiments disclosed herein can be implemented using at least one software program running on at least one hardware device and performing network management functions to control the elements.
[0089] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the spirit and scope of the embodiments as described herein.
[0090] It will be apparent to those skilled in the art that other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention. While the foregoing written description of the invention enables one of ordinary skill to make and use what is considered presently to be the best mode thereof, those of ordinary skill will understand and appreciate the existence of variations, combinations, and equivalents of the specific embodiment, method, and examples herein. The invention should therefore not be limited by the above-described embodiment, method, and examples, but by all embodiments and methods within the scope of the invention. It is

intended that the specification and examples be considered as exemplary, with the true scope of the invention being indicated by the claims.

CLAIMS

We claim:

1. A method of enabling users in a network to access predefined set of
functions, the method comprising:
allowing users to access one or more Machine learning (ML) model catalogues, wherein each ML model catalogue is trained on data from multiple network nodes, each model being associated with a predefined set of functions.
2. A method of enabling users in a network to access predefined set of functions, further comprising collecting raw data from one or more network nodes into a data lake
3. A method of enabling users in a network to access predefined set of functions for collecting raw data claimed in claim 2 further comprising data preparation in the data lake
Data Cleaning Data Ingestion Data Analysis Data Transformation Data Validation Data Splitting
4. A method of enabling users in a network to access predefined set of
functions, further comprising creating one or more data pipeline
between the data lake and ML training host

5. A method of enabling users in a network to access predefined set of
functions, further comprising training, a ML model at the ML training
host, using the prepared data received from the data lake for training-
at-scale, wherein training the ML model comprising:
processing the prepared data, received from the data lake, in a predefined structure at an ML training host
Optimizing Validating
6. A method of enabling users in a network to access predefined set of functions, further comprising preparing one or more model catalogues at ML training host, wherein each catalogue is associated with a predefined set of functions
7. A method of enabling users in a network to access predefined set of functions, further comprising deploying the ML model at the orchestrator on edge or cloud, allowing end-users to access one or more of the one or more model catalogues
8. A method of enabling users in a network to access predefined set of functions, further comprising monitoring and fine tuning in real time
9. An architecture comprising:
an orchestrator configured with a data lake where all kinds of raw data being collected and prepared from the network nodes,

the prepared data (training data sets) received from data lake can move in ML training host and processes through Machine Learning (ML) /Deep Learning (DL) algorithms to train the model,
once the model is trained, the ML model catalogue is deployed onto the orchestrator where end users can use the one or more of ML models that require to perform the actions in real time.