Description:Complete Specification:
The following specification describes and ascertains the nature of this invention and the manner in which it is to be performed

Field of the invention
[0001] The present disclosure relates to the field of Artificial Intelligence security. In particular, the present disclosure proposes a method to prevent exploitation of an AI module in an AI system and the AI system including the AI module thereof.

Background of the invention
[0002] With the advent of data science, data processing and decision making systems are implemented using artificial intelligence modules. The artificial intelligence modules use different techniques like machine learning, neural networks, deep learning etc. Most of the AI based systems, receive large amounts of data and process the data to train AI models. Trained AI models generate output based on the use cases requested by the user. Typically, the AI systems are used in the fields of computer vision, speech recognition, natural language processing, audio recognition, healthcare, autonomous driving, manufacturing, robotics etc. where they process data to generate required output based on certain rules/intelligence acquired through training.

[0003] To process the inputs and give a desired output, the AI systems use various models/algorithms which are trained using the training data. Once the AI system is trained using the training data, the AI systems use the models to analyze the real time data and generate appropriate result. The models may be fine-tuned in real-time based on the results. The models in the AI systems form the core of the system. Lots of effort, resources (tangible and intangible), and knowledge goes into developing these models.

[0004] It is possible that some adversary may try to capture/copy/extract the model from AI systems. The adversary may use different techniques to exploitation the model from the AI systems. One of the simple techniques used by the adversaries is where the adversary sends different queries to the AI system iteratively, using its own test data. The test data may be designed in a way to extract internal information about the working of the models in the AI system. The adversary uses the generated results to train its own models. By doing these steps iteratively, it is possible to exploitation the internals of the model and a parallel model can be built using similar logic. This will cause hardships to the original developer of the AI systems. The hardships may be in the form of business disadvantages, loss of confidential information, loss of lead time spent in development, loss of intellectual properties, loss of future revenues etc. Hence there is a need for an AI system that is self-sufficient in averting adversarial attacks and extraction of internal information regarding working of the AI model.

[0005] There are methods known in the prior arts to identify such attacks by the adversaries and to protect the models used in the AI system. The prior art US 20190095629A1- Protecting Cognitive Systems from Model Stealing Attacks discloses one such method. It discloses a method wherein the input data is processed by applying a trained model to the input data to generate an output vector having values for each of the plurality of pre-defined classes. A query engine modifies the output vector by inserting a query in a function associated with generating the output vector, to thereby generate a modified output vector. The modified output vector is then output. The query engine modifies one or more values to disguise the trained configuration of the trained model logic while maintaining accuracy of classification of the input data.

Brief description of the accompanying drawings
[0006] An embodiment of the invention is described with reference to the following accompanying drawings:
[0007] Figure 1 depicts an AI system (100); and at least
[0008] Figure 2 illustrates method steps (200) to process an input in an AI system (100).

Detailed description of the drawings
[0009] It is important to understand some aspects of artificial intelligence (AI) technology and artificial intelligence (AI) based systems. Some important aspects of the AI technology and AI systems can be explained as follows. Depending on the architecture an AI systems may include many components. One such component is an AI module. An AI module with reference to this disclosure can be explained as a component which runs a model. A model can be defined as reference or an inference set of data, which uses different forms of correlation matrices. Using these models and the data from these models, correlations can be established between different types of data to arrive at some logical understanding of the data. A person skilled in the art would be aware of the different types of AI models such as linear regression, naïve bayes classifier, support vector machine, neural networks and the like. It must be understood that this disclosure is not specific to the type of model being executed in the AI module and can be applied to any AI module irrespective of the AI model being executed. A person skilled in the art will also appreciate that the AI module may be implemented as a set of software instructions, combination of software and hardware or any combination of the same.

[0010] Some of the typical tasks performed by AI systems are classification, clustering, regression etc. Majority of classification tasks depend upon labeled datasets; that is, the data sets are labelled manually in order for a neural network to learn the correlation between labels and data. This is known as supervised learning. Some of the typical applications of classifications are: face recognition, object identification, gesture recognition, voice recognition etc. Clustering or grouping is the detection of similarities in the inputs. The cluster learning techniques do not require labels to detect similarities. Learning without labels is called unsupervised learning. Unlabeled data is the majority of data in the world. One law of machine learning is: the more data an algorithm can train on, the more accurate it will be. Therefore, unsupervised learning models/algorithms has the potential to produce accurate models as training dataset size grows.

[0011] As the AI module forms the core of the AI system, the module needs to be protected against attacks. AI adversarial threats can be largely categorized into – model extraction attacks, inference attacks, evasion attacks, and data poisoning attacks. In poisoning attacks, the adversarial carefully inject crafted data to contaminate the training data which eventually affects the functionality of the AI system. Inference attacks attempt to infer the training data from the corresponding output or other information leaked by the target model. Studies have shown that it is possible to recover training data associated with arbitrary model output. Ability to extract this data further possess data privacy issues. Evasion attacks are the most prevalent kind of attack that may occur during AI system operations. In this method, the attacker works on the AI algorithm's inputs to find small perturbations leading to large modifications of its outputs (e.g., decision errors) which leads to evasion of the AI model.

[0012] In Model Extraction Attacks (MEA), the attacker gains information about the model internals through analysis of input, output, and other external information. Stealing such a model reveals the important intellectual properties of the organization and enables the attacker to craft other adversarial attacks such as evasion attacks. This attack is initiated through an attack vector. In the computing technology a vector may be defined as a method in which a malicious code/virus data uses to propagate itself such as to infect a computer, a computer system or a computer network. Similarly, an attack vector is defined a path or means by which a hacker can gain access to a computer or a network in order to deliver a payload or a malicious outcome. A model stealing attack uses a kind of attack vector that can make a digital twin/replica/copy of an AI module.

[0013] The attacker typically generates random queries of the size and shape of the input specifications and starts querying the model with these arbitrary queries. This querying produces input-output pairs for random queries and generates a secondary dataset that is inferred from the pre-trained model. The attacker then take this I/O pairs and trains the new model from scratch using this secondary dataset. This is black box model attack vector where no prior knowledge of original model is required. As the prior information regarding model is available and increasing, attacker moves towards more intelligent attacks. The attacker chooses relevant dataset at his disposal to extract model more efficiently. This is domain intelligence model-based attack vector. With these approaches, it is possible to demonstrate model stealing attack across different models and datasets. This invention discloses a robust AI system and it’s components including a novel architecture for the AI module such that it is able to prevent adversarial attacks and exploitation of the AI module.

[0014] Figure 1 depicts an AI system (100) . The AI system (100) comprises an input interface (12), a dynamic switching module (14), an AI module (16) and at least an output interface (20). The input interface (12) can be a hardware or a combination of software and hardware both that receives input from at least one user. The AI module (16) configured to process the input and generate an output displayed on the output interface (20).

[0015] The AI module (16) as explained above runs a AI model that is trained process the input and give a logical output. AI model could be a software or hardware or a combination of both. AI models such as linear regression, naïve bayes classifier, support vector machine could be implemented as a software or combination of software and hardware. Other AI models such as neural networks are implemented via specialized silicon chips. The AI module (16) is the present invention comprises a plurality of subunits. Each of the subunits are arranged parallel to each other. Each subunit comprises a sub-model, having different network parameters and hyperparameters. Hyper parameters is a parameter whose value is used to control the learning process. While networks parameters are learned during the training stage, hyper parameters are given/chosen. Hyper parameters are typically characterized by the learning rate, learning pattern and the batch size. They in principle have limited influence on the performance of the model but affect the speed and quality of the learning process.

[0016] Each sub-model comprises at least one processing layer amongst a plurality of processing layers (1,2…n). A processing layer for an AI model can be defined as a container that usually receives weighted input, transforms it with a set of mostly non-linear functions and then passes these values as output to the next layer.

[0017] The dynamic switching module (14) is configured to transmit the input from the input interface (12) to at least one selected subunit of the AI module (16). The selected subunit could either be a sub-model with plurality of processing layers or even a sub-model with a single processing layer or even a combination of plurality of sub-models. The dynamic switching module (14) can be a conventional electronic control mechanisms to selectively enable or disable different subunits in an ensemble dynamically. It determines which subunits are active and which are inactive during inference. Dynamic switching module rely on control parameters or criteria to decide when to switch between subunits. These parameters could be either random in simple way or it could be based on different factors, such as the input data characteristics, the confidence level of predictions, or the historical performance of individual models. One approach is to use confidence thresholds to decide whether to switch subunits. For example, if the confidence of a model's prediction falls below a certain threshold, the dynamic switching module can trigger a switch to another subunit in the ensemble that might be more confident for that particular input. Dynamic switching module can also leverage the diversity of the ensemble to make decisions. By continuously monitoring the differences in predictions made by different models, the module can switch to subunits that provide diverse outputs for a given input, indicating potential adversarial examples.

[0018] As used in this application, the terms "component," "system," "module," "interface," are intended to refer to a computer-related entity or an entity related to, or that is part of, an operational apparatus with one or more specific functionalities, wherein such entities can be either hardware, a combination of hardware and software, software, or software in execution. As further yet another example, interface(s) can include input/output (I/O) components as well as associated processor, application, or Application Programming Interface (API) components. The AI system (100) could be a hardware combination of these modules or could be deployed remotely on a cloud or server.

[0019] It should be understood at the outset that, although exemplary embodiments are illustrated in the figures and described below, the present disclosure should in no way be limited to the exemplary implementations and techniques illustrated in the drawings and described below.

[0020] Figure 2 illustrates method steps to process an input in an AI system (100). The components of the AI system (100) have been explained in accordance with figure 1. For clarity it is re-iterated that an AI system (100) comprises an input interface (12), a dynamic switching module (14), an AI module (16) and at least an output interface (20).

[0021] Method Step 201 comprises receiving input data from at least one user through the input interface (12). Method step 202 comprises selecting at least one subunit amongst a plurality of parallelly arranged subunits in the AI module (16) by means of the dynamic switching module (14). Each subunit comprises a sub-model, each sub-model having different network parameters and hyperparameters. Each sub-model comprises at least one processing layer.

[0022] Method step 203 comprises transmitting the received input to the selected at least one subunit in the AI module (16) by means of the dynamic switching module (14). The selection of at least one subunit is such that it minimizes information gain of the user with regard to internal structure of the AI module (16). The information gain is calculated in the dynamic switching module (14) based on the output generated by the AI module (16) is the previous iteration. The information gain basically analyzes the output to tells us the amount of information regarding the internal structure and function of the AI module (16) that could have been extracted by an adversarial input.

[0023] Method step 204 comprises executing the AI module (16) to generate an output. The AI module (16) utilizes multiple layers/sub-units forming an array of sub-models for making prediction. Every time the dynamic switching module (14) selects the sub-units (s), the path taken by the input for processing will have a different architectures, training data, or initializations, making them diverse in nature. The use of the dynamic switching module (14) changes the model decision path in use over time, hence the system creates a moving target. This dynamic behavior complicates the attacker's task since they cannot predict which specific sub-unit or sub-model will be operational at any given time. Method step 205 comprises communicating the generated input to the user using the output interface (20).

[0024] The core idea behind this structure of the AI system (100) specifically the structure of the AI module (16) and incorporation of the dynamic switching module (14) is to increase the attacker's burden. To successfully attack the AI system (100), the attacker would need to compromise every sub-model in the ensemble of the AI module (16). As there are multiple sub-models, each with its unique characteristics and vulnerabilities, the attacker faces a significantly more challenging task. Adversaries may attempt to craft malicious data (poisoning) or find adversarial examples that mislead the AI models. With dynamic switching module (14) and the plurality of sub-models functioning as the AI module (16), such attacks become more laborious and complex because the adversary needs to account for multiple models, each with different weaknesses.

[0025] In an embodiment of the present invention if the information gain exceeds a pre-defined threshold, the user is blocked, and the notification is sent the owner of the AI system (100). In addition, the user profile may be used to determine whether the user is habitual attacker or was it one time attack or was it only incidental attack etc. Depending upon the user profile, the steps for unlocking of the system may be determined. If it was first time attacker, the user may be locked out temporarily. If the attacker is habitual attacker, then a stricter locking steps may be suggested.

[0026] A person skilled in the art will appreciate that while these method steps describes only a series of steps to accomplish the objectives, these methodologies may be implemented with modifications to the AI system (100) described herein. It must be understood that the embodiments explained in the above detailed description are only illustrative and do not limit the scope of this invention. Any variation and adaptation to the method to process an input in the AI system (100) and the method thereof are envisaged and form a part of this invention. The scope of this invention is limited only by the claims.
, Claims:We Claim:
1. An AI system (100) for processing an input, said AI system (100) comprising an input interface (12) configured to receive an input from a user, an AI module (16) configured to process the input and generate an output displayed on an output interface (20), characterized in that AI system (100):
the AI module (16) comprising a plurality of subunits, each of the subunits arranged parallel to each other; and at least
a dynamic switching module (14) configured to transmit the input from the input interface (12) to at least one selected subunit of the AI module (16).

2. The AI system (100) for processing an input as claimed in claim 1, wherein each subunit comprises a sub-model, each sub-model having different network parameters and hyperparameters.

3. The AI system (100) for processing an input as claimed in claim 1, wherein each sub-model comprises at least one processing layer.

4. A method (200) for processing an input in an AI system (100), said AI system (100) comprising an input interface (12) configured to receive an input from a user, an AI module (16) configured to process the input and generate an output displayed on an output interface (20), the method step comprising: receiving (201) an input from at least one user via the input interface (12); characterized in that method:
selecting (202) at least one subunit amongst a plurality of parallelly arranged subunits in the AI module (16) by means of a dynamic switching module (14);
transmitting (203) the received input to the selected at least one subunit in the AI module (16) by means of the dynamic switching module (14);
executing (204) the AI module (16) to generate an output;
communicating (205) the generated input to the user using the output interface (20).

5. The method (200) for processing an input in an AI system (100) as claimed in claim 4, wherein each subunit comprises a sub-model, each sub-model having different network parameters and hyperparameters.

6. The method (200) for processing an input in an AI system (100) as claimed in claim 4, wherein each sub-model comprises at least one processing layer.

7. The method (200) for processing an input in an AI system (100) as claimed in claim 4, wherein selection of at least one subunit is such that it minimizes information gain of the user with regard to internal structure of the AI module (16).