A SYSTEM AND METHOD FOR DETERMINING EFFICIENCY OF ARTIFICIAL INTELLIGENCE APPLICATIONS
DESCRIPTION
Technical Field
[001] This disclosure relates generally to Artificial Intelligence (AI), and more particularly to a method for determining efficiency of AI applications.

BACKGROUND

[002] Artificial Intelligence (AI) is a branch of computer science that enables emulation of human intelligence in machines. Based on AI, the machines are programmed in such a way that they can perform task that especially requires human intelligence. Due to this, many organizations are integrating AI in order to automate task performed by humans and increase business benefits. Since applications or tools integrated with AI are increasing its usage in today’s world. Therefore, there is a need for identifying functional capabilities and ethics of these powerful and potential AI applications and tools being developed.
[003] As already known, AI ethics is field that deals with issues related with AI. In order to identify functional capabilities and ethics of an AI application, ethical issues associated with the AI application may needs to be determined. Currently, in order to perform ethical assessment for the AI application, existing solutions provide a static framework to perform assessment of the AI applications. The existing static framework may provide an overview of different aspects related with ethical assessment of the AI applications, to a user. Further, based on the overview provided to the user, the user of the AI application may understand key areas where he needs to focus and put efforts in order improve functionalities of the AI application being developed. In addition, the existing static frameworks may provide high level vision for the AI ethics.
[004] However, none of the existing static frameworks are capable to show steps that required to be performed by organizations for achieving its goals of developing an efficient AI application. Further, the existing static framework provided, may often require expertise in technology and legislation in order to understand and synthesize ideas related with development of the AI application. This leaves out business stakeholders and decision makers in interacting with such existing static framework, and thus limits their reach and application. In addition, none of the existing static framework is capable to determine current state of an organization in different aspect required during implementation of the AI application. For example. an organization working with customer vision and another organization working with survey data require very different set of ethics considerations. Additionally, the existing static framework has all guidelines with one-size fit that are required for ethical assessment of the AI application. Hence, guidelines of the existing static framework may not be efficient to perform ethical assessment of the AI application as challenges or pertinent legislations associated with the AI ethics may considerably vary across industries and geographies.
[005] Therefore, there is a need of an efficient and robust mechanism that may determining efficiency of Artificial Intelligence (AI) applications being developed.

SUMMARY
[006] In an embodiment, a method for determining efficiency of Artificial Intelligence (AI) applications is disclosed. In one embodiment, the method may include generating a plurality of performance parameters corresponding to an AI application. It should be noted that, the plurality of performance parameters comprises diversity, security and safety, explainability, robustness and accuracy, privacy and data governance, and human agency and oversights. The method may further include receiving an input corresponding to each of the plurality of performance parameters generated for the AI application. It should be noted that, the input is provided by a user of the AI application. The method may further include generating a subjective score corresponding to each of the plurality of performance parameters, based on the input provided by the user. The method may further include determining distribution of each of the plurality of performance parameters corresponding to the AI application, based on the generated subjective score. The method may further include generating a risk score for the AI application. It should be noted that, the risk score is generated based on distribution determined for each of the plurality of performance parameters.
[007] In another embodiment, a system for determining efficiency of Artificial Intelligence (AI) applications is disclosed. The system includes a processor and a memory communicatively coupled to the processor, wherein the memory stores processor executable instructions, which, on execution, causes the processor to generate a plurality of performance parameters corresponding to an AI application. It should be noted that, the plurality of performance parameters comprises diversity, security and safety, explainability, robustness and accuracy, privacy and data governance, and human agency and oversights. The processor executable instructions further cause the processor to receive an input corresponding to each of the plurality of performance parameters generated for the AI application. It should be noted that, the input is provided by a user of the AI application. The processor executable instructions further cause the processor to generate a subjective score corresponding to each of the plurality of performance parameters, based on the input provided by the user. The processor executable instructions further cause the processor to determine distribution of each of the plurality of performance parameters corresponding to the AI application, based on the generated subjective score. The processor executable instructions further cause the processor to generate a risk score for the AI application. It should be noted that, the risk score is generated based on distribution determined for each of the plurality of performance parameters.
[008] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS
[009] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles.
[010] FIG. 1 is a block diagram illustrating a system for determining efficiency of Artificial Intelligence (AI) applications, in accordance with some embodiments of the present disclosure.
[011] FIG. 2 illustrates a block diagram of various module within a memory of an assessment device configured to determine efficiency of Artificial Intelligence (AI) applications, in accordance with some embodiments of the present disclosure.
[012] FIG. 3 illustrates a flowchart of a method for determining efficiency of Artificial Intelligence (AI) applications, in accordance with some embodiments of the present disclosure.
[013] FIG. 4 is an exemplary representation of an assessment performed to predict an applicability and a success of the AI application, in accordance with an exemplary embodiment of the present disclosure.
DETAILED DESCRIPTION
[014] The following description is presented to enable a person of ordinary skill in the art to make and use the invention and is provided in the context of particular applications and their requirements. Various modifications to the embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the invention. Moreover, in the following description, numerous details are set forth for the purpose of explanation. However, one of ordinary skill in the art will realize that the invention might be practiced without the use of these specific details. In other instances, well-known structures and devices are shown in block diagram form in order not to obscure the description of the invention with unnecessary detail. Thus, the invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
[015] While the invention is described in terms of particular examples and illustrative figures, those of ordinary skill in the art will recognize that the invention is not limited to the examples or figures described. Those skilled in the art will recognize that the operations of the various embodiments may be implemented using hardware, software, firmware, or combinations thereof, as appropriate. For example, some processes can be carried out using processors or other digital circuitry under the control of software, firmware, or hard-wired logic. (The term “logic” herein refers to fixed hardware, programmable logic and/or an appropriate combination thereof, as would be recognized by one skilled in the art to carry out the recited functions.) Software and firmware can be stored on computer-readable storage media. Some other processes can be implemented using analog circuitry, as is well known to one of ordinary skill in the art. Additionally, memory or other storage, as well as communication components, may be employed in embodiments of the invention.
[016] Referring to FIG. 1, a system 100, for determining efficiency of Artificial Intelligence (AI) applications, is illustrated, in accordance with some embodiments of the present disclosure. The system 100 may include an assessment device 102 with processing capabilities to predict an applicability and a success of an AI application against a plurality of existing AI applications. In an embodiment, the assessment device 102 may predict the applicability and the success of the AI application based on a generated risk score. The risk score may be generated based on distribution determined for each of a plurality of performance parameters. The plurality of performance parameters may include, but is not limited to diversity, security and safety, explainability, robustness and accuracy, privacy and data governance, and human agency and oversights. In addition, in order to generate the risk score, the assessment device 102 may receive input corresponding to each of the plurality of performance parameters for the AI application. Moreover, the input received may enable a user of the assessment device 102 to determine risk associated with the AI application at a hierarchy of levels. In an embodiment, the hierarchy of levels may include organization response level, Industry level, and vertical level.
[017] By way of an example, the user may correspond to a developer of the AI application or a manager of the organization. The assessment performed by the developer or the manager, may enable them to modify the AI application. The AI application may be modified based on recommendations provided by the assessment device 102 after evaluation of the inputs. By way of an another examples, a black box testing may be employed that allow a set of random users (not skilled in the AI application), to provide inputs for the plurality of performance parameters associated with the AI application. Based on the inputs provided by each of the set of users and the recommendation of the assessment device 102, the developer may enhance or modify the AI application. Examples of the assessment device 102 may include, but are not limited to a server, a desktop, a laptop, a notebook, a net book, a tablet, a Smartphone, or a mobile phone.
[018] The assessment device 102 may include a memory 104, a processor 106, and an input/output unit 108. The input/output unit (I/O) 108 may further include a display 110 and a user interface 112. A user (e.g., the developer) or an administrator (e.g., the manager) may interact with the assessment device 102 and vice versa through the I/O unit 108. In one embodiment, the I/O unit 108 may be responsible for capturing the inputs corresponding to each of the plurality of performance parameters associated with the AI application, for processing, and in turn displays a processed output. In an embodiment, the processed output may correspond to a comparative benchmark score (i.e., a risk score) generated for the AI application. The processed output may be displayed by the I/O unit 108 via the display 110. The display 110 may be used to display results of analysis performed by the assessment device 102, to the user.
[019] In alternate embodiment, the inputs corresponding to each of the plurality of performance parameters associated with the AI application may be captured from the user by external devices 120. Additionally, the processed output may be displayed by the external devices 120. The external devices 120 may include, but may not be limited to a desktop, a laptop, a notebook, a netbook, a tablet, a smartphone, a remote server, a mobile phone, or another computing system/device. By way of an example, the user interface 112 may be used by the user to provide inputs to the assessment device 102. Thus, for example, in some embodiments, the assessment device 102 may ingest the inputs corresponding to each of the plurality of performance parameters via the user interface 112.
[020] Further, for example, in some embodiments, the assessment device 102 may render intermediate results (e.g., an input received for each of a first set of customized questions corresponding to the AI application, an input received for each of a second set of customized questions corresponding to the AI application, determined distribution, a subjective score, etc.) or final results (e.g., the risk score and the prediction of the applicability and success of the AI application) to the user or the administrator via the user interface 112. In some embodiments, the user or the administrator may provide inputs to the assessment device 102 via the user interface 112.
[021] The memory 104 may store instructions that, when executed by the processor 106, may cause the processor 106 to predict the applicability and the success of the AI application based on the generated risk score, in accordance with some embodiments. As will be described in greater detail in conjunction with FIG. 2 and FIG. 4, in order to predict the applicability and the success of the AI application, the processor 106 in conjunction with the memory 104 may perform various functions including generation of the plurality of performance parameters corresponding to the AI application, reception of the input for each of the plurality of performance parameters, generation of subjective score, determination of distribution of each of the plurality of performance parameters, and generation of the risk score associated with the AI application.
[022] The memory 104 may also store various data (e.g., the received inputs, the subjective score, the determined distribution, the risk score, the identified coefficient, etc.) that may be captured, processed, and/or required by the assessment device 102. The memory 104 may be a non-volatile memory (e.g., flash memory, Read Only Memory (ROM), Programmable ROM (PROM), Erasable PROM (EPROM), Electrically EPROM (EEPROM) memory, etc.) or a volatile memory (e.g., Dynamic Random-Access Memory (DRAM), Static Random-Access memory (SRAM), etc.).
[023] Further, the assessment device 102 may be connected to a machine learning model 114. In an embodiment, the machine learning model 114 may utilize at least one of a supervised machine learning algorithm and an unsupervised machine learning algorithm. The machine learning model 114 may be configured to identify the coefficient corresponding to each of the plurality of performance parameters. The machine learning model 114 may identify the coefficient based on the subjective score generated. In addition, in order to identify the coefficient, the machine learning model 114 may normalize the subjective score generated corresponding to each of the plurality of performance parameters. Examples of the supervised machine learning algorithm may include but is not limited to regression, classification, naïve Bayesian model, random forest model, neural network, and support vector machines. Example of unsupervised machine learning algorithm may include, but is not limited to k-means clustering, hidden-markov model, principal component analysis, singular value decomposition, and association rule.
[024] Further, the assessment device 102 may interact with a server 116 or the external devices 120 over a network 122 for sending and receiving various data. The network 122, for example, may be any wired or wireless communication network and the examples may include, but may be not limited to, the Internet, Wireless Local Area Network (WLAN), Wi-Fi, Long Term Evolution (LTE), Worldwide Interoperability for Microwave Access (WiMAX), and General Packet Radio Service (GPRS).
[025] In some embodiments, the assessment device 102 may receive information associated with the AI application from the server 116. The server 116 may be configured to extract an information associated with previous similar existing AI applications to generate the plurality of performance parameters corresponding to the AI application. The server 116 may further include a database 118, which may store information related to a set of existing AI applications on the server 116. Alternatively, the user may access information associated with each of the set of existing AI applications via the external devices 120 coupled with the assessment device 102 via network 122 in order to predict the applicability and the success of the AI application. It should be noted that, the database 120 may be periodically updated based a new AI application developed.
[026] Referring now to FIG. 2, a block diagram of various module within the memory 104 of the assessment device 102 configured to determine efficiency of Artificial Intelligence (AI) applications is illustrated, in accordance with some embodiments of the present disclosure. As explained in conjunction to FIG. 1, the assessment device 102, may act as an interactive interface for predicting the applicability and the success of the AI application. The system 100 may include various modules within the memory 104 configured to work together. The memory 104 may include a database 204, a generation module 206, a reception module 208, a score generation module 210, a determination module 212, and a prediction module 214. The memory 104 may receive an input 202, and based on processing of the input 202, the memory 104 may predict the applicability and the success of the AI application.
[027] In one embodiment, the input 202 received by the memory 104 may correspond to the AI application. The AI application may correspond to the AI application for which the applicability and the success needs to be predicted against a plurality of existing AI applications. In another embodiment, the input 202 received by the memory 104 may correspond the input received corresponding to each of the plurality of performance parameters. As will be appreciated, the input received for each of the plurality of performance parameters may be provided by the user of the AI application. In an embodiment, the user may correspond to the developer of the AI application or the manager of an organization.
[028] Upon receiving the input 202 (i.e., the AI application), the generation module 206 may be configured to generate the plurality of performance parameters associated with the AI application. The plurality of performance parameters may include, but is not limited to, diversity, security and safety, explainability, robustness and accuracy, privacy and data governance, and human agency and oversights. Once the plurality of performance parameters is generated, the generation module 206 be configured to send and store, each of the plurality of performance parameters generated to the database 204.
[029] Further, a first set of customized questions may be generated for the AI application. Moreover, each of the first set customized questions may be generated based on a first performance parameter (i.e., the diversity) from the plurality of performance parameters. In an embodiment, each of the first set of customized may be generated by experts via the generation module 206. Examples of experts, may correspond to software testing engineers, quality assurance engineers, test analysts, test automation development engineers, and technical testers, etc. Once the first set of customized questions are generated, the generation module 206 may be configured to send each of the first set of customized questions to the reception module 208.
[030] The reception module 208 may be configured to receive each of the first set of customized questions for the first performance parameter from the generation module 206. Upon receiving the first set of customized question, the reception module 208 may be configured to receive an input (i.e., the input 202) against each of the first set of customized questions. The input received for each of the first set of customized questions may be provided by the user of the AI application. In reference to FIG. 1, the input may be provided by the user via the I/O unit 108 or the external devices 120. Upon receiving the input for each of the first set of customized questions, the reception module 208 may send and store, the input received along with the first set of customized questions to the database 204.
[031] Once the input received and the first set of customized question are store in the database 204, the generation module 206 may fetch the input received for each of the first set of customized question to generate a second set of customized questions. It should be noted that, the generation module 206 may generate the second set of customized questions based on the input received for each of the first set of customized questions. The second set of customized questions may be generated for a second performance parameter (i.e., the security and safety) from the plurality of performance parameters. Moreover, the second set of customized question generated may correspond to more specific questions related to the AI application. Thereafter, the generation module 206 may send each of the second set of customized questions to the reception module 208.
[032] Upon receiving each of the second set of customized question, the reception module 208 may be configured to receive an input (i.e., the input 202) for each of the second set of customized questions from the user. The reception module 208 may then send and store the input received for each of the second set of customized questions along with the second set of customized questions to the database 204. This process may continue until the input is received for each of the plurality of performance parameter and is stored the database 204.
[033] Once the input is received for each of the plurality of performance parameters and is stored in the database 204, the score generation module 210 may fetch input received for each of the plurality of performance parameters from the database 204. Upon fetching each of the input received, the score generation module 210 may be configured to generate a subjective score corresponding to each of the plurality of performance parameters. In an embodiment, the score generation module 210 may generate the subjective score based on the input received from the user for each of the plurality of performance parameters. Further, the score generation module 210 may be configured to send the subjective score generated to the determination module 212. In addition, the score generation module 210 may send and store the subjective score generated for each of the plurality of performance parameters in the database 204.
[034] In one embodiment, the determination module 212 may receive the generated subjective score from the score generation module 210. In another embodiment, the determination module 212 may be configured to fetch the generated subjective score from the database 204. Upon receiving the subjective score, the determination module 212 may be configured to determine distribution of each of the plurality of performance parameters corresponding to the AI application. Further, the determination module 212 may send and store the distribution determined in the database 204.
[035] Thereafter, the score generation module 210 may fetch the distribution determined from the database 204. The score generation module 210 may then generate a risk score associated with the AI application. In an embodiment, the score generation module 210 may generate the risk score based on the distribution determined for each of the plurality of performance parameters. Once the risk score is generated, the score generation module 210 may send the generated risk score to the prediction module 214. In addition, the score generation module 210 may send and store the generated risk score in the database 204.
[036] In one embodiment, the prediction module 214 may be configured to receive the generated risk score from the score generation module 210. In another embodiment, the prediction module 214 may fetch the generated risk score from the database 204. Upon receiving the risk score, the prediction module 214 may be configured to predict the applicability and the success of the AI application, based on the risk score received. It should be noted that, the applicability and the success of the AI application may be predicted against the existing AI applications.
[037] It should be noted that all such aforementioned modules 206 – 214 may be represented as a single module or a combination of different modules. Further, as will be appreciated by those skilled in the art, each of the modules 206 – 214 may reside, in whole or in parts, on one device or multiple devices in communication with each other. In some embodiments, each of the modules 206 – 214 may be implemented as dedicated hardware circuit comprising custom application-specific integrated circuit (ASIC) or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. Each of the modules 206 – 214 may also be implemented in a programmable hardware device such as a field programmable gate array (FPGA), programmable array logic, programmable logic device, and so forth. Alternatively, each of the modules 206 – 214 may be implemented in software for execution by various types of processors (e.g., processor 106). An identified module of executable code may, for instance, include one or more physical or logical blocks of computer instructions which may, for instance, be organized as an object, procedure, function, or other construct. Nevertheless, the executables of an identified module or component need not be physically located together but may include disparate instructions stored in different locations which, when joined logically together, include the module and achieve the stated purpose of the module. Indeed, a module of executable code could be a single instruction, or many instructions, and may even be distributed over several different code segments, among different applications, and across several memory devices.
[038] Referring now to Fig. 3, a flowchart of a method for determining efficiency of Artificial Intelligence (AI) applications is depicted via flow diagram, in accordance with some embodiments of the present disclosure. At step 302, a plurality of performance parameters may be generated. Each of the plurality of performance parameters may be generated corresponding to an AI application. In an embodiment, the AI application may correspond to a new AI application of whose applicability and success needs to be predicted. The plurality of performance parameters may include diversity, security and safety, explainability, robustness and accuracy, privacy and data governance, and human agency and oversights.
[039] Once the plurality of performance parameters is generated, at step 304, an input corresponding to each of the plurality of performance parameters may be received. In an embodiment, the input for each of the plurality of performance parameters may be provided by the user of the AI application. Moreover, the input corresponding to each of the plurality of performance parameters may be received in order to determine the risk associated with the AI application. In embodiment the risk associated with the AI application may be determined at a hierarchy of levels, and wherein the hierarchy of levels includes organization response level, Industry level, and vertical level.
[040] In an embodiment, in order to receive the input corresponding to each of the plurality of performance parameter, a set of customized questions may be consecutively generated. Each of the set of customized questions may be generated corresponding to each performance parameter from the plurality of performance parameters associated with the AI application. Moreover, the set of customized questions corresponding to each performance parameter may be generated based on the input received for each of the set of customized questions generated for a previous performance parameter. In an embodiment, the set of customized question corresponding to each of the plurality of performance parameter may be generated by experts.
[041] Examples of experts, may correspond to software testing engineers, quality assurance engineers, test analysts, test automation development engineers, and technical testers, etc. By way of an example, suppose a first set of customized questions is generated corresponding to the first parameter, i.e., the diversity. Then, a second set of customized questions corresponding to the second parameter, i.e., security and safety may be generated based on an input received by the user of the AI application for each of the first set of customized questions.
[042] Upon receiving the input by the user, at step 306, a subjective score corresponding to each of the plurality of performance parameters may be generated. Once the subjective score is generated, at step 308, distribution of the plurality of performance parameters corresponding to the AI application may be determined. The distribution of the plurality of performance parameters may be determined based on the subjective score generated. In order to determine the distribution, the subjective score generated may be regressed. The subjective generated may be regressed in order to identify a coefficient corresponding to each of the plurality of performance parameters. In an embodiment, the coefficient may be identified based on the supervised learning algorithms. Moreover, in order to identify the coefficient, the subjective score generated corresponding to each of the plurality of performance parameters may be normalized. In an embodiment, the normalization of the subjective score may be performed based on the unsupervised learning algorithms.
[043] Once, distribution of each of the plurality of performance parameters is determined, at step 310, a risk score may be generated for the AI application. The risk score may be generated based on distribution determined for each of the plurality of performance parameters. Thereafter, based on the risk score generated, the applicability and the success of the AI application may be predicted. In an embodiment, the applicability and the success of the AI application may be predicted against a plurality of existing AI applications.
[044] Referring now to FIG. 4, an exemplary representation of an assessment performed to predict an applicability and a success of the AI application is depicted, in accordance with an exemplary embodiment of the present disclosure. In reference to FIG. 1, an exemplary system 400 depicts key components of the assessment device 102. The system 400 represents a user 402, a user input 404, an application logic 406, and insights of database 408. In an embodiment, the user may correspond to an individual using the AI application developed. When the system 400 receives the AI application, the system 400 may generated a plurality of performance parameters corresponding to the AI application may be generated. Based on the generated performance parameters the user input 404, corresponding to each of the plurality of performance parameter may be received by the user 402. In reference to FIG. 1 to FIG. 3, the user input 404 may correspond to the input received for each of the plurality of performance parameters generated.
[045] In order to receive the input a set of customized questions may be generated by experts as depicted by a user interface of the system 400. The user interface may guide the user 402 through each of the set of customized questions generated. In reference to FIG. 1, the user interface may correspond to the user interface 110. It should be noted that, the set of customized questions may be generated corresponding to each of the plurality of performance parameters. Based on the input received corresponding to each of the plurality of performance parameters, the application logic 406 may determine risk associated with the AI application at the hierarchy of levels. The hierarchy of levels may include organization response level, Industry level, and vertical level. In other words, the application logic 406 may be configured to evaluate the user input 404 received for each of the plurality of performance parameters to generate the risk score for the AI application.
[046] Thereafter, based on the generated risk score the applicability and the success of the AI application may be predicted as depicted by insights database 408. In an embodiment, final result about the applicability and the success of the AI application may be displayed in a form of a report. The report may depict the risk score associated with the applicability and the success of the AI application to the user 402, based on the user input 404 received for each of the plurality of performance parameters. In an embodiment, the generated risk score may also be referred as a comparative benchmark score. By way of an example, based on processing performed by the system 400, the applicability and the success of the AI application among other existing AI applications may be generated in percentile format. For example, the applicability and the success of the AI application developed may be 45 percentiles. Additionally, the system 400 may provide recommendations to the user 402, to modify the AI application developed to increase its applicability and success among other existing AI applications.
[047] It should be noted that, the system 400 disclosed may continuously improve its scoring functionality based on the user input 404 received for a plurality of users. This may enable, the system 400 to evolve with time, thus providing more targeted insights associated with the AI application, thereby increasing the applicability and the success of the AI application. Moreover, the comparative benchmark score generated may provide bias free evaluation of the AI application.
[048] As will be appreciated, the system 400 disclosed may find its applicability in various organization. For example, organizations in which the system 400 may find its applicability in technology or media industries, telecom industries, healthcare industries, financial services, consumer markets, industries products, educations, government sectors, etc. In addition, the system 400 may be deployed in various domains of organization. Examples of domains in which the system 400 may be deployed may include human resource, finance, manufacturing process, procurement, sales, etc. Moreover, the system 400 disclosed may be used by any decision makers who are action oriented, thus eliminating a need to understand esoteric technologies and AI legislations behind the AI application developed.
[049] Various embodiments provide method and system for determining efficiency of Artificial Intelligence (AI) applications. In particular, the disclosed method and system may generate a plurality of performance parameters corresponding to an AI application. The disclosed method and system may receive an input corresponding to each of the plurality of performance parameters generated for the AI application. Further, the disclosed method and the system may generate a subjective score corresponding to each of the plurality of performance parameters, based on the input provided by the user. Additionally, the disclosed method and the system may determine distribution of each of the plurality of performance parameters corresponding to the AI application, based on the generated subjective score. Thereafter, the disclosed method and the system may generate a risk score for the AI application. The generated risk score may enable the user of the AI application to predict its applicability and success against existing AI applications.
[050] The disclosed method and system provide some advantages like, the disclosed system and the method may not only focus on goals in terms of AI ethics but also focus on areas for improvement and prioritization based on AI applications area/geography/industry etc. Further, the disclosed system and the method may enable users of the AI applications to compare present state and expected state associated with the AI applications based on various performance parameters such as transparency, privacy, security etc. In addition, the disclosed system and the method may provide a self-learning assessment for the AI applications based on inputs received from the users for the set of customized questions generated by experts.
[051] It will be appreciated that, for clarity purposes, the above description has described embodiments of the invention with reference to different functional units and processors. However, it will be apparent that any suitable distribution of functionality between different functional units, processors or domains may be used without detracting from the invention. For example, functionality illustrated to be performed by separate processors or controllers may be performed by the same processor or controller. Hence, references to specific functional units are only to be seen as references to suitable means for providing the described functionality, rather than indicative of a strict logical or physical structure or organization.
[052] Although the present invention has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein. Rather, the scope of the present invention is limited only by the claims. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in accordance with the invention.
[053] Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.
[054] It is intended that the disclosure and examples be considered as exemplary only.

CLAIMS

What is claimed is:
1. A method for determining efficiency of Artificial Intelligence (AI) applications, the method comprising:
generating (302), by an assessment device (102), a plurality of performance parameters corresponding to an AI application, wherein the plurality of performance parameters comprises diversity, security and safety, explainability, robustness and accuracy, privacy and data governance, and human agency and oversights;
receiving (304), by the assessment device (102), an input corresponding to each of the plurality of performance parameters generated for the AI application, wherein the input is provided by a user of the AI application;
generating (306), by the assessment device (102), a subjective score corresponding to each of the plurality of performance parameters, based on the input provided by the user;
determining (308), by the assessment device (102), distribution of each of the plurality of performance parameters corresponding to the AI application, based on the generated subjective score; and
generating (310), by the assessment device (102), a risk score for the AI application, wherein the risk score is generated based on distribution determined for each of the plurality of performance parameters.

2. The method of claim 1, wherein the input received corresponding to each of the plurality of performance parameters determines risk associated with the AI application at a hierarchy of levels, and wherein the hierarchy of levels includes organization response level, Industry level, and vertical level.

3. The method of claim 1, wherein receiving the input corresponding to each of the plurality of performance parameter comprises:
consecutively generating a set of customized questions corresponding to each performance parameter from the plurality of performance parameters associated with the AI application, wherein the set of customized questions corresponding to each performance parameter is generated based on the input received for each of the set of customized questions generated for a previous performance parameter, and wherein the set of customized questions corresponding to each performance parameter is created by experts.

4. The method of claim 1, wherein determining distribution of each of the plurality of performance parameter further comprises:
regressing the subjective score generated to identify a coefficient corresponding to each of the plurality of performance parameters, and wherein regressing the subjective score further comprises:
normalizing each of the subjective score generated corresponding to each of the plurality of performance parameters, wherein each of the subjective score is normalized based on an unsupervised machine learning.

5. The method of claim 1, further comprises predicting an applicability and a success of the AI application against a plurality of existing AI applications based on the generated risk score.

6. A system (100) for determining efficiency of Artificial Intelligence (AI) applications, the system (100) comprising:
a processor (106); and
a memory (104) communicatively coupled to the processor (106), wherein the memory (104) stores processor executable instructions, which, on execution, causes the processor (106) to:
generate a plurality of performance parameters corresponding to an AI application, wherein the plurality of performance parameters comprises diversity, security and safety, explainability, robustness and accuracy, privacy and data governance, and human agency and oversights;
receive an input corresponding to each of the plurality of performance parameters generated for the AI application, wherein the input is provided by a user of the AI application;
generate a subjective score corresponding to each of the plurality of performance parameters, based on the input provided by the user;
determine distribution of each of the plurality of performance parameters corresponding to the AI application, based on the generated subjective score; and
generate a risk score for the AI application, wherein the risk score is generated based on distribution determined for each of the plurality of performance parameters.

7. The system (100) of claim 6, wherein the input received corresponding to each of the plurality of performance parameters determines risk associated with the AI application at a hierarchy of levels, and wherein the hierarchy of levels includes organization response level, Industry level, and vertical level.

8. The system (100) of claim 6, wherein the processor executable instructions further cause the processor (106) to:
receive the input corresponding to each of the plurality of performance parameter by consecutively generating a set of customized questions corresponding to each performance parameter from the plurality of performance parameters associated with the AI application, wherein the set of customized questions corresponding to each performance parameter is generated based on the input received for each of the set of customized questions generated for a previous performance parameter, and wherein the set of customized questions corresponding to each performance parameter is created by experts.

9. The system (100) of claim 6, wherein the processor executable instructions further cause the processor (106) to:
determine distribution of each of the plurality of performance parameter by regressing the subjective score generated to identify a coefficient corresponding to each of the plurality of performance parameters, and wherein, to regress the subjective score, the processor executable instructions further cause the processor to:
normalize each of the subjective score generated corresponding to each of the plurality of performance parameters, and wherein each of the subjective score is normalized based on an unsupervised machine learning.

10. The system (100) of claim 6, wherein the processor executable instructions further cause the processor (106) to predict an applicability and a success of the AI application against a plurality of existing AI applications based on the generated risk score.