Description:FIELD OF THE INVENTION

[0001] The present invention relates to an explainable artificial intelligence system for enabling multitask robot learning that is capable of monitoring dynamic environments, detecting anomalies, providing human-understandable explanations, and assisting operators in task monitoring and optimization.

BACKGROUND OF THE INVENTION

[0002] The recent advancements in robotics and machine learning have made it possible for respective means to perform multiple tasks with increasing autonomy and precision. Approaches that incorporate explainable artificial intelligence allow the means to learn efficiently while providing interpretable insights into their decision-making processes. By analyzing patterns, reasoning steps, and task strategies, the methods enhance transparency, improve adaptability, and enable safe collaboration in real-world applications like manufacturing, logistics, healthcare, and domestic assistance, ensuring more reliable, accountable, and intelligent robotic performance.

[0003] The traditional approaches to robotic learning largely rely on single-task programming and traditional learning models, where robots follow pre-defined instructions or acquire behaviors without providing interpretable reasoning. While the methods allow basic automation, however, lack in flexibility and adaptability, making it challenging for robots to perform multiple tasks simultaneously or adjust to new scenarios. The absence of transparent decision-making limits trust, safety, and effective deployment in dynamic environments such as manufacturing floors, warehouses, healthcare settings, or domestic assistance, reducing overall efficiency and reliability in real-time operations.

[0004] US8843236B2 discloses a method for training a robot to execute a robotic task in a work environment includes moving the robot across its configuration
[0005] space through multiple states of the task and recording motor schema describing a sequence of behavior of the robot. Sensory data describing performance and state values of the robot is recorded while moving the robot. The method includes detecting perceptual features of objects located in the environment, assigning virtual deictic markers to the detected perceptual features, and using the assigned markers and the recorded motor schema to subsequently control the robot in an automated execution of another robotic task. Markers may be combined to produce a generalized marker. A system includes the robot, a sensor array for detecting the performance and state values, a perceptual sensor for imaging objects in the environment, and an electronic control unit that executes the present method.

[0006] CN118014054B discloses a multi-task reinforcement learning method of a mechanical arm based on a parallel recombination network, which belongs to the technical field of mechanical arm motion control, and comprises the steps of constructing a multi-task reinforcement learning model PR-SAC of the mechanical arm based on the parallel recombination network and training; the trained mechanical arm multitasking reinforcement learning model PR-SAC is utilized to control the mechanical arm so as to realize multitasking control of the mechanical arm through a single network. The invention enables the information sharing in the network layer to be more sufficient by recombining the relationship between the layers in the network layer, then automatically selects the optimal path of each task through the weight network, and outputs the probability of each module being selected. Thus, this architecture can obtain the benefits of multitasking as much as possible. In addition, a sample correction module is added in the learning method so as to avoid the problem of policy update caused by the fact that the current policy is not in accordance with the sample.

[0007] Conventionally, many systems disclosed in the prior art provides a means for robotic solutions that rely on single-task programming or opaque learning models to perform specific functions. However, the systems offer limited adaptability and are incapable in handling multiple tasks efficiently. The lack of interpretability reduces transparency and trust, making it difficult for operators to understand decisions, ensure safety, or deploy robots reliably in dynamic and complex real-world environments.

[0008] In order to overcome the aforementioned drawbacks, there exists a need in the art to develop a system that requires to be capable of allowing robots to perform multiple tasks while adapting to changing environments. By identifying irregularities, providing explanations that are easily understood by humans, and supporting operators in supervising and refining operations, the system also needs to improve transparency, reliability, and overall efficiency in complex, real-world robotic applications.

OBJECTS OF THE INVENTION

[0009] The principal object of the present invention is to overcome the disadvantages of the prior art.

[0010] An object of the present invention is to develop a system that enables multitask robots to operate efficiently in dynamic environments while providing meaningful explanations of their actions to human operators.

[0011] Another object of the present invention is to develop a system that monitors robot performance, detects anomalies or deviations, and communicates them in a human-understandable manner to facilitate informed decision-making.

[0012] Yet another object of the present invention is to develop a system that assists human operators in task optimization, debugging, thereby improving adaptability, reliability, and overall operational efficiency.

[0013] The foregoing and other objects, features, and advantages of the present invention will become readily apparent upon further review of the following detailed description of the preferred embodiment as illustrated in the accompanying drawings.

SUMMARY OF THE INVENTION

[0014] The present invention relates to an explainable artificial intelligence system for enabling multitask robot learning in dynamic environments, detecting anomalies, generating human-understandable explanations, and assisting operators in monitoring, decision-making, and optimizing robotic task performance.

[0015] According to an aspect of the present invention, an explainable artificial intelligence system for enabling multitask robot learning comprises of an input layer for receiving multi-modal sensor data from the dynamic environment, a neural network with the input layer functioning as a multitask learning (MTRL) core to process the sensor data for determining policy of multiple tasks to be performed by the robot, a failure and anomaly handling module to monitor the robot’s performance, looking for deviations from expected outcomes, an explainable AI (XAI) module to generate an explanation upon receiving a failure signal or a query, a human-robot interface to translate the explanation into a human-understandable format and present to the human operator, and a processing unit to execute the input layer, the network, the modules, and the interface embedded in a non-volatile memory.

[0016] According to another aspect of the present invention, the system further comprises of the sensor data received form the dynamic environment includes visual input from cameras, depth data from LiDAR or RGB-D sensors, and proprioceptive data from the joints and actuators of a robot to be trained, the MTRL to continuously perceive the environment from the sensors and adjusts the policy in real-time, the failure and anomaly handling module for undertaking pre-execution and post-condition verification, generating a flag upon detection of an anomaly in multitask execution by the robot upto a desired level and generating a signal to the XAI module, and the XAI module accesses a log of the internal states and sensor inputs that led to the decision by the robot employs local interpretable model-agnostic explanation (LIME) or Shapley Additive exPlanations (SHAP), to determine the key factors that influenced the robot's action.

[0017] While the invention has been described and shown with particular reference to the preferred embodiment, it will be apparent that variations might be possible that would fall within the scope of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

[0018] These and other features, aspects, and advantages of the present invention will become better understood with regard to the following description, appended claims, and accompanying drawings where:
Figure 1 illustrates a block diagram of an explainable artificial intelligence system for enabling multitask robot learning.

DETAILED DESCRIPTION OF THE INVENTION

[0019] The following description includes the preferred best mode of one embodiment of the present invention. It will be clear from this description of the invention that the invention is not limited to these illustrated embodiments but that the invention also includes a variety of modifications and embodiments thereto. Therefore, the present description should be seen as illustrative and not limiting. While the invention is susceptible to various modifications and alternative constructions, it should be understood, that there is no intention to limit the invention to the specific form disclosed, but, on the contrary, the invention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the invention as defined in the claims.

[0020] In any embodiment described herein, the open-ended terms "comprising," "comprises,” and the like (which are synonymous with "including," "having” and "characterized by") may be replaced by the respective partially closed phrases "consisting essentially of," consists essentially of," and the like or the respective closed phrases "consisting of," "consists of, the like.

[0021] As used herein, the singular forms “a,” “an,” and “the” designate both the singular and the plural, unless expressly stated to designate the singular only.

[0022] The present invention relates to an explainable artificial intelligence system for enabling multitask robot learning that is capable of interacting with dynamic environments, monitoring performance, identifying anomalies, generating human-understandable explanations, and assisting human operators in decision-making, debugging, and task optimization, thereby enhancing safety, adaptability, and efficiency of robotic operations.

[0023] Referring to Figure 1, a block diagram of an explainable artificial intelligence system for enabling multitask robot learning is illustrated, comprising an input layer 101, a multi-modal sensor 102, a neural network 103, a failure and anomaly handling module 104, an explainable AI (XAI) module 105, a human-robot interface 106, a processing unit 107, a memory 108.

[0024] The system disclosed herein comprises of an input layer 101 configured for receiving data from a multi-modal sensor 102 from the dynamic environment. The input layer 101 functions as the initial interface for receiving multi-modal sensor 102 data from the robot’s environment, including visual, depth, and proprioceptive signals. The input layer 101 normalizes and formats the incoming signals into a structured representation suitable for downstream processing. A processing unit 107 associated with the system execute these operations by coordinating data acquisition, performing real-time pre-processing, and buffering inputs to ensure synchronization across modalities. This allows the multitask learning core to receive accurate, temporally aligned sensor information for decision-making and policy generation.

[0025] The multi-modal sensor 102 includes cameras, LiDAR sensor, and RGB-D sensors. When activated by the processing unit 107, the cameras capture optical images by focusing light onto an image sensor. The image sensor converts photons into electrical signals, which are digitized into pixel data.. The cameras also perform initial filtering, color correction, or compression to ensure the data is synchronized and ready for downstream multitask learning processing.

[0026] The LiDAR sensor emits laser pulses into the environment and measures the time taken for each pulse to reflect back from objects. This time-of-flight data is converted into distance measurements, generating a 3D (three-dimensional) point cloud representing the surroundings. The processing module activates the pulse emission, controls scanning patterns, and synchronizes returned signals. It preprocesses the distance data, filters noise, and aligns the point cloud with other sensor data, ensuring accurate real-time spatial perception for the robot.

[0027] The RGB-D sensors simultaneously capture color (RGB) images and depth (D) information. The RGB camera captures standard color images, while the depth sensor measures distance to objects using structured light, time-of-flight, or stereo vision. When triggered by the processing unit 107, it synchronizes the color and depth streams, converts raw signals into digital images and depth maps, and performs alignment and filtering. The processed RGB-D data is then transmitted to the input layer 101, providing a rich multi-modal representation of the environment. The sensor data includes all the visual input from cameras, depth data from LiDAR or RGB-D sensors, and proprioceptive data from the joints and actuators of the robot to be trained.

[0028] A neural network 103 associated with the system, functioning as a multitask learning (MTRL) core to process the sensor data for determining policy of multiple tasks to be performed by the robot. The MTRL core processes pre-processed, multi-modal sensor 102 data to simultaneously learn and execute multiple robot tasks. Internally, the MTRL core consists of shared layers for extracting common features and task-specific layers for generating policies for each task. The MTRL core is configured to execute forward passes to predict actions and backward passes for weight updates during training. By continuously integrating sensory feedback, the MTRL core adjusts policies in real-time, optimizing performance across tasks while maintaining knowledge transfer and preventing interference between concurrent tasks.

[0029] The policy’s output includes an action, such as a movement command, a grasp instruction, or a release and as the robot executes the action in the real-world, dynamic environment. The MTRL continuously perceives the environment from the sensors and adjusts the policy in real-time. As the robot performs the action, the MTRL core continuously receives updated multi-modal sensory data, evaluates discrepancies between expected and actual outcomes, and refines the task policy in real-time. This enables adaptive decision-making, ensures accurate execution of multiple tasks, and allows the robot to respond dynamically to changes in the environment while maintaining safe and efficient operation.

[0030] A failure and anomaly handling module 104 is associated with the system to monitor the robot’s performance, looking for deviations from expected outcomes. The handling module 104 is further configured for undertaking pre-execution and post-condition verification. The failure and anomaly handling module 104 continuously monitors the robot’s actions and sensory feedback to detect deviations from expected task outcomes. Internally, the handling module 104 performs pre-execution checks, post-condition verification, and real-time comparisons against predicted behaviors. Upon detecting an anomaly, it generates the flag and transmits a signal to an explainable AI (XAI) module 105 associated with the system. The processing unit 107 executes these operations, coordinating sensor input analysis, anomaly detection protocols, and communication, ensuring timely identification of failures, enabling adaptive response, and supporting operator decision-making.

[0031] Upon receiving a failure signal or a query, the processing unit 107 activates the XAI module 105 to generate an explanation. Internally, the XAI module 105 analyzes patterns in the input-output mapping and identifies factors influencing actions. Upon receiving the anomaly signal, the processing unit 107 executes explanation protocols and generates interpretable outputs. The XAI module 105 accesses a log of the internal states and sensor inputs that led to the decision by the robot employs local interpretable model-agnostic explanation (LIME) or Shapley Additive exPlanations (SHAP) to determine the key factors that influenced the robot's action.

[0032] The LIME works by creating a local surrogate model around a specific prediction. Internally, the LIME perturbs the input data, generating multiple slightly altered versions, and observes corresponding outputs from the original MTRL model. The LIME then fits a simple interpretable model, such as linear regression, to approximate the behaviour of the complex model in that local region. The processing unit 107 executes these computations, identifying input features most responsible for the decision, enabling the XAI module 105 to highlight key factors influencing the robot’s action.

[0033] The SHAP assigns a contribution value to each input feature based on cooperative game theory. Internally, the SHAP evaluates the effect of including or excluding each feature across all possible feature combinations, calculating Shapley values to quantify each feature’s impact on the model’s output. The processing unit 107 performs these combinatorial calculations, aggregating the results to determine feature importance. The XAI module 105 then translates these values into human-understandable explanations, such as visualizations or numerical summaries, showing how each sensor input influenced the robot’s selected action.

[0034] A human operator, armed with the diagnostic information in translated explanation enables the operator for debugging the system, adjusting the environment, or providing new data to retrain the MTRL core. Using the insights provided by the translated explanations, the human operator make informed decisions to correct unexpected behaviors, fine-tune task parameters, or modify environmental conditions. The operator’s interventions feed back into the system, enabling iterative improvement of the MTRL core, enhancing task performance, ensuring safe operation, and allowing the robot to adapt more effectively to dynamic, real-world environments.

[0035] Further, a human-robot interface 106 is configured with the system to translate the explanation into a human-understandable format and present to the human operator. The translated explanation includes natural language summary, a visual representation like a saliency map or a numerical breakdown of feature contributions. The human-robot interface 106 receives processed explanations from the XAI module 105 and converts them into human-understandable formats. Internally, the human-robot interface 106 maps numerical, visual, and textual data into intuitive representations, such as summaries, saliency maps, or charts. The processing unit 107 manages rendering, synchronization, and user interactions, ensuring real-time updates. The human-robot interface 106 also facilitates operator feedback, allowing inputs for task adjustments, debugging, or retraining. This enables seamless communication between the robot’s autonomous system and the human operator, improving interpretability and operational decision-making.

[0036] The layer 101, the network 103, the modules, and the interface 106 are embedded in a non-volatile memory 108 to be executed by the processing unit 107. Internally, the memory 108 enables the processing unit 107 to access sensor logs, task policies, and operational parameters in real-time. By providing persistent storage, the memory 108 ensures continuity of learning, maintains historical execution data for diagnostics, and supports reliable execution of the system across sessions.

[0037] The present invention works best in the following manner, where the robot is deployed in the dynamic environment for performing multiple tasks. The input layer 101 first receives the data from multi-modal sensor 102, including visual, depth, and proprioceptive signals, which are pre-processed and synchronized by the processing unit 107. The MTRL core processes these inputs to generate task-specific policies, providing real-time action commands, such as movement, grasping, or release. While executing the actions, the failure and anomaly handling module 104 continuously monitors performance, performing pre-execution checks and post-condition verifications to detect deviations or unexpected behaviour. Upon identifying the anomaly, it generates the flag and signals the XAI module 105. The XAI module 105 then accesses internal logs, sensor inputs, and policy decisions to determine the key factors influencing the robot’s actions. Using methods such as LIME or SHAP, it generates interpretable explanations that quantify feature contributions or highlight critical inputs. The human-robot interface 106 receives these explanations and translates them into human-understandable formats, such as natural language summaries, visual saliency maps, or numerical breakdowns. The operator then reviews the information in real-time, enabling informed decisions for debugging, adjusting environmental conditions, or providing additional data to retrain the MTRL core. Through this iterative workflow, the robot continuously adapts its policies, learns from new data, and maintains safe and efficient operation across multiple tasks. The persistent storage in the non-volatile memory 108 ensures continuity of learning, historical record-keeping, and reliable execution, even during power interruptions, allowing the system to operate optimally in real-world, dynamic conditions.

[0038] Although the field of the invention has been described herein with limited reference to specific embodiments, this description is not meant to be construed in a limiting sense. Various modifications of the disclosed embodiments, as well as alternate embodiments of the invention, will become apparent to persons skilled in the art upon reference to the description of the invention. , Claims:1) An explainable artificial intelligence system for enabling multitask robot learning, the system comprises:
a) an input layer 101 configured for receiving multi-modal sensor 102 data from a dynamic environment;
b) a neural network 103 functioning as a multitask learning (MTRL) core, the network 103 configured to process the sensor data for determining policy of multiple tasks to be performed by a robot;
c) a failure and anomaly handling module 104 configured to monitor the robot’s performance, looking for deviations from expected outcomes;
d) an explainable AI (XAI) module 105 configured to generate an explanation upon receiving a failure signal or a query;
e) a human-robot interface 106 configured to translate the explanation into a human-understandable format and present to a human operator; and
f) a processing unit 107, wherein the layer 101, the network 103, the modules 104, 105, and the interface 106 are embedded in a non-volatile memory 108 to be executed by the processing unit 107.

2) The system as claimed in claim 1, wherein the sensor data includes visual input from cameras, depth data from LiDAR or RGB-D sensors, and proprioceptive data from the joints and actuators of the robot to be trained.

3) The system as claimed in claim 1, wherein the policy’s output includes an action, such as a movement command, a grasp instruction, or a release and as the robot executes the action in the real-world dynamic environment, the MTRL continuously perceives the environment from the sensors and adjusts the policy in real-time.

4) The system as claimed in claim 1, wherein the failure and anomaly handling module 104 is further configured for undertaking pre-execution and post-condition verification, generating a flag upon detection of an anomaly in multitask execution by the robot upto a desired level and generating a signal to the XAI module 105.

5) The system as claimed in claim 1, wherein the XAI module 105 accesses a log of the internal states and sensor inputs that led to the decision by the robot, employing local interpretable model-agnostic explanation (LIME) or Shapley Additive exPlanations (SHAP), to determine the key factors that influenced the robot's action.

6) The system as claimed in claim 1, wherein the translated explanation includes natural language summary, a visual representation like a saliency map or a numerical breakdown of feature contributions.

7) The system as claimed in claim 1, wherein the human operator, armed with the diagnostic information in translated explanation enables the operator for debugging the system, adjusting the environment, or providing new data to retrain the MTRL core.