DESC:FIELD OF INVENTION
The present invention relates to real-time data capturing and processing system for ongoing qualitative assessment of process anomalies. More specifically, the present invention relates to the field of digital control, vision sensor, data analysis and machine learning for replicating the physical process using digital twin and performing real-time detection and corrective actions by robotic arm for the detected anomalies.

BACKGROUND OF INVENTION
In the domain of industrial automation and process optimization, the manufacturing sector has underwent significant technological advancements, increasing availability of different types of low-cost sensors and cameras which have made possible to monitor every aspect of manufacturing activity. Which in-turn could be communicated to a server, the process related data could be analyzed and the manufacturing process could be eventually automated. Further, industry 4.0 is defined for an intelligent networking of machines and manufacturing processes; with the assistance of different technologies of information and communication technology (ICT). In which the communication implies that the different aspects of production get networked; and can be communicated, for making way for new production, creation of items, real-time adaptation and optimization of the overall process. In that regard, the cyber physical system (CPS) creates the capabilities and the requisite technologies needed for a smart factory environment, which requires extensive analysis and precision decision making. Importantly, these require integration of technologies such as sensors, communication, digital control, along with vision sensor, Data Analytics, Machine Learning, etc.

Said CPS functions to enable the engineering interactions among the physical and other hardware components; ordinarily, it defines the system of micro-computers, networking devices, and processes to be at the forefront of Automation of the industries. Same would also handle the complex problem solving towards product improvements, efficient resource deployments and decision making. However, there are few challenges yet to be solved which includes: reliability, security, mutli-modal information analysis, product inspection under unconstrained scenarios, knowledge transfer, etc.

In an automated manufacturing environment, if there exists any anomalies in the manufacturing process, it needs to be identified in real-time and the corrective actions is required to be performed in an automated manner, so as to optimize output of the involved manufacturing process. Optionally, which can be carried out using a Robotic Arm being networked with the manufacturing environment and made intelligent using state-of-the-art computation technologies for the real-time analysis of anomalies.

Further, CPS enables the engineering interactions among the physical and other hardware components. Typically, it defines the system of micro-computers, networking devices, and processes to be at the forefront of Automation of the industries, and would also handle the complex problem solving towards product improvements, efficient resource deployments and decision making. However, there are few challenges yet to be solved which includes: reliability, security, mutli-modal information analysis, product inspection under unconstrained scenarios, knowledge transfer, etc. An integrated solution that would address these challenges can be viewed as an Intelligent-Networked Robotic Arm.

In order to ensure an efficient manufacturing process over a continuous time period, the concept of Digital Twin has been developed over the years, wherein the entire functionality of sensing, communication, control and intelligence are replicated in a digital mode. The digital twin will also enable development towards self-correcting smart process control facilities. Hence, the overall motivation of this work is to have a Digital Twin (DT) model for anomaly detection in the manufacturing process and use a Robotic arm with several degrees of freedom (DoF) to correct the anomaly.

The document by K. Jahnavi & P. Sivraj, in "Teaching and Learning Robotic Arm Model", of 2017 International Conference on Intelligent Computing Instrumentation and Control Technologies (ICICICT) establishes the base for DT. It first provides the design and development of a 5 Degree of Freedom (DoF) Robotic Arm. The robotic arm has been deployed externally with a basic Arduino micro controller along with the motors actuating to the system. The environment is controlled through a graphical interface, which allows the robot to self-learn it’s movements. The Arm movements were defined to perform only tasks such as holding, picking and placing. However, some focus on vision sensors for real-time deployments with low latency is still missing. Though the work provides the definition and visualisation of a Digital Twin of Robotic Arm, it does not provide information on real-time analysis for manufacturing industries, missing the details of networking components with pipelined structure.

The connected multiple robotic arms and deployments in a network are discussed by Lee at el., in Multi-robotic Arms Automated Production Line.”, of 4th International Conference on Control, Automation and Robotics (ICCAR); in which the deployed robotic arms were able to perform pick and place operations with a 6-axis arm, Visual Inspection Gantry Arm, Delta Arm etc. which replicates the main process of an industrial assembly line. In addition, various industrial examples solved by the designed system were discussed in this work. However, the limitation is that it was explored only for a surface analysis on data observation and flow control. Another aspect that is not addressed is a network interface establishment of a robotic arm with an effective fault tolerance mechanism.

Further, in the document by S. Bratchikov at el. in "Development of Digital Twin for Robotic Arm" of 2021 IEEE 19th International Power Electronics and Motion Control Conference (PEMC) the DT is deployed by a combination of dynamics, intelligence, and networking characteristics. One of the important tasks in developing a DT for Robotic Arm is to create a dynamic model accounting to all kinematics and dynamics. Kinematics explains the properties of mass and acceleration of the arm pieces and objects, whereas the dynamics explains the complete body’s concepts of inertia and stability. The action of the robotic arm was replicated using polynomial-based techniques in control systems to represent behaviour of the arm through both Kinematics and Dynamics. Even though the work sufficiently demonstrates the behaviour of the robotic arm, the challenge that still is missing is on reliability. Reliability can be strengthened by infusing the real-time with ablation study and comparisons on developed solutions of DT connecting the concepts of Kinematics and Dynamics. Also, this work does not talk about data analytics and network interface.

As the extension of teaching provided by S. Bratchikov at el. for 4 DoF robotic arm in the precedent para, the work by S. Khrueangsakun at el. in "Design and Development of System for Real-Time Web-based Visualization and Control of Robot Arm", of 2020 5th International Conference on Control and Robotics Engineering (ICCRE) teaches deployment of robotic arm visualization enabled the concept of Digital Shadowing of CPS. It realizes the joints and links actuation in real-time with less latency (less than one sec). However, this work provides a real-time response in between real and digital worlds only using the angles of arm’s joints. The computational analysis with the factors such as inference time (compute and response time) is not addressed. Another aspect that is not carried out in the prior art is the operational analysis with dynamics of robotic arm movements.

The afore annexed conceptual demonstrations of Digital-Twins and Robotic-Arms open the possibility of bringing the aspect of advanced product analysis into research with real-time considerations, as the analysis on real and digital worlds bridges the uninterpreted knowledge of the working. Consequently, real-time deployment of analyzed data enhances the representation. Hence the proposed invention called DT-RAAD (Digital Twin based Robotic Arm for Anomaly Detection) establishes the primary base in dealing with the industrial level analysis of Digital Twins, including the required concepts of anomaly detection.

Moreover, the conventional technologies available in the field of Industrial Automation when compared with present DT-RAAD in accordance with the cited documents it can be observed that the known state of art lacks in several aspects that has been covered by the present DT-RAAD system in the view of configuring a Digital-Twin, networking the models, analysing the fetched data for the corrective movement by robotic arm; further, the combination of robotic arm and pipelined architecture is found to be absent and networking of the modules is found not to be performed eventually impeding the inspection thereby the overall production process. Therefore, to mitigate the afore-stated bottlenecks in the present DT-RAAD system an adaptive yet integrated approach is employed for combining modules such as Robotic Arm, Kinematics and Dynamics, Networking, Data Analysis, and Pipeline structure.

Thereby, the general purpose of the present invention is to provide an improved combination of convenience and utility, to include the advantages of the prior art, and to overcome the drawbacks inherent therein.

SUMMARY OF THE PRESENT INVENTION
The principal object of the present invention is a method and system for identification of process anomalies in an industrial automation system in the aspect of material handling, object classification, and automated data retrieval by detecting various anomalies associated with the said automated process, thereby optimization of the same.

Consistent with the precedent object further object of the present invention is to facilitate monitoring of the flow of industrial processes by shadowing the said process using a digital-twin functioning as counterpart of the robotic-arm, which provides means for sensing, communication, & control for imparting intelligence in the industrial processes with replication in the digital mode.

Further object of the present invention is to facilitate networking of the robotic arm with manufacturing environment so as to perform real-time detection and corrective actions for rectifying anomalies present in the industrial processes, for optimum resource deployment and robust decision making.

Another object of the present invention is to provide a method integrating digital-twin model and robotic manipulator involving product inspection, and multi-modal information analysis based on the approaches such as digital control, vision sensor, data analysis, and machine learning.

Another object of the present invention is to provide an apparatus relating to the robotic manipulator entailing an end-effector having multiple degrees of freedom for initializing self-corrective actions communicated by the replicated digital-twin for development of the required control facilities.

Another object of the present invention is to provide means for real-time remote communication with the ongoing industrial manufacturing processes, for reformation of the said physical process by employing adaptive approaches that combines output of the modules such as robotic arm, kinetics, dynamics, networking, data analysis, & pipeline structure.

In one aspect of the present invention for collecting data from industrial manufacturing process in the pictorial/image/video format, shortening it to the designated groups, processing it to the readable format, so as to detecting anomalies and performing corrective actions a robotic-arm is integrated with a digital-twin; wherein, said digital-twin functions to monitor overall operation of robotic-arm to maintain processing of products and sharing data to the central digital-twin system; whereas the robotic-arm is regarded as real-twin which is placed in closed-loop with its digitally shadowed version termed as digital-twin, to perform corrective actions communicated from the said digital-twin.

In further aspect of the present invention related to entailed hardware components, a robotic-arm made of multiplicity of rigid links movably/rotatably interconnected with one another for facilitating displacement of the end-effector in at least 7 individual senses, and made to interact with workparts present involved in the manufacturing process preferentially through visual communication for detection of flaws/anomalies and rectifying the influence of the same in an adaptive manner; in which plurality of sensors conferred with the robot-arm are made to fetch information related to but not limited to external shape & size of workpart, surface & sub-surface defects, colour & contour of workpart, etc. from the prevailing manufacturing environment for comparison with pre-fed data, and determination of presence of anomalies.

In another aspect of the present invention related to replicating the physical process for performing debugging, testing, & reforming the said process before physical implementation in the manufacturing environment, a digital-twin is deployed in the said manufacturing environment and networked with robotic-arm capturing visual data which is further fed to the said central digital-twin for initializing the corrective actions required for optimization of the overall process, and maintaining processing of workparts involved therein.

In another aspect of the present invention, the present DT-RAAD system integrating robotic-manipulator with digital-twin involves a microcontroller & graphical engine initiation, fetching & conveying of data, and arm activation aided by the system back-end for performing data actuation in the architecture of said DT-RAAD model; in which in the system backend real-time analysis is carried-out along with data verification and resource check for detection of anomalies; further, for properly executing the real-time analysis of the anomalies a object visibility check and occlusion check is preformed followed by multi-object detection involving sampling and weighing of the captured data; thereafter through single-stage model, or two-stage model, or multi-stage model data segregation is carried-out for categorizing anomalies in texture or feature, for determining type of said detected anomaly, so as to correspondingly preform corrective actions, and sustaining the optimized state of the involved manufacturing process.

BRIEF DESCRIPTION OF DRAWINGS
The advantages and features of the present invention will become better understood with reference to the following more detailed description taken in conjunction with the accompanying drawings in which:
Fig. 1 illustrates flow of instructions for real -twin and digital-twin for identifying the process anomalies and interlinking between operations of real-twin and digital-twin, according to one embodiment of the present invention;
Fig. 2 illustrates perspective view of simulated model of robotic arm manipulator, according to one embodiment of the present invention;
Fig. 3 illustrates kinematic representation of robotic arm manipulator, according to one embodiment of the present invention;
Fig. 4 illustrates flow of instructions involved in the back-end of the digital-twin for running system back-end along with robotic arm activation, according to one embodiment of the present invention;
Fig. 5 illustrates flow of instructions involved in the back-end of the digital-twin for real-time analysis of the data communicated from the real-twin, according to one embodiment of the present invention;
Fig. 6 illustrates flow of instructions involved in the back-end of digital-twin for performing multi-object classification for object visibility check, according to one embodiment of the present invention;
Fig. 7 illustrates flow of instructions involved in the back-end of digital-twin for performing multi-object classification for occlusion check, according to one embodiment of the present invention;
Fig. 8 illustrates flow of instructions involved in the back-end of the digital-twin for performing anomaly detection from the data communicated from the real-twin, according to one embodiment of the present invention;
Fig. 9 illustrates architecture of Developed Industrial Mode, according to one of the illustrative embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without these specific details.

Reference herein to “one embodiment” or “another embodiment” means that a particular feature, structure, or characteristics described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in a specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Further, the diagrams representing one or more embodiments of the invention do not inherently indicate any particular order nor imply any limitations in the invention.

As used herein, the term “plurality? refers to the presence of more than one of the referenced item and the terms “a”, “an”, and “at least” do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced item.

It should be emphasized that the term “comprises/comprising” when used in this specification is taken to specify the presence of stated features, integers, steps or components, but does not prelude the presence or addition of one or more other features, integers, steps, components or groups thereof.

The term ‘present system’ is interchangeably used to show presence of the ‘present invention’ in the following description, and relates to essentially the same subject matter.
The term ‘Digital Twin’ is defined as a simulation model that uses real world data to create simulations which can predict the performance of product or a process, and accountable for centrally monitoring overall production flow for actuating the robotic arm in order to initializing corrective measures.

The term ‘Real twin’ is defined as hardware element involving use of robot-hand like structure being replicated by the digital twin and accountable for real-time visual inspection, anomaly detection, and automated corrective actions.

The term ‘Real twin’ is interchangeably used to show presence of the ‘Robotic Arm’ in the following description, and relates to essentially the same subject matter.

Further, the integrated output of said Real-twin and Digital-twin is regarded as DT-RAAD system relating to the system of Digital Twin based Robotic Arm for Anomaly Detection.

The term ‘Manufacturing System’ is defined as a close loop system performing set of operations essential for either production of tangible product, or a group of process involving value adding operations.

The term ‘Workpart’ is defined as an object which is involved in the production process or onto which value adding operations are performed.

The term ‘Workpart having Flaw’ is defined as an object involving at least one anomaly, or considered as defective, faulty, flaw-full, or inappropriate when compared with pre-fed data designating standard for product or process performed at any part of the manufacturing process.

Fig. 1 & Fig. 4-8 describes flow of instructions for various process anomalies influencing output of the manufacturing system, which essentially indicates quality or characteristics or attributes vested with the involved automated manufacturing system, thereby describes method and/or flow of operations mandated for identifying anomalies, whilst Fig. 2 & 3 describes system hardware required for taking corrective actions for the detected anomalies and kinetic aspect involved at each element of the said hardware relating to the arm of a robot closely networked with the manufacturing environment; and lastly Fig. 9 describes functioning of the Industrial Model; embodying several aspects of the present invention as described hereinafter.

Referring to the Fig. 1, it shows overall architecture employed for integrating real-twin with digital-twin and essentially delineates data interlinking/communication aspect resulting into data actuation for health of overall manufacturing system and analysing presence of anomaly at given part of the manufacturing system for originating a decision over subsequent flow of workpart in the manufacturing process or separating said workpart into the group of workparts having flaw.
The initial step of data acquisition process as depicted in Fig. 1 starts with Microcontroller initiation (10) and Graphical engine-initiation (60), in which the Microcontrollers are Programmable Logic Controllers (PLC) or similar devices used in industrial adapted systems for assembly lines and robotic arms, while reason for considering PLCs is that it provides a much more reliable system compared to Arduinos and Raspberry modules. Further, the graphical engine initiation consisting part of the digital-twin allocates the memory to the real-world visualisation module to replicate the process of the real twin, also known as ‘Shadowing’ the workpart.

Further, the second step of the data acquisition process comprises activity of TCP/IP request (20, 70) on the case of both real-twin and digital-twin termed as connection to system and connection to the controller, respectively. Wherein, in the connection to system the processor initiates connection with the digital twin through TCP/IP protocols for ensuring security of data communication with their standardised architecture; whereas, while preforming connection to the controller said activity carries out the process of accepting or re-requesting the connection to the controller in the real-world which forms a two-way communication between the real twin and digital twin modules.

Then, in the real-twin the connection with the system facilitates manipulator to enable its sate of action for further activation and acquiring responses from its parallelly placed digital-twin; conversely, in the digital-twin the activity for identifying manipulator’s functioning and the environment perception is carried out in which workparts placed in the sensor's visible range defines adjustment of the manipulator's workspace for further analysis.

After configuring standalone state (30) in real-twin and acquiring real-twin’s environment perception (80) by the digital twin, in the stage of Sensing and Communicating data (40) of preformed wherein manipulator’s idle state would allow the visual sensor for data capturing and sensor data analysis, considering the real-time scenarios and the data is thereafter stored into the memory of the controller [or PLC] for communication with the digital-twin through established network connections. Similarly in the digital twin command of receive data is executed for providing input of sensed data being sensed from the robotic arm to the manufacturing process for subsequent analysis of visual cues of the workpart.

The processed and analysed data received from microcontroller initiation (10) stage is acknowledged back to the manipulator from digital-twin module or the arm activation (50), the said data further stimulates the robotic arm for dynamic activation and carries out the operations such as ‘Adjust’ or ‘Pick and Place’ the anomalies in the conveyor belt; the said robotic arm is also deployed with vision sensors having fast and superior generalizing capability with trained models, as further can be referred from Fig. 2 & 3.

Consistent with the stage of arm activation (50) in the real-twin for the digital-twin Run command for system backend is executed in the block Run: System Backend (100) in which transitions among the digital twin initiates the backend system, to perform the crucial analysis that includes - identification, classification, description, explanation and anomaly detection. The architecture of the said backend is designed to perform task analysis in a single instanced data flow, which can reduce the additional computation, while the said backend architecture is prescribed in Fig. 4.

Further in the stage of Send: Analysis Data (110), the communication takes place among the manipulator and digital twin based on the data analysis, classification and anomaly detection activities.

The step Data Actuation (120) is triggered from the final data received from both real-twin module and digital-twin module; then if the package is analysed to be healthy, actuation provides access to the workpart to flow towards the next stage of manufacturing process; the description of the workpart is also shared to the central system helping to track the details. Notably, in case the workpart analysed is found to be workpart having flaws, the manipulator picks the said type of workpart with the extended freedom in movement and transfers the same to the respective station with details of flaws/anomaly.

Referring to the Fig. 2 illustrating perspective view of simulated model of robotic arm manipulator configured in accordance with emerging concurrent need of industrial applications and smart manufacturing practices necessitating the 360 degree visual inspection of workparts. In the conventional practise, current industrial environments consider Robotic Arm or the Manipulator with 4 to 6 Degrees of Freedom (DoF) to be sufficient for automation of visual inspection and packaging flow. However, use of afore-stated configuration leads to collisions in links of robot arm, which at times impedes speed of actuation process and also reduces the durability of the arm according to the teaching provided by M. Crenganis and O. Bologa in ‘Efficeint Method for redundancy Resosution of a 7 DOF Manipulator’. Additionally, the persistent collisions between the links and joints of the robotic arm not only limits the workspace of robots, but could also potentially decrease the overall productivity.

To mitigate the bottlenecks laid-out in precedent para of the know-state of art having limitation of 6 Degrees of Freedom, a configuration of Robot Arm containing total 7 Degrees of Freedom is exploited in the disclosed work.

The addition of 7th degree facilitates the usage of Null space in this model in developing an advantageous system according to teaching provide by W. Nikolas, et. al. in “Exploiting Null Space Potentials to Control Arm Robots Compliantly Performing Nonlinear Tactile Tasks” of International Journal of Advanced Robotic Systems.

Wherein, said null space is defined as additional space, when the number of degrees is more than required. The presently disclosed 7th DoF of Robot Arm/ Manipulator is optimized to perform tasks in the null space. The optimization affects the end operator/end effector (i.e. the last link of the manipulator) while maintaining all other properties. Importantly, introduction of 7th degree of freedom brings robustness into the robot activation and provides a collision free environment, while sense of motion of each of the links involved in the said robotic arm is described in Fig. 3.

Visualization of Real-Twin: The Robotic Arm or Manipulator having 7 DoF in standalone state being connected to a parallel processed Digital-Twin is shown in Fig. 2, in which a dedicated vision sensor handles the data sensing process and is further attached to the Robotic Arm base. The vicinity range of Robotic Arm is adjusted and visualised by blue boundaries as illustrated in Fig. 3 and the kinematical robotic arm model along with 7 DoF is shown in Fig. 3 while the arrowed references in the Fig. 3 represents the freedom of movement of links and joints illustrating the reachability of the said robotic arm.

Referring to the Fig. 4, illustrating flow of instructions involved in the back-end of the digital-twin for running system back-end, which describing use of real-time analysis for performing hardware diagnosis through response from the robotic arm and accompanied with system initiation, and anomaly detection.
The architecture of digital-twin at the back-end comprises a ladder system decision block (200) which is an process flow to initiate and verify the working mechanism of the back-end of the digital-twin. In the case of any component level failures, the present decision block directed to data loss or a wrong communication for determining either proceed with the pre-set process sequence or alerting towards checking the proper hardware functionality.

The stage of System Initiation (300) performs troubleshooting of the Programmable Logic Controller (PLC), then, in the said stage allocation of necessary memory and resources to the backend system is provided for each sub-module functioning which includes but is not limited to: simulation, storage devices, sensors, actuators, motors, processing units, etc.

Further, in the stage of Visual Sensor System and Storage Access (400) collection of raw data streaming from multiple sensors and components embedded in the real-twin system is performed, wherein the said storage access maintains the required workpart analysis information and helps to handle the updates in synchronization with the backend system.

The pictorial data fetched from visual sensors are directed to the stage of Real-Time Analysis (500) for exercising visual inspection of flaws/anomalies of workparts, which is further referred to integrated vision and machine learning approaches; which in turn would help to extract and segment the spatial features from raw data that would further help in visual inspection, followed by the stage of Anomaly Detection.

The said stage of Anomaly Detection (600) is regarded as main recurring decision block of the backend architecture, which first examines details of Real-Time Analysis for identifying information related to pipeline behaviour and the workpart. Wherein, ordinarily the workpart includes flaws/anomalies such as but not limited to: missing object, object deformation, misplacement, etc.; Moreover, the present stage considers all types of anomalies in the conveyor belt and alerts the system as required for back-propagation. Importantly, the anomaly detection part is designed with recurrence and retracing pipeline; and is one of the essential operation of presently disclosed DT-RAAD system.

The consequence of detection of flaw/anomaly in the workpart at the stage of anomaly detection (600) leads to the stage of Process Interruption (700) which focuses on controlling and interrupting the traffic in the flow. In which, interruption is a temporary state action in the backend system, disallowing congestion of data in digital machines and workparts present over the conveyor belt; without process interruption, the data is likely to overflow and rewrite the back-propagation task that may produce inferior results.

Subsequent in the sequence, faulty data verification (800) involving verification of faulty data and receives inputs from the abstracted details from the real-time analysis (500), whereas faulty data (800) is represented as missing or unclassified data resulting from the unavailability of relevant information from the stage of Real-Time Analysis (500). In addition, in order to limiting/eliminating the influence of missing or unclassified data the operation of Run Fault Diagnosis (900) is exercised with the help of existing data stored in the memory which reinforces the Real-Time analysis activity to repeatedly check for new data and update itself from time to time, which enables recursive learning for improved anomaly detection mechanisms.

Inline with the fault verification and its diagnosis, in the case when workpart is misclassified and not analysed a decision block named Resource Check (1000) is activated to take corrective actions further system alerted by said resource check step takes control over resources and directs troubleshooting processes that include: depletion of resource check, overflowing memory or insufficient memory. Further, to support the action originated from the Resource Check step a Reallocation Alert (1100) responds the said action and acts according to the error identified, and reallocation and replenishing tasks are carried out by and memory is reset in the system's back-end.

At last step of back-end processing of digital-twin, a Hardware Diagnosis (1200) step alerts the central system of digital-twin for checking the damage in hardware components which includes but not limited to: power failure, heat damage or short-circuited connections.

Referring to Fig. 5 illustrating flow of instructions involved in the back-end of the digital-twin for real-time analysis of the data communicated from the real-twin; and specifically describing role of multi-object detection for visibility and occlusion check, in order to perform anomaly detection.

As a part of performing real-time analysis the process of detecting anomalies starts with receiving input data (510) in Image/Video format which would be the raw images or the video frames collected using the vision sensor or alike, said Image/Video stacks the image frames based on the window limit size, wherein window limit implies the maximum number of frames per window and is based on the frame rate.

Followed by, stage of image processing (520) in which raw images/videos require the removal of distortio1ns and noise, and applies various image processing techniques such as noise removal filters, camera calibration, frame conversion, etc as part of pre-processing the raw information, further the image transformations are applied in processing the raw pictorial data to analyze the geometric properties of the workparts. Thereafter, Object Visibility Check (530) which characterizes the subsequent flow of the image processing stage is performed; however, given a real-time scenario there is likely be to vision challenges such as illumination, specular and diffuse reflections, occlusions, cluttered regions, etc., in order to address these challenges, an image needs to be analyzed further for feature extraction so as to avoid missing of any granularity of information and precise workpart classification.

Further, in the stage of Multi-object Classification (531) once the workpart is visible in the given input image frame, the feature extraction is initiated and a machine learning model is trained to acquire the finer granularity details of spatial images. Notably, various machine learning techniques could be used. However, traditional classification models such as but not limited to Support Vector Machine (SVM), decision trees, etc. cannot always precisely identify the object and the associated anomaly unless the features are rigorously trained. Hence, for automated feature extraction and generalizing capability, advanced techniques such as but not limited to: VGGNet, EfficientNet, and ensemble approach etc. for multi-object classification, as illustrated in the Fig. 6.

Consistent with the multi-object classification (531), Image Enhancement (532) activity is performed as in any real-time scenario visual inspection of a workpart in any industrial application is complex due to factors such as: illumination, blur and low-lighting. Hence, in the stage of Image Enhancement (532) image enhancement using techniques such as (but not limited to): Histogram Equalization (HE), Adaptive Histogram Equalization (AHE), Contrast Limited Adaptive Histogram Equalization (CLAHE), Gamma correction, colour transformations, etc are carries out for improved visibility of low and high pass information in the image.

In the stage of Occlusion Check (540) any possible occlusions during the workpart classification is performed, in which classification of any single workpart at a particular timestamp/frame can be achieved with better precision, however, multi-object classification under occluded or cluttered scenarios is very hard to attain phenomena. Further, in the stage of Tracking and Grid Voting (541) presence of workparts is identified, which may require the boundary and edge understanding of an existing object, tracking the workpart during t-1, t, and t+1 time frames; however, it would be still challenging to do so, if the occlusions are at different levels of hierarchy, hence, an integrated approach with Multi-Object Tracking [MOT] trackers such as but not limited to: DeepSort, Channel and Spatial Reliability of Discriminative Correlation Filter [CSRT] and sliding window-based majority voting, is required to classify the object with maximum likelihood ratio and probability. Furthermore, in another stage of Multi-object Detection it gives advantage in the functioning of present system to automatically learn the features at different granularity levels during the convolution process, deep learning architectures can be implemented and used for classification. Notably, state-of-the-art deeper architectures for workpart detection tasks are You-Look-Only-Once (YOLO), FasterRCNN, EfficientDet etc. further, the ensemble approach can be employed in processing of Multi-object Detection (542) for better accuracy, as described in Fig. 7.

For evaluating proposed model’s performance in generalizing workparts in real-time a Validation and Deployment (550) stage is employed which proposes architecture with lightweight design to be tested by deploying in edge devices such as Nvidia Jetson Nano, Raspberry Pi, etc with low computations and processing cost.

Thereafter in the stage of Vision for Anomaly Detection (560) which forms an important part in Anomaly identification for identifying the anomaly in a workpart depending on the factors such as but not limited to: task, object, environment, visible range, etc., wherein the classification of workparts and anomalies is done in a two-step approach: The first step is to classify the category of the workpart, and the second step is to localize the anomaly type and the anomaly region, in the entire image space. This involves analysis of varied image features, texture details, geometric properties, height, width, length, etc of the involved workpart.

Once the defect/anomaly is observed Reporting Module (570) concludes the single instance of analytical process by reporting to the central digital-twin, with the details such as but not limited to image features, image frames, timestamp, and other attributes like batch number, etc. for assisting in the further anomaly detection. After which, in the stage of Augmented Synthesis Image generation (580) a parallel processing block of anomaly inspection comes with image regeneration and style transfers, wherein new artificial images can be used to validate the model’s generalizing capability and different generative models/networks that can be used for augmentation and style transfers includes but not limited to: Generative Adversarial Networks (GAN), Style GAN, Cycle GAN, Efficient GAN, etc.

At the last stage of Real-time Analysis alongside executing Reporting Module (570) to improve knowledge of the digital-twin a block called unnatural Anomalies (590) is employed to store generated images, further, when the fake images are generated, the features would be fooling the classification network, degrading the performance. If the classification is frequently reinforced and retrained with the generated data, the classification models become more robust to in identifying and classifying workparts.

Referring to Fig. 6 illustrating flow of instructions involved in the back-end of digital-twin for performing multi-object classification for performing visibility check; specifically describing various modes of data sampling and functional importance of Weighing Module.

As depicted in Fig. 6 processed data from image/video frames is directed to the data sampling (535) block in which the input data in the images/videos form are further sampled to avoid imbalance data and overfitting issues, which would help in training and test splits with batch wise training, while the feature representations for abstract and hierarchical learning depends on the design of models.

Further in the process of data sampling a block names as Ensemble Models (536), wherein the ensemble module is defined as the classification models that can be used for training of central digital-twin, which includes but is not limited to: Resnet18, VGG16, VGG19, EfficientNet. However, in the Convolutional Neural Network (CNN) network design process, there would be an impact on the learning process given the varied parameters such as optimizers, kernel depth and size, complexity of network, etc. typically, this requires a detailed ablation study with varying hyper-parameters; hence, a grid search mechanism can be introduced to identify the optimal fit of the model for generalized capability.

Lastly, at the end of data sampling process a Weighing Module (537) is employed is implemented with the mathematical intuition to establish relation between output vectors of Ensemble Models block and loss of each component. The said relation is mapped onto the resultant vector and then output as a single numerical value for computing the probability of the identified workpart as per the class.

Referring to Fig. 7 illustrating flow of instructions involved in the back-end of digital-twin for performing multi-object classification for occlusion check, and specifically describing functioning of Single stage modelling network and Two stage modelling network.

Wherein, in the Single stage modelling block (545) the detection network focuses on detecting the presence of workpart(s) in input image/video frames, and includes the faster detecting models such as but is not limited to: YOLO, Single Shot Detector (SSD), etc, further the output of the network is determined by loss and a bounding box which is overlaid on each identified workpart for prediction. In case of the Two stage modelling network (546) the detection network focuses on detecting the presence of workpart(s) in input/video frames through a sliding window with multiple scales, and the said models include but are not limited to: FasterRCNN, EfficientDet, etc.; further, to avoid multiple detections a threshold with Non-max suppression follows the windowed detection to select relevant entities to localize the workpart, and the bounding box is then overlaid on each identified workpart for accurate predictions.

Referring to Fig. 8 illustrating flow of instructions involved in the back-end of the digital-twin for performing anomaly detection from the data communicated from the real-twin, and specifically describes importance of error analysis and mode of identifying the anomalies.

The first step of identifying the errors addresses the use of the block named Extracted Details Inflow (610) In which classifying and learning in the present stage connects to the requirements provided in the sub-part of the Anomaly detection, and allows the flow of segmented data and extracted data from the validation and deployment of the present stage.

Further, in the stage of data segregation (620) it considers the features analyzed in the Real-Time Analysis stage [500], and the obtained features are further segregated based on the texture and feature details, which is to be used in segmentation models that would allow pixel-level grouping and vectored analysis for enhanced precision. Followed by, the stage of Error analysis (630, 650) where the segmented texture data arrives, and the decision block loops through the cases of knowledgeable texture anomalies; for instance, in case of a deformed workpart the segmentation explains the deformed structure through a heat-map or contour map of its own which can be regionally analysed. Furthermore, in the stage of Anomaly detection receives the decision for the type of anomaly as prescribed by the stage of Error analysis (630) and the deformed structure is then mapped onto the workparts and is explained.

Inline with Data Segregation, another stage of Error Analysis (640) is employed for the cases of knowledgeable feature anomalies, given any false positives or unidentified new instances from the classes, in which two branches of decision made by this block are: to explain anomaly by features or, proceed to AND gate; wherein the AND gate when finds no anomaly by both error analysis blocks (630) and (650) outputs as Not Anomaly, then the decisions imply the workparts are less likely to be imperfect.

For categorizing type of anomaly based on the features of the detected anomaly the stage of Anomaly Identification (660) is employed, which receive information from the stage of error Analysis (650). The said object is attained by means of techniques such as but not limited to: Explainable AI Class Activation Mapping (XAI-CAM), and Gradient CAM, which would enable the system to handle the problem solving ability without manual intervention in the analysis of workparts in the conveyor belt.
Apart from the architecture of the system involving identification of anomalies as prescribed in the foregoing disclosure, the functioning aspect of the said present system is enunciated through the help of Case-Studies having direct application in the domain of Industrial Automation, and are elaborated as followed:

Example-1: Industrial Example of Assembly Line including Multiple Robotic Arms
In the field of industrial automation & automated material handling having Low ring & Complexity such as Multiple robotic Arms involved in the assembly line, wherein an assembly line includes multiple robotic arms that replicates the process of pallet assembly and processing; and pallets that are portable wooden platforms which can be used to protect, store and transport industrial goods and materials, and also used in furniture designing and manufacturing. In which the objective of prescribed Example-1 is to include the activities of assembling the pallet from the parts flowing through parts supplier and milling the assembled parts. The assembled parts are further required for an inspection of quality to avoid defective parts in final packaging; according to teaching provided by A Martins, et al., in “Shop Floor Virtualization and Industry 4.0”, of 2019 IEEE International Conference on Autonomous Robot Systems and Competitions.

Referring to the Fig. 9 which demonstrates the industrial process described in this scenario begins with the flow of plastic parts from a parts supplier machine; the parts are picked by Robotic Arm 1 (R1) from the supplier and are placed on Conveyor 1 (C1); the C1 transports the parts to the location of Robotic Arm 2 (R2) which the arrived parts and places in the following Machinery Unit. The placed parts are structurally refined by the milling process in this unit, and again are placed onto C1 by R2. Then the refined parts are picked by the Robotic Arm 3 (R3) and are placed onto Conveyor 2 (C2) where the Assembly and Inspection processing would happen. The refined parts arriving at Robotic Arm 4 (R4) on C2 are picked and placed at the assembly station, and the assembled part is processed to a substation having multiple modules which take action of Inspecting the quality of the workpart. The first inspection module Vision System 1 (V1) identifies the workpart, picks it and places it at the inspection table and the Vision System 2 (V2) further inspects the quality of the arrived workpart and classifies if it is an anomaly. Then, the anomaly is moved to a defective part box by a Robotic Arm 5 (R5), else it is processed. Whereas if the processed workpart considered as non-anomaly, is again inspected for details of the workpart from the database and is manually processed by working staff. If it is a case of anomaly, the defective part is inspected in depth by engineers/staff; the decision is then based to remodel the part or recycle it into other parts.

The aspect of industrial usefulness of the Industrial Model prescribed in said Example-1 includes Multiple networks of robotic arms seamlessly work to provide details of complete pallet flow; and Precise prediction of anomalies during the manufacturing process and reduction in the operational and functional costs.

Wherein, said implementation allows the vision systems V1, V2 and Robotic Arm R5 to be replaced with a single Robotic Arm enabled with Digital Twin. The workpart/pallets when assembled on previous conveyor belts C1, C2 arrive at the station of inspection. The Real Twin: Automated Manipulator is initiated along with the Digital Twin as represented in the blocks (10) and (60) of Fig. 1 on the arrival of the workparts, followed by the initiation process, communication between both worlds is established with TCP/IP protocols. This is correlated to blocks (20) and (70) of Fig. 1, and the communication therefore allows sending the data sensed by the vision sensor (40) integrated on the Automated Manipulator/Robotic Arm to the Digital Twin, where it is received (90)., and the process till these blocks can be defined as the preparation stage.

The analysis stage follows the preparation stage in this implementation. In this stage, The non-error environment acknowledged by the vision sensor allows this stage of PLC to follow the next ladder of actuation. The mechanical components of the robotic arm: Links and Joints connected to the PLC through multiple connections get activated in this ladder, referring to block [50] of Fig. 1, which is also represented in Fig. 2. The ladder implements a mathematical solution through a dynamic matrix M defined by:

wij: weight for relative dynamics ?dof between mechanical components i and j respectively.
?dof: the change in the dynamic properties of the respective mechanical component which is restricted to explain the degree of its own.
In this arm activation process, the matrix elements wij is defined as
wij = 1 for {i = j}; 0 for {i ? j}

Thus, the zero values diminish the relative change in between different components providing stability or the RESET condition to the robotic arm.

The activation process also includes resetting the kinematics and dynamics of the arm. Parallel, the system’s back end is run which is related to the block [100] of Fig. 1, to start the process of object analysis.

For instance, if it is assumed that, the workparts are assembled at a rate of 10 parts per hour and are to be inspected for quality, then, the existing industrial solution would be taking another 3-5 minutes more for inspecting each part. Further, the part should be then placed into a defective parts box, which must be further inspected and action needs to be taken. This would result in increased delay slowing down the actual end-end workpart's process flow. Given the architectural benefits of the proposed DT-RAAD system, this issue would be addressed for better productivity.

The DT-RAAD’s process is carried out by the system's backend, the architecture of which is being represented in Fig. 4, in which the main computation of DT is initiated and then further checked for initiation errors. The visual sensor system of the Real Twin module of Fig. 1 is accessed by this system backend, and justifies the need for secure communication between two real and digital modules. The analysis is then carried out by the real-time analysis process, which refers to the block (500) of Fig. 4.
Further, in the real-time analysis, the acquired image is pre-processed and enhanced for better feature extraction; and Multi-object classification (531) and Multi-object detection (542) blocks of Fig. 5 analyze the assembled part and extract the features completely in a very short time, typically less than 5 seconds. The analysis is further carried out to inspect anomalies and regenerate images for improving knowledge in self, and the anomaly observation block (560) inspects the anomaly in the workpart, thereafter been reported for qualitative anomaly inspection.

For the classification is the ensemble network which is trained with a dataset of 1650 images of multiple classes: Wooden pallet, Covered pallet, Damaged pallet. This dataset is divided into 65% train, 10% validation, and 25% of testing to observe the optimized network as shown in Table 1. The depiction of the ensemble network is referred to in Fig. 5.
Table. 1 Description of the dataset of available knowledge
Class No. of images Division for training Division for validation Division for testing
Wooden Pallet 595 400 50 145
Covered Pallet 595 400 49 146
Environmental 460 325 43 92
The analysis is further carried out to inspect anomalies and regenerate images for improving knowledge in self. The anomaly observation block [560] inspects the anomaly in the part. It is then reported for qualitative anomaly inspection. The unguided CSRT further localized unrecognized objects in the system, allowing a vision for anomaly detection.

Visual anomaly inspection carried out by the block (600) is shown in Fig. 4; in the view of said Example-1 of automated part assembly, anomalies in production are mostly deformed structures or misalignments; and if the workpart is observed to be an anomaly, the explanation of the workpart is carried out by identifying the type of anomalies by texture and features. These identification and explanation can be referred to the blocks (640) and (660) of Fig. 8. The anomaly explanation here is provided by contour maps of assembled parts, overlaid with properties of non-defective parts. The explanation further proceeds to the process of diagnosis, which is carried out by the block (900) of Fig. 4. The diagnosis provides a solution to the detected anomaly and is looped through analysis fixing the defective workpart. In case of a non-defective part, the workpart just flows through the same conveyor belt, explained with its details like part number, time, production rate and resources consumed, which concludes the processing of the analysis stage.

Finally, the actuation stage follows the analysis stage, in which the action is taken by the data actuation block (120) of Fig. 1; and action in the Example-1 includes moving the assembled part to either a workpart having flaw or the next station. The main advantage of this process is that the defective workpart box does not need another station for defect analysis, and a complete explanation of the anomaly along with the solution is provided to the central system in only less than a minute time.

The advantageous merits of employing said Example-1 as a part of the industrial automation includes reduced Complexity on the belt C2, with reduction in the number of systems embedded on it; reduction in human intervention thereby possibility of human errors in the process of identifying anomalies; reduction in trade-off between quality inspection and time and promotion of real-time monitoring; and optimization of overall process of inspection as the result of inclusion of present DT-RAAD model.

Example-2: Pharmaceutical 3D Printing of Medicines
In the field of industrial automation & automated material handling having High Risk and Fast Production wherein the said medical pills are printed with complex structures aiming for accuracy in dosage and intake, which would prominently take position to promote drug absorption with high level control on adverse reactions in the human bodies. Further, the objective of the printing of the pills includes activities such as to monitor the process of structural printing, materials and resources, while the system needs to determine the dosage of the printed pill and explain any deformation to avoid misinformation. Thus, unsupervised printing leads to problems in packaging, which then affects the person taking the medical pills.

According to the teaching provided by Li H. Fan, et al. in “3D Printing: the potential technology Widely Used in Medical Fields” of Journal of Biomedical Materials Research Part-A, 3D printing was introduced in the early 1980s in manufacturing industries as Additive Manufacturing, and it is advanced with time to manufacture complex layers of medical devices, organs and medicines, and Robotic arms in these applications also were used to organize and analyse the production process to maintain the risk-free environment.

According to the teaching provided by A. Goyanes, et al. the procedures of printing pills have also been flexible with upgraded medical equipment and printing devices; one of the procedures: Fused Deposition Modeling (FDM) of 3D printed pills focuses on printing pills with different sizes, shapes and dosages. Further, in several cases the printing device in this process is infused with the materials: Hydroxypropylcellulose, Mannitol, and Magnesium stearate, and the properties of 3D printers are set to the requirements of dosage and pills are processed in batches. The printed pills were then tested with manual engineering and involved students of universities to further check for results. The process is highly supervised to avoid any mistakes and monitored very carefully.

Implementation of presently disclosed DT-RAAD system in Example-2 aims in networking 3D printing devices to a central system to help the production and monitoring process, and allows the printing system to communicate real-time information of pills processing with visual structure and time.

The preparation stage is initiated to start the process; firstly, the stage initiates both real and digital twin modules referred to blocks (10) and (60) of Fig. 1; further, communication between both the modules is established with TCP/IP protocols to ensure security of data transmission. Which, correlates to the blocks (20) and (70) of Fig. 1 and the communication establishment allows the digital twin module to sense the environment around the printing device, referring to block (80) of Fig. 1. The perception allows to define the workspace and behavioural properties of the printing arm, further enabling quicker deployment of the printing environment. Following the perception, the vision sensor integrated on the printing arm senses the inflowing materials and unprocessed medical structures, and communicates to the digital twin module where it is received, and the communication of visual data is correlated with the block (40) and (90) of Fig. 1 which concludes the initiation of the printing system and defines the preparation stage.

The analysis stage follows the preparation stage, firstly focusing on the identified problem statements: classification and tracking, and the system’s backend is run/initiated as described in the block (100) of Fig. 1 which also correlates with Fig. 3 describing the process in depth. Continuing the initiation, a visual sensor system is accessed to get the real-time data of the environment, block (400) of Fig. 4 real-time analysis is then carried out by the (500) on receiving input data.

Real-Time Analysis block (500) of Fig. 4 processes the data received in the format of video, which is used to capture fast production systems. Said video frames are pre-processed and enhanced by the block (532) of Fig. 5 and immediately the block (531) processes the medicinal objects based on material and size observed. Then the same will be based on the existing knowledge of the medical industry which can be input prior to system initiation. Such classification eliminates the challenge to later classify medicines based on observation of human response. Further, the process is controlled by the blocks (541) and (542) of Fig. 5 to track the classified medicines to provide detailed information, and the Information here includes the dosage and manufactured date.
For classification is the ensemble network, which is trained with a dataset of 480000 images of multiple shapes: PyTorch Shapes Dataset. This dataset is divided into 65% train, 10% validation, and 25% of testing to observe the optimized network. The depiction of the ensemble network is referred to in Fig. 5 Ensemble Network of Multi Object Classification.
Table. 2 Description of the Shapes dataset
Dataset Features Values used
Shape Cubic, Spherical, Capsule, Plate
Scale of Object In zooming range of 0.5 - 1.0
Orientation of Object [-30, 30] in degrees relative to straight object

The anomaly and quality inspection is then carried out by Anomaly Detection block (600) of Fig. 4, in which the said anomalies in the present Example-2 includes but are not limited to: Heavy dosage, low-demand dosage, and missing material. The explanation by texture and features are processed by blocks (640) and (660) of Fig. 8 in case of anomalies, then the anomalies are carried to fault diagnosis block (900) of Fig. 4 and solution is provided.

Finally, the actuation stage follows the analysis stage, and processed by the data actuation block (120), in which, the printing arm responds to the inspection analysis performed by the digital twin and takes action on the processed package. The action includes picking and placing the medical palette in the right direction or alerting the central system with all the explanatory details of real-time analysis, referred to block (110) of Fig. 1 which concludes the process and reports to the central system with information such as but not limited to: dosage type, batch number, time of processing, quality of medicine, resources left in the production system, etc.; Thus, the presently disclosed DT-RAAD system advantageous for FDM procedure of printing pills, monitoring the complete process within the seconds for the deployment in real-time, and visual inspection module of DT-RAAD employing a self-learning mechanism helps in analyzing the medicine and reducing the risks in actual manual testing.

In the aspect of optimization of Real-Time manipulator the Real-twin and Digital-twin modules promote real-time through collaborative working and processing; a single instance of digital twin will be capable of handling multiple tasks of classification, detection, inspection, and monitoring. Additionally, the robotic arm feasibly handles the flowing objects on the factory belt and responds to the analysis in parallel thereby providing an optimized workflow with reduced complexity for real-time response.

The presently disclosed system meant for visual anomaly inspection and deployment provides a base for multi-arm deployment, and a single instance of an arm network capable of communicating with Robotic-Arm and Digital-Twin, when extended to multiple robotic arms lead to better inspection towards industrial automation. These proposed advancements are required not only to reduce the computational costs but also to centralize the work for a reliable system, which can be continuously monitored and controlled remotely over a network at any time.

Although a particular exemplary embodiment of the invention has been disclosed in detail for illustrative purposes, it will be recognized to those skilled in the art that variations or modifications of the disclosed invention, including the rearrangement in the configurations of the parts, changes in steps and their sequences may be possible. Accordingly, the invention is intended to embrace all such alternatives, modifications and variations as may fall within the spirit and scope of the present invention.

The foregoing descriptions of specific embodiments of the present invention have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and obviously many modifications and variations are possible in light of the above teaching.

A robotic arm is as a physical networked machine that helps in monitoring the process flow given any factory environment. This will involve developing a self-correcting smart process control facility where the digital twin would be used to replicate the physical process and use the Digital Twin (DT) model to extend the debugging, testing and reforming processes before physical implementation in the factory environment. Importantly, the robotic arm would be used to correct and reform the physical process.

The work discussed in K. Jahnavi and P. Sivraj, "Teaching and Learning Robotic Arm Model", 2017 International Conference on Intelligent Computing Instrumentation and Control Technologies (ICICICT), pp. 1570-1575, Jul. 2017, Kerala, India [Ref: 1] establishes the base for DT. It first provides the design and development of a 5 Degree of Freedom (DoF) Robotic Arm. The robotic arm has been deployed externally with a basic Arduino micro controller along with the motors actuating to the system. The environment is controlled through a graphical interface, which allows the robot to self-learn it’s movements. The Arm movements were defined to perform only tasks such as holding, picking and placing. However, some focus on vision sensors for real-time deployments with low latency is still missing. Though the work provides the definition and visualisation of a Digital Twin of Robotic Arm, it does not provide information on real-time analysis for manufacturing industries, missing the details of networking components with pipelined structure.

The connected multiple robotic arms and deployments in a network are discussed in Lee, J.-D., Li, W.-C., Shen, J.-H., & Chuang, C.-W. “Multi-robotic Arms Automated Production Line.” 4th International Conference on Control, Automation and Robotics (ICCAR), pp. 26-30, Apr. 2018, Auckland, Newzealand [Ref: 2]. In this work, the deployed robotic arms were able to perform pick and place operations with a 6-axis arm, Visual Inspection Gantry Arm, Delta Arm etc. which replicates the main process of an industrial assembly line. In addition, various industrial examples solved by the designed system were discussed in this work. However, the limitation is that it was explored only for a surface analysis on data observation and flow control. Another aspect that is not addressed is a network interface establishment of a robotic arm with an effective fault tolerance mechanism.

In the work S. Bratchikov, A. Abdullin, G. L. Demidova and D. V. Lukichev, "Development of Digital Twin for Robotic Arm," 2021 IEEE 19th International Power Electronics and Motion Control Conference (PEMC), 2021, pp. 717-723, Apr. 2021, Gliwice, Poland [Ref: 3], the DT is deployed by a combination of dynamics, intelligence, and networking characteristics. One of the important tasks in developing a DT for Robotic Arm is to create a dynamic model accounting to all kinematics and dynamics. Kinematics explains the properties of mass and acceleration of the arm pieces and objects, whereas the dynamics explains the complete body’s concepts of inertia and stability. The action of the robotic arm was replicated using polynomial-based techniques in control systems to represent behaviour of the arm through both Kinematics and Dynamics. Even though the work sufficiently demonstrates the behaviour of the robotic arm, the challenge that still is missing is on reliability. Reliability can be strengthened by infusing the real-time with ablation study and comparisons on developed solutions of DT connecting the concepts of Kinematics and Dynamics. Also, this work does not talk about data analytics and network interface.

An extension to work [Ref: 3] with the demonstration of a 4 DoF robotic arm along with the connection is done in S. Khrueangsakun, S. Nuratch and P. Boon Pramuk, "Design and Development of System for Real-Time Web-based Visualization and Control of Robot Arm," 2020 5th International Conference on Control and Robotics Engineering (ICCRE), 2020, pp. 11-14, Apr. 2020, Osaka, Japan [Ref: 4]. The deployed robotic arm visualization enabled the concept of Digital Shadowing of CPS. It realizes the joints and links actuation in real-time with less latency (less than one sec). However, this work provides a real-time response in between real and digital worlds only using the angles of arm’s joints. The computational analysis with the factors such as inference time (compute and response time) is not addressed. Another aspect that is not carried out in the prior art is the operational analysis with dynamics of robotic arm movements.

These mentioned conceptual demonstrations of Digital Twins and Robotic Arms open the possibility of bringing the aspect of advanced product analysis into research with real-time considerations. An analysis on real and digital worlds bridges the uninterpreted knowledge of the working. Consequently, real-time deployment of analyzed data enhances the representation. Hence the proposed invention called DT-RAAD (Digital Twin based Robotic Arm for Anomaly Detection) establishes the primary base in dealing with the industrial level analysis of Digital Twins, including the required concepts of anomaly detection.

Table I and Table II provides a detailed comparison of the features of the prior art with the proposed DT-RAAD system.

Table 1. Prior Art Comparison - Data Modeling and System dynamics for Robotic Arm
Prior Art
(references) Robotic Arm Concepting Kinematics and Dynamics Networking components and Multiple Systems Data Analysis & Learning system Pipelined Structure
[Ref: 1] Yes Yes No Yes No
[Ref: 2] No Yes Yes No No
[Ref: 3] Yes Yes No No No
[Ref: 4] Yes Yes Yes No No
Proposed Work - DT-
RAAD Yes Yes Yes Yes Yes
Table. 1 compares the existing applications of industrial models with DT-RAAD. It can be seen that the Prior Art does not look into all aspects of Digital Twin, Networked model, Data Analysis and the Robotic Arm movement. In fact, the combination of Robotic Arm and pipelined architecture has not been done in any of the previous work. Further, many of the existing works have not considered networking the components which slows down the process of production. The proposed invention, DT-RAAD provides an adaptive yet integrated approach that combines all modules: Robotic Arm, Kinematics and Dynamics, Networking, Data Analysis, and Pipeline structure.

Table 2. Prior-Art comparison of analytical techniques for Anomaly Detection with Digital Twin
Prior Art
(references) Data Analysis and Learning System Pipelined Structure
Real-World Data
(Industrial requirements) Machine Learning Models Anomaly Inspection Real Twin Processing System Digital Twin Processing System
[Ref: 1] No Yes No Yes No
[Ref: 6]
US Patent, US20180345496A1 Yes Yes No No Yes
[Ref: 7]
US Patent, US9652354B2 No Yes Yes No Yes
[Ref: 8]
US Patent, US10875176B2 No Yes No Yes Yes
[Ref: 9]
US Patent, US9671777B1 No No No Yes Yes
[Ref: 10]
EP Patent, EP3376325A1 Yes No No No Yes
Proposed Work - DT-
RAAD Yes Yes Yes Yes Yes

Table. 2 further compares the proposed DT-RAAD with existing prior-art. It can be observed that except the work presented in US Patent, US9652354B2 [Ref: 7], existing prior art does not combine anomaly inspection and Digital Twin processing. Importantly, the prior art [Ref: 7] does not provide any real-time analysis and also does not capture the industrial requirements.

,CLAIMS:We Claim,

1. A method for identification of process anomalies comprises
a. at least one Microcontroller for assembly lines and robotic arms;
b. a memory connected to the at least one process;
c. at least one visual sensor for data capturing connected to robotic arms;
d. at least one graphical engine consisting part of the digital-twin allocated to the memory to the real-world visualisation module to replicate the process of the real twin;
e. connection to system and connection to the controller wherein the processor of robotic arms is initiated and connected with the digital twin through data communication protocols;
f. robotic arms activation acquires response from its parallel digital-twin;
g. the digital-twin identifies robotic arms functioning and the environment perception from the sensor's visible range defines adjustment of the robotic arms;
h. acquiring data of robotic arms by visual sensor and the captured data and sensor data are analysed by digital twin for analysis of visual cues of the objects;
i. activation of robotic arms in the real-twin along with the digital-twin will execute backend system, to perform the crucial analysis that includes - identification, classification, description, explanation and anomaly detection;
j. analysis Data from the communication among the robotic arms and digital twin based on the data analysis, classification anomaly is detected;
k. the data received from both real-twin module and digital-twin module analyze test results of the execution to select an optimized robotic arms control.

2. The method for identification of process anomalies as claimed in claim 1 wherein, real-time data capturing comprises:
a. data capturing of processing system for ongoing qualitative assessment of process;
b. replicating the physical process using digital twin for networking of the industrial automation system with manufacturing environment so as to perform real-time detection;
c. performing real-time detection with visual sensor for data capturing;
d. corrective actions by industrial automation system for the detected anomalies.

3. The method for identification of process anomalies as claimed in claim 1 wherein, monitoring of the flow of industrial processes by shadowing the said process using a digital-twin functioning as counterpart of the robotic-arm, which provides means for sensing, communication, & control for imparting intelligence in the industrial processes with replication in the digital mode.

4. The method for identification of process anomalies as claimed in claim 1 wherein, real-time remote communication with the ongoing industrial manufacturing processes, for reformation of the said physical process by employing adaptive approaches that combines output of the modules such as robotic arm, kinetics, dynamics, networking, data analysis, & pipeline structure.

5. The method for identification of process anomalies in robotic arm comprises:
a. detecting anomalies in an robotic arm with visual sensor for data capturing in material handling, object classification, and automated data retrieval;
b. integrating robotic-arm and simulation model using a digital-twin for product inspection and multi-modal information analysis by digital control, vision sensor, data analysis, and machine learning;
c. corrective actions by industrial automation system for the detected anomalies.

6. The method for identification of process anomalies as claimed in claim 5 wherein, the robotic-arm entailing an end-effector having multiple degrees of freedom for initializing self-corrective actions communicated by the replicated digital-twin.

7. A system for identification of anomalies comprises:
a. at least one visual sensor on robotic-arm for collecting data from industrial manufacturing process in the pictorial, image or video;
b. robotic-arm is integrated with a digital-twin;
c. shortening it to the designated groups, processing it to the readable format, for detecting anomalies;
d. sharing data to the central digital-twin system; wherein, digital-twin functions to monitor overall operation of robotic-arm to maintain processing of products and sharing data to the central digital-twin system;
e. whereas the robotic-arm in closed-loop with its digital-twin, to perform corrective actions communicated from the digital-twin in robotic-arm.

8. The system for identification of anomalies as claimed in claim 7 wherein, a robotic-arm made of multiplicity of rigid links movably and rotatably interconnected with one another for facilitating displacement and made to interact with manufacturing process through
a. at least one visual sensor for detection of anomalies and rectifying the influence of the same in an adaptive manner;
b. at least one sensors conferred with the robot-arm to fetch data of external shape & size, surface & sub-surface defects, colour & contour from the manufacturing process line and compare with pre-fed data, and determine anomalies.

9. The system for identification of anomalies as claimed in claim 7 wherein, digital-twin involved for replicating the physical process for performing debugging, testing, & reforming the said process before physical implementation in the manufacturing environment, a digital-twin is deployed in the said manufacturing environment and networked with robotic-arm capturing visual data which is further fed to the said central digital-twin for initializing the corrective actions required for optimization of the overall process.

10. The system for identification of anomalies comprises
a. integrating robotic-arm with digital-twin;
b. initiation of robotic-arm microcontroller and digital-twin graphical engine;
c. fetching data of robotic-arm activation and conveying robotic-arm activation data to digital-twin the system back-end for performing data actuation, wherein real-time analysis is carried-out along with data verification and resource check for detection of anomalies;
d. real-time analysis of the anomalies of an object visibility check and occlusion check is preformed followed by multi-object detection involving sampling and weighing of the captured data;
e. data segregation is carried-out for categorizing anomalies in texture or feature, for determining type of said detected anomaly, so as to correspondingly preform corrective actions, and sustaining the optimized state of the involved manufacturing process.

Dated this 24th Day of Dec, 2021.