Description:SYSTEM FOR PROVIDING AN AUTOMATIC ASSEMBLY OF MECHANICAL PARTS OF PRODUCT AND METHOD THEREOF
BACKGROUND
Technical Field
[0001] The embodiment herein generally relates to assembly of mechanical part and more particularly, to a system for providing an automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback and method thereof.
Description of the Related Art
[0002] Nowadays, Flexible automatic assembly systems have become an essential aspect in the mechanization of industry. The Flexible automatic assembly systems enables these systems to adapt and reconfigure themselves to handle various tasks and product variations in industrial settings. The challenge concentrates on exploring the intricacies adhered with the part assembly processes within the context of real world industrial automation. The issues aroused are dependent on factors like variations in product design and assembly where it is easy to opt a robotic hand to assemble a single part but it becomes tedious when multiple parts of different sizes are involved.
[0003] Accordingly, there remains a need for system for providing an automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback and method thereof.
SUMMARY
[0004] In view of the foregoing, embodiments herein provide a system for providing an automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback is disclosed. The system includes one or more vision sensors, and a control platform. The one or more vision sensors that is integrated with a designed pneumatic robotic end-effector to observe environment through captured real-time image file. The control platform is configured to perform: deciding one or more control points for the end-effector to pick an object and decide, deciding target points to perform assembly task using template matching with the captured real-time image file, determining the control point in a next frame using optical flow to assemble one or more different parts, obtaining visual deviations through the template matching for the assembly of the one or more different parts, and determining one or more input commands for the robot; and instruct the robot’s operating software drive command to operate the industrial robotic arm.
[0005] In some embodiments, the system comprises a graphical user interface.
[0006] In some embodiments, the robotic arm comprises a robot servo motor.
[0007] In some embodiments, the system comprises a data acquisition module to receive the one or more commands from the controller.
[0008] In an aspect of the embodiments herein discloses a method for providing an automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback. The method includes integrating one or more vision sensors with a designed pneumatic robotic end-effector to observe environment through captured real-time image file. The method further includes configuring a control platform to preform deciding one or more control points for the end-effector to pick an object and decide, deciding target points to perform assembly task using template matching with the captured real-time image file, determining the control point in a next frame using optical flow to assemble one or more different parts, obtaining visual deviations through the template matching for the assembly of the one or more different parts, and determining one or more input commands for the robot; and instruct the robot’s operating software drive command to operate the industrial robotic arm.
[0009] In some embodiments, the method comprises receiving the one or more commands by a data acquisition module from the controller.
[00010] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein, and the embodiments herein include all such modifications.
BRIEF DESCRIPTION OF THE DRAWINGS
[00011] The embodiments herein will be better understood from the following detailed description with reference to the drawings, in which:
[00012] FIG. 1A illustrates a block diagram of system for providing an automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback, according to some embodiments herein;
[00013] FIG. 2 illustrates a block diagram of data acquisition and image processing, according to some embodiments herein;
[00014] FIG. 3 shows a flow chart of the automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback, according to some embodiments herein; and
[00015] FIG. 4 illustrates a method for providing an automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback, according to some embodiments herein.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
[00016] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
[00017] As mentioned, there remains a need for a system and a method for providing an automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback. Referring now to the drawings, and more particularly to FIGS. 1 through 4, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments.
[00018] FIG. 1A illustrates a block diagram of system 100 for providing an automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback, according to some embodiments herein. The system 100 includes one or more vision sensors 102A-102C, USB port 104, a power supply 106, a graphical interfacing with system 108, a software programming with control platform 110, robot protocols 112, robot servo motor 114, robot 116, a workspace 118, and one or more mechanical parts 120. The one or more vision sensors 102A-102C are integrated with a designed pneumatic robotic end-effector to observe environment through captured real-time image file. The control platform 110 is configured to: deciding one or more control points for the end-effector to pick an object and decide, deciding target points to perform assembly task using template matching with the captured real-time image file, determining the control point in a next frame using optical flow to assemble one or more different parts, obtaining visual deviations through the template matching for the assembly of the one or more different parts, and determining one or more input commands for the robot; and instruct the robot’s operating software drive command to operate the industrial robotic arm.
[00019] In some embodiments, the system 100 includes a graphical user interface. The robotic arm includes a robot servo motor. The system includes a data acquisition module to receive the one or more commands from the controller.
[00020] In some embodiments, the Vision sensor Real Sense SR300 has been integrated with designed pneumatic robotic end-effector to observe the environment through captured real-time image file. The system 100 decides the Control Points from where, end-effectors can pick the object and decide the Target Points to perform the task accordingly by using Template matching. The system 100 ddetermines the control point in the next frame using optical flow to assemble the different parts. At the last get visual deviations through Template matching. The system 100 determines input command (Visual deviation multiplied by gain). The system 100 enters the robot operating software drive command to operate the industrial robotic arm. The system 100 implements the developed sensor integrated automatic control algorithm for industrial application and to perform operational tasks.
[00021] In a non-limiting example embodiment, the system 100 is operated as per following steps: start an Operating System Microsoft Window or others OS, first open the Industrial robot Driver and software Toolbox to operate any kind of Industrial Robot, open the workspace of Industrial robot Driver, Click the Control icon will be a green icon in online mode, open the robot controller, and fix the robot axis mode to operate the system, programming the control system and recode values on decided position and orientation to operate the system, now open the visual Studio programs, open the industrial robot control picking and placing program to operate the system, then to decide the template run the program for pick and place operation, after decide the template, run the control mode for pick and place operation, Select the program according to part of assembly operation, to decide the template, debug the selected program, to decide the template, debug the selected program, then select and decide the template at Blue box, than clos the running icon of program to start control mode, and finally you can control the whole system according to selected program.
[00022] Another significant issue in industrial automation is the ability to identify and locate objects accurately. Vision sensors provide the robotic arm with the capability to recognize objects based on visual cues such as shape, color, or texture. This enables the robotic arm to distinguish between different parts or components, even in complex and cluttered environments, facilitating efficient pick-and-place operations and assembly tasks. By incorporating vision sensors, the robotic arm gains the ability to adapt its actions based on real-time feedback from the environment.
[00023] Robotic arms with vision sensors can assist with household chores such as cleaning, cooking, and organizing. The vision sensors enable the arm to perceive and navigate through the environment, making it capable of performing tasks autonomously.
[00024] Vision sensor-based robotic arms can serve as assistive devices for individuals with disabilities, helping with tasks such as feeding, dressing, and personal care. The vision sensors enable the arm to adapt to the user's movements and provide assistance tailored to their needs.
[00025] The system 100 provide feasibility of development of a vision sensor integrated robotic control algorithm to be used in an unstructured environment for automatic assembly operations. It’s easy to choose a robotic hand to assemble one part, but problem arises for the cases where multiple and different sizes of parts are involved, to handle without any calibration. As per the requirements, developed an algorithm to control the automated assembly planning, using vision integration and image processing, for its use in industrial automation, specifically for part assembly involving mechanical and electronic products.
[00026] FIG. 2 illustrates a block diagram 200 of data acquisition and image processing, according to some embodiments herein. The block diagram 200 includes an input 202, a sense module 204, a think module 206, an act module 208, an output 210, and a feedback module 212. The sense module 204 receives the input 202. The sense module 204 retrieves data from the vision camera. The input 202 is a live feed from the vision camera. The think module 204 executes control algorithms. The act module 208 connects to the robot motors. The output 210 is received by the robot.
[00027] FIG. 3 shows a flow chart of the automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback, according to some embodiments herein. At step 302, start. At step 304, obtaining vision sensor-based image file. At step 306, deciding control points and target points (template matching). At step 308, determining the control point in the next frame using optical flow. At step 310, getting visual deviation. At step 312, determining input command (visual deviation multiplied by gain). At step 314, enter the command to operate the robot. At step 316, implementation of the developed system for various assembly operations. At step 318, end.
[00028] FIG. 4 illustrates a method 400 for providing an automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback, according to some embodiments herein. At step 402, the method 400 includes integrating one or more vision sensors with a designed pneumatic robotic end-effector to observe environment through captured real-time image file. At step 404, the method 400 includes deciding one or more control points for the end-effector to pick an object and decide. At step 406, the method 400 includes deciding target points to perform assembly task using template matching with the captured real-time image file. At step 408, the method 400 includes determining the control point in a next frame using optical flow to assemble one or more different parts. At step 410, the method 400 includes obtaining visual deviations through the template matching for the assembly of the one or more different parts. At step 412, the method 400 includes determining one or more input commands for the robot. At step 414, the method 400 includes instructing the robot’s operating software drive command to operate the industrial robotic arm.
[00029] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the scope of the appended claims.
, Claims:We claim:
1. A system (100) for providing an automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback, the system (100) comprising:
one or more vision sensors (102A-102C) that is integrated with a designed pneumatic robotic end-effector to observe environment through captured real-time image file; and
a controller platform (110) that is configured to control a developed algorithm, wherein the controller platform (110) is configured to perform:
deciding one or more control points for the end-effector to pick an object and decide;
deciding target points to perform assembly task using template matching with the captured real-time image file;
determining the control point in a next frame using optical flow to assemble one or more different parts; obtain visual deviations through the template matching for the assembly of the one or more different parts; and
determining one or more input commands for the robot; and instruct the robot’s operating software drive command to operate the industrial robotic arm.
2. The system (100) as claimed in claim 1, wherein the system comprises a graphical user interface.
3. The system (100) as claimed in claim 1, wherein the robotic arm comprises a robot servo motor.
4. The system (100) as claimed in claim 1, wherein the system comprises a data acquisition module to receive the one or more commands from the controller.
5. A method (400) for providing an automatic assembly of one or more mechanical parts of a product by an industrial robotic arm using a visual based senor feedback, the method (400) comprising:
integrating one or more vision sensors with a designed pneumatic robotic end-effector to observe environment through captured real-time image file; and
configuring a control platform to preform:
deciding one or more control points for the end-effector to pick an object and decide;
deciding target points to perform assembly task using template matching with the captured real-time image file;
determining the control point in a next frame using optical flow to assemble one or more different parts; obtain visual deviations through the template matching for the assembly of the one or more different parts; and
determining one or more input commands for the robot; and instruct the robot’s operating software drive command to operate the industrial robotic arm.
6. The method (400) as claimed in claim 5, wherein the method comprises receiving the one or more commands by a data acquisition module from the controller.