Description:FIELD OF THE INVENTION
This invention relates to pet robot system to mimic human gestures.
BACKGROUND OF THE INVENTION
Pets are human friends who will go to any length to defend them and keep intruders out of their homes. We suggest designing a robot that resembles a dog employing servo motors based on our computer vision algorithm and hand gesture detection that can instruct the robot to move and accomplish a certain task. This robot's outside body is made of sun board sheets, a camera identifies hand gestures, servo motors move the body, and a lithium polymer battery provides long-term power.
The robot can now understand and respond to human instructions without a hitch as the computer vision and hand gesture detection have been integrated. It is possible to interact with the machine naturally and intuitively by having the algorithm trained to identify various hand signs, therefore, making it an ideal companion for all daily activities or even an added home security layer.
This will be extremely useful for disabled people who would have a trustworthy assistant to get things for them and warn them about threats from their surroundings such as fire. This makes it possible for the robot’s movements to glide smoothly due to the overall low weight of the robot, since lightweight sun board sheets are used on its outer surface for durability purposes. For instance, servo motors that have been meticulously adjusted provide precise action which emulates natural dog behavior. Moreover, when paired with a lithium polymer battery this ensures longer operation periods hence one does not need to charge it time after time becoming more convenient over extended periods of time. The aim of this new design is not only to offer companionship like pets but also integrate modern technology into everyday life in order to help solve some problems associated with it.
CN112571433A The invention discloses an emotion communication method of a pet robot, which detects the action, expression and sound instruction information of a person through a data acquisition module; analyzing the detected information content through a processor, and making a corresponding control instruction to a motion module, an image output module and a voice output module; the motion module controls the motion of the head and the four limbs according to the control instruction of the processor, the image output module displays the movement of eyes and/or displays different images to express the emotion of the pet robot, and the voice output module simulates and outputs the sound of an animal to express the emotional state of the pet robot. Different actions of the head and the four limbs of the pet robot are achieved, a display is arranged at the eye position of the pet robot, the actions of the eyes are simulated, different icons are displayed to show different emotions of the pet robot, conversation is not carried out simply through sound and people, and therefore the communication mode between the pet robot and the people with richer emotional expression is achieved.
RESEARCH GAP: Portability: a power bank and Li-Po battery makes the system very portable and independent of fixed power sources. Real-Time Processing: By using YOLOv5, real-time gesture recognition is promised to be in a responsive action to the command.
CN101474481B The invention relates to an emotion robot system, in particular to a robot which can generate human-simulated facial expression and can interact with people. The emotion robot system is composed of a head part system of the robot with six facial expressions and a software platform which takes PC as a control center; the emotion robot detects the information of external environment by the equipment such as an infrared sensor, a mic, a camera and the like. The PC carries out emotive feature extraction by the collected information of the external environment, and then voice emotion is analyzed and the facial expression of human face is detected, and then the emotion expressed by the robot is determined. The emotion robot expresses the emotion by voice output, facial expression and body language. The PC sends out instructions to a singlechip by serial ports, and the singlechip drives the motor to move for generating the facial expression and the body language of the robot after receiving the instructions. The emotion robot system can be used for domestic service robots, guest-greeting robots, explication robots and the man-to-machine interaction research platform.
RESEARCH GAP: Customization: The system is highly customizable for recognizing an extensive number of gestures and actions. Durability: Sun board sheets provide a tough yet lightweight outer body of the robotic pet.
US10166680B2 Using various embodiments, an autonomous robot using data captured from a living subject are disclosed. In one embodiment, an autonomous robot is described comprising a robotic skeleton designed similar to that of a human skeleton to simulate similar movements as performed by living subjects. The movements of the robotic skeleton are resultant due to control signals received by effectors present near or on the robotic skeleton. The robot can be configured to receive sensor data transmitted from a sensor apparatus that periodically gathers the sensor data from a living subject. The robot can then process the sensor data to transmit control signals to the effectors to simulate the actions performed by the living subject and perform a predictive analysis to learn the capability of generating spontaneous and adaptive actions, resulting in an autonomous robot that can adapt to its surroundings.
RESEARCH GAP: Scalability: The 16-channel servo module means that if required, additional motors could be added to the robot. Strong Performance: Powerful machine learning in TensorFlow ensures accurate and true gesture recognition.
None of the prior art indicate above either alone or in combination with one another disclose what the present invention has disclosed. This invention relates to Pet Robot to Mimic Human Gestures.
SUMMARY OF THE INVENTION
This summary is provided to introduce a selection of concepts, in a simplified format, that are further described in the detailed description of the invention.
This summary is neither intended to identify key or essential inventive concepts of the invention and nor is it intended for determining the scope of the invention.
To further clarify advantages and features of the present invention, a more particular description of the invention will be rendered by reference to specific embodiments thereof, which is illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail with the accompanying drawings.
The hand gesture recognition system is an advanced integration of hardware and software, designed to facilitate natural and intuitive interaction between humans and robotic pets, in this case, a self-assisted robotic pet. The major components include a camera, Raspberry Pi, power bank, 16-channel servo module, 12 servo motors, voltage Buck converter, Li-Po battery, and sun board sheets. These components are intricately connected and synchronized to ensure the system operates efficiently and effectively.
The camera, placed in front of the user, captures pictures of the user's hand gestures continuously to initiate the system operations. This captures images and processes them on a Raspberry Pi. The OpenCV library is used to pre-process the photos taken. The image pre-processing step will undergo a series of procedures that set the image size at some standardized dimension and normalize the pixel values. Image resizing makes all the images even; it is crucial for proper analysis. Reshaping does this by adjusting the pixel value of an image to a standard scale, improving the exactness of a machine learning analysis.
The pre-processed images are passed through machine learning-implemented models in TensorFlow and YOLOv5. TensorFlow has good support for training and deploying the models on a machine learning tool, but YOLOv5 has been optimized for speed, hence the real-time detection of objects. Because YOLOv5 is fast and accurate in the detection of objects, its efficiency is beneficial in an application with real-time demand.
Hand gesture recognition would then refer to detecting specific patterns in these particular images, patterns that correspond to already predetermined commands. The pre-trained YOLOv5 model detects such patterns while it is exposed to a dataset of different hand gestures. These large numbers of labelled images fed into the system allow the system to learn the features and patterns associated with each type of hand gesture. After detecting the gesture, the Raspberry Pi maps it to a corresponding command. The command is sent to the 16-channel servo module. The servo module translates the command into actions. This in return, controls the 12 servo motors. Each motor executes an action that moves the legs, head, or tail of the robotic pet.
The servo motors literally do the various movements that the robotic pet does—when the pet moves forward, turns around, or does a trick. The voltage Buck converter supplies the servo motors with a constant and smooth voltage so that the motors will be exact and provide a very fluid motion. If there were unsteady voltage, it could make erratic movements, but it wouldn't be noticeable if just by a small amount. The Li-Po battery is used for providing the necessary energy to power the entire system for continuous operation. A high energy density, long life-type Li-Po battery is used with a long service life without frequent recharging and thus finds application with robotic pets for use over extended periods. The outer sun-board sheet body of the robotic pet makes it lightweight yet strong enough to perform and sustain various interactions with easy movements.
More technically speaking, the Raspberry Pi is like the central processing unit that effectively coordinates the whole operation by managing data flow from the camera to the servo motors. So when the image of a hand gesture is taken, raw visual data is relayed to the Raspberry Pi. The raw data relayed is then pre-processed by the Raspberry Pi using the OpenCV library, which includes the resizing of the image into a standard dimension and normalizing pixel values. It is a crucial prepossessing step in which the images are brought to the most appropriate form for further analysis so that gesture recognition can become accurate and reliable.
After preprocessing, the prepared image data undergoes implemented machine learning models via TensorFlow and YOLOv5. TensorFlow gives quite a robust framework for building and deploying machine learning models that can interpret complex visual patterns.
YOLOv5 is known to be fast and accurate in object detection in real-time; in this regard, we use YOLOv5 to detect and recognize hand gestures. The model has been trained on large datasets in this task to capture different variations of hand movement patterns to quickly identify the specific pattern of pictures corresponding to predefined commands.
This recognition process involves sophisticated computations and real-time processing—tasks that are easily handled by the computing capabilities of the Raspberry Pi. Once a hand gesture has been identified, the system translates that visual input into an actionable command. The 16-channel servo module then sends out this command to control the servo motors that drive the physical corresponding movements of the robotic pet. This whole process, from image capture and pre-processing to gesture recognition and motor control, is controlled by the Raspberry Pi. In general, the overall system can be said to portray an integration of highly advanced computer vision algorithms with down-to-earth hardware components. This hand gesture recognition system developed for a self-assisted robotic pet; advanced computer vision algorithms, combined with robust hardware, implement interactivity in realizing the concept of an easy-to-use robotic companion. The design focuses on portability and real-time processing, and the open architecture is flexible and can be adapted to improve human–robot interaction.
BRIEF DESCRIPTION OF THE DRAWINGS
The illustrated embodiments of the subject matter will be understood by reference to the drawings, wherein like parts are designated by like numerals throughout. The following description is intended only by way of example, and simply illustrates certain selected embodiments of devices, systems, and methods that are consistent with the subject matter as claimed herein, wherein:
Figure 1: General Architecture of the System
Figure 2: Components architecture of the system
Figure 3: Algorithmic structure of the system.
The figures depict embodiments of the present subject matter for the purposes of illustration only. A person skilled in the art will easily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the disclosure described herein.
DETAILED DESCRIPTION OF THE INVENTION
The detailed description of various exemplary embodiments of the disclosure is described herein with reference to the accompanying drawings. It should be noted that the embodiments are described herein in such details as to clearly communicate the disclosure. However, the amount of details provided herein is not intended to limit the anticipated variations of embodiments; on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the scope of the present disclosure as defined by the appended claims.
It is also to be understood that various arrangements may be devised that, although not explicitly described or shown herein, embody the principles of the present disclosure. Moreover, all statements herein reciting principles, aspects, and embodiments of the present disclosure, as well as specific examples, are intended to encompass equivalents thereof.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms “a",” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes” and/or “including,” when used herein, specify the presence of stated features, integers, steps, operations, elements and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or groups thereof.
It should also be noted that in some alternative implementations, the functions/acts noted may occur out of the order noted in the figures. For example, two figures shown in succession may, in fact, be executed concurrently or may sometimes be executed in the reverse order, depending upon the functionality/acts involved.
In addition, the descriptions of "first", "second", “third”, and the like in the present invention are used for the purpose of description only, and are not to be construed as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" and "second" may include at least one of the features, either explicitly or implicitly.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which example embodiments belong. It will be further understood that terms, e.g., those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The hand gesture recognition system is an advanced integration of hardware and software, designed to facilitate natural and intuitive interaction between humans and robotic pets, in this case, a self-assisted robotic pet. The major components include a camera, Raspberry Pi, power bank, 16-channel servo module, 12 servo motors, voltage Buck converter, Li-Po battery, and sun board sheets. These components are intricately connected and synchronized to ensure the system operates efficiently and effectively.
The camera, placed in front of the user, captures pictures of the user's hand gestures continuously to initiate the system operations. This captures images and processes them on a Raspberry Pi. The OpenCV library is used to pre-process the photos taken. The image pre-processing step will undergo a series of procedures that set the image size at some standardized dimension and normalize the pixel values. Image resizing makes all the images even; it is crucial for proper analysis. Reshaping does this by adjusting the pixel value of an image to a standard scale, improving the exactness of a machine learning analysis.
The pre-processed images are passed through machine learning-implemented models in TensorFlow and YOLOv5. TensorFlow has good support for training and deploying the models on a machine learning tool, but YOLOv5 has been optimized for speed, hence the real-time detection of objects. Because YOLOv5 is fast and accurate in the detection of objects, its efficiency is beneficial in an application with real-time demand.
Hand gesture recognition would then refer to detecting specific patterns in these particular images, patterns that correspond to already predetermined commands. The pre-trained YOLOv5 model detects such patterns while it is exposed to a dataset of different hand gestures. These large numbers of labelled images fed into the system allow the system to learn the features and patterns associated with each type of hand gesture. After detecting the gesture, the Raspberry Pi maps it to a corresponding command. The command is sent to the 16-channel servo module. The servo module translates the command into actions. This in return, controls the 12 servo motors. Each motor executes an action that moves the legs, head, or tail of the robotic pet.
The servo motors literally do the various movements that the robotic pet does—when the pet moves forward, turns around, or does a trick. The voltage Buck converter supplies the servo motors with a constant and smooth voltage so that the motors will be exact and provide a very fluid motion. If there were unsteady voltage, it could make erratic movements, but it wouldn't be noticeable if just by a small amount. The Li-Po battery is used for providing the necessary energy to power the entire system for continuous operation. A high energy density, long life-type Li-Po battery is used with a long service life without frequent recharging and thus finds application with robotic pets for use over extended periods. The outer sun-board sheet body of the robotic pet makes it lightweight yet strong enough to perform and sustain various interactions with easy movements.
More technically speaking, the Raspberry Pi is like the central processing unit that effectively coordinates the whole operation by managing data flow from the camera to the servo motors. So when the image of a hand gesture is taken, raw visual data is relayed to the Raspberry Pi. The raw data relayed is then pre-processed by the Raspberry Pi using the OpenCV library, which includes the resizing of the image into a standard dimension and normalizing pixel values. It is a crucial prepossessing step in which the images are brought to the most appropriate form for further analysis so that gesture recognition can become accurate and reliable.
After preprocessing, the prepared image data undergoes implemented machine learning models via TensorFlow and YOLOv5. TensorFlow gives quite a robust framework for building and deploying machine learning models that can interpret complex visual patterns.
YOLOv5 is known to be fast and accurate in object detection in real-time; in this regard, we use YOLOv5 to detect and recognize hand gestures. The model has been trained on large datasets in this task to capture different variations of hand movement patterns to quickly identify the specific pattern of pictures corresponding to predefined commands.
This recognition process involves sophisticated computations and real-time processing—tasks that are easily handled by the computing capabilities of the Raspberry Pi. Once a hand gesture has been identified, the system translates that visual input into an actionable command. The 16-channel servo module then sends out this command to control the servo motors that drive the physical corresponding movements of the robotic pet. This whole process, from image capture and pre-processing to gesture recognition and motor control, is controlled by the Raspberry Pi. In general, the overall system can be said to portray an integration of highly advanced computer vision algorithms with down-to-earth hardware components. This hand gesture recognition system developed for a self-assisted robotic pet; advanced computer vision algorithms, combined with robust hardware, implement interactivity in realizing the concept of an easy-to-use robotic companion. The design focuses on portability and real-time processing, and the open architecture is flexible and can be adapted to improve human–robot interaction.
A Pet robot system to mimic Human Gestures comprises a plurality Robotic Pet (1.1, .1.2, 1.N) Hand Gesture (2), Cloud Server (3), Neural Stick (3), Raspberry Pi 3v+ (4), Lidar Sensor (5), Camera (1280x720p) (6), 16 Channel Servo Module (7), 1 to 12 Servo Motors (8), Keyboard (9), Mouse (10), 12v 3amp Lithium Polymer (Battery) (11), Charger (12), AC Outlet (13), Charging Current (14), wherein the camera (6) placed in front of the user, captures pictures of the user's hand gestures continuously to initiate the system operations; and the microprocessor process the capture images.
In another embodiment the microprocessor processes the capture images.
In another embodiment the power bank makes the system very portable and independent of fixed power sources and the Li-Po battery is used for providing the necessary energy to power the entire system for continuous operation.
In another embodiment the servo module translates the command into actions.
In another embodiment the servo motor executes an action that moves the legs, head, or tail of the robotic pet.
In another embodiment the voltage Buck converter supplies the servo motors with a constant and smooth voltage so that the motors will be exact and provide a very fluid motion.
In another embodiment the outer sun-board sheet body of the robotic pet makes it lightweight yet strong enough to perform and sustain various interactions with easy movements.
ADVANTAGES OF THE INVENTION
Intuitive Interaction: The system allows for the very natural and intuitive communication of the human to the robotic pet through hand gestures.
Cost-effective: it uses off-the-shelf components like Raspberry Pi and servo motors, making the system very cost-effective.
Efficient Power: The Li-Po battery facilitates a long-lasting power supply, which reduces the need for regular recharging.
User friendliness: systems are easy to set up and use, making users with little technical knowledge have access to them."
, Claims:1. A Pet robot system to mimic Human Gestures comprises a plurality Robotic Pet (1.1, .1.2, 1.N) Hand Gesture (2), Cloud Server (3), Neural Stick (3), Raspberry Pi 3v+ (4), Lidar Sensor (5), Camera (1280x720p) (6), 16 Channel Servo Module (7), 1 to 12 Servo Motors (8), Keyboard (9), Mouse (10), 12v 3amp Lithium Polymer (Battery) (11), Charger (12), AC Outlet (13), Charging Current (14), wherein the camera (6) placed in front of the user, captures pictures of the user's hand gestures continuously to initiate the system operations; and the microprocessor process the capture images.
2. The system as claimed in claim 1, wherein the power bank makes the system very portable and independent of fixed power sources and the Li-Po battery is used for providing the necessary energy to power the entire system for continuous operation
3. The system as claimed in claim 1, wherein the servo module translates the command into actions.
4. The system as claimed in claim 1, wherein the servo motor executes an action that moves the legs, head, or tail of the robotic pet.
5. The system as claimed in claim 1, wherein the voltage Buck converter supplies the servo motors with a constant and smooth voltage so that the motors will be exact and provide a very fluid motion.
6. The system as claimed in claim 1, wherein the outer sun-board sheet body of the robotic pet makes it lightweight yet strong enough to perform and sustain various interactions with easy movements.
7. The system as claimed in claim 1, wherein the processor uses a machine learning model to detect hand gestures.
8. The system as claimed in claim 1, wherein the machine learning model is trained on a dataset of labeled hand gestures.
9. The system as claimed in claim 1, wherein the processor uses a convolutional neural network (CNN) to detect hand gestures.
10. The system as claimed in claim 1, wherein the servo module is configured to control the movements of the robotic pet's head, body, and limbs; and the robotic pet is a self-assisted robotic pet.