Description:Field of the Invention
The present invention pertains to the field of robotics and artificial intelligence, specifically focusing on developing a chatbot robot capable of object and face recognition and extracting relevant information.
Objective of this Invention
The primary objective of this invention is to develop a conversational robot with advanced speech- based interaction capabilities. This robot will be equipped to engage users in friendly conversations through speech, recognize objects and faces, and provide tailored responses. Additionally, it will serve as an infobot for known faces and gather information about unknown visitors, enhancing human-robot interaction in diverse scenarios.
Background of the Invention
The inspiration for the proposed invention arose from the desire to create an advanced and interactive robotic system capable of engaging in natural and friendly conversations with users through speech. This innovative idea covers a range of aspects, including speech-based interaction and the ability to recognize objects and faces. The idea of creating a robot, that can not only communicate with users but also understand and reply through speech gives a futuristic picture of human-robot interaction. The idea was to develop a complex system that not only offers enlightening responses but also forges a close bond with the user, much like speaking with a human counterpart. The first idea came from seeing the possibility of combining speech synthesis and recognition technologies with advanced object and face recognition capabilities, which would improve the entire user experience and make the robot a flexible assistance in a variety of circumstances.
For instance, CN106446816B relates to face recognition methods and devices. It encloses that the current technology for face recognition involves comparing a bunch of pictures in a base library with the given picture, which is more time-consuming and memory-consuming. The proposed system includes the method of extracting the facial features and recognizing the faces based on those features' historical base library. If no match is found then the recognition is done based on the normal base library. The method also includes adding recognized facial features to the historical base library. In this method, after recognizing the
3 | P a g e
faces it uses annotations for screening and judging the recognition results. The face recognition devices include modules for image acquisition, feature extraction, historical base recognition, and normal base recognition.
Similarly, CN106228628B provides a check-in system, method, and device for face recognition. The system includes a camera and an embedded face-recognition processing device. The device extracts the face features through a continuous video stream collected by a camera and detects if the registered user matches the check-in user. This improves check-in efficiency and can be used in high-density check-in sites like conferences. The device includes a CPU, GPU, memory, and input/output interface. The method involves receiving the video stream, extracting the facial features, matching the extracted facial features with the registered user’s facial features, and confirming the check-in. The device includes a video acquisition module, feature extraction module, user matching module, and check-in confirmation module. The system provides a faster and more efficient way to check-in using face recognition technology.
US20200042775A1 provides an artificial intelligence (AI) server and methods for de-identifying a face area of an unspecific person from an image file including a video or a picture. The Ai servers receive the image file from the user’s AI apparatus and use a face recognition model to determine the faces includes in the image file. Then the server compares the faces in the image file with the previously stored face information associated with the user to find if the face is known or unknown. If the face is unknown the AI server de-identifies the unknown face from the image file. This technology aims to prevent the faces of unknown individuals from being exposed when uploaded to a social network service.
US20200026907A1 relates to object detection based on joint feature extraction. This method for object detection involves the extraction of features from an image in two stages. In the first stage, a feature is extracted to characterize the whole image, while in the second stage, another feature is extracted from the candidate object region identified in the image. These features are then used to determine the target object and its confidence. This approach improves the accuracy of object detection by considering features from both the whole scale and the local scale of the image.
The autonomous vehicle operation technique described in US11443148B2 involves object detection and recognition using multiple-stage classification. Receiving the object data, using a first machine-learning model to identify first-stage characteristics and features of objects, a
4 | P a g e
second machine-learned model to identify second-stage characteristics and features of objects, and producing indications of identified items are all included in the technique. cars, including driverless cars, can use this technology to precisely determine the condition of things in the environment.
US11417082B2 proposes an image processing system for object detection in images and videos. The goal of the system is to precisely locate each object in the image and identify it. It addresses the challenges like noise, scale, occlusion, and appearance similarity. The system utilizes deep learning techniques and hierarchal detection to improve object detection performance. It also incorporates a similar object search framework that models the correlations between different object categories. The system can identify and add new object categories without the need for retraining. The system aims to improve the accuracy and robustness of object detection in various computer vision tasks.
Systems and methods for continual updating of response generation by an artificial intelligence chatbot application are covered by US20230085061A1. The application describes the systems and methods for dialog generation in chatbots. Chatbots are increasingly used to provide efficient and timely responses to customer queries. Traditional rule-based and grammar-based approaches have limitations in handling new questions and understanding spoken language. The application suggests a neural network-based approach to overcome these constraints, combining a variational autoencoder (VAE) and a generative adversarial network (GAN) to produce realistic and interesting responses. The neural network is continuously improved through user interaction afterbeing trained using a large dataset of language-based queries. This approach improves the efficiency and effectiveness of chatbot responses.
WO2020042925A1 relates to a human-machine dialogue method, device, electronic device, and computer-readable medium. The process consists of extracting keywords from user input data, selecting candidates’ articles based on the keywords, getting sentences from the candidates’ articles, calculating their scores, and producing return data based on the score results. The apparatus has modules for extracting keywords, identifying articles, scoring, and producing return data. The electronic device and computer-readable medium are also provided.
US20200394366A1 provides a virtual assistant for generating personalized responses within a communication session. It describes a virtual assistant that enhances communication sessions
5 | P a g e
by analyzing the content exchanged in the session, identifying relevant portions, and generating personalized responses. The virtual assistant can monitor user activity and extract user patterns to generate user-interest and content-relevance models. It can also monitor multiple communication sessions to identify patterns within the content. The virtual assistant takes into account contextual features of the communication session and generates responses based on content-substance and content-style features. It can also generate summarized versions of the content and encode relevant features in the responses. The virtual assistant aims to improve the user experience in multitasking situations by providing timely and personalized responses within communication sessions.
The realization of this visionary invention draws extensively from the research insights and advancements presented in the referenced papers. The research underpinning this endeavor delves into multiple domains, including face recognition, object detection, and dialogue generation. We can build a robot that embodies the key functionalities, techniques, and tools described in these papers. Techniques such as extracting facial features for recognition, real-time face matching, and object detection based on joint feature extraction serve as crucial components for the proposed robot's ability to identify and interact with users. Moreover, the application of artificial intelligence (AI) techniques, as detailed in the cited research, opens avenues for enhancing the robot's conversational capabilities. The proposed combination of these research-backed methods empowers the robot to be an adept conversationalist, an efficient infobot for known faces, and a meticulous recorder of unknown visitors' interactions, thereby paving the way for an innovative and versatile human-robot interaction platform.
Summary of the Invention
The robot is designed to engage in friendly conversations with users, while also possessing the ability to recognize specific objects and faces. The system can extract relevant information about certain users, enabling personalized interactions for an enhanced user experience. Additionally, the robot functions as an "infobot" for known faces, providing tailored information and assistance. The innovative aspect lies in its capacity to identify unknown faces, gathering essential information about their visit and the person they are intending to meet. Leveraging modern AI and computer vision technologies, the proposed invention addresses the challenges of efficient object and face recognition and strives to
6 | P a g e
transform human-robot interactions in various real-world applications. The system's adaptability, privacy safeguards, and seamless integration of conversational abilities make it an innovative solution for personalized, intelligent, and user-friendly robot interactions.
Brief Description of Drawings
The invention will be described in detail with reference to the exemplary embodiments shown in the figures wherein:
Figure-1. Flowgorithm representing the workflow of the Infobot.
Figure-2. Flowgorithm representing the workflow of Chatbot used in Infobot.
Figure-3. Flowgorithm representing the workflow of the object recognition module used in Infobot.
Detailed Description of the Invention
The present invention pertains to a revolutionary project aimed at creating an innovative robot that introduces a new era of interactive human-robot interactions. The infobot encompasses the design, development, and integration of a sophisticated chatbot robot with exceptional capabilities, including friendly conversations, object and face recognition, information extraction, personalized assistance, and visitor tracking.
Chat Interface and Communication:
The robot has a user-friendly chat interface that is driven by advanced natural language processing (NLP) techniques. This enables the robot to interact with users in a natural and fluid conversation, replicating human-like interactions. The integration of a Raspberry Pi microcomputer acts as the central processing unit, ensuring seamless conversation flow and facilitating real-time communication.
Object Recognition System:
The inclusion of an object recognition system is a crucial component of the invention. The robot has a high-resolution camera module attached to the Raspberry Pi that uses cutting-edge computer vision methods. This camera module gathers visual information from the environment, enabling the robot to recognize and classify numerous things with remarkable precision.
7 | P a g e
Face Recognition and Information Extraction:
The robot has a sophisticated facial recognition system that uses deep learning models to distinguish between various users and identify them. Facial features are photographed using a Raspberry Pi-compatible camera and processed through neural networks for accurate identification. Once a user is identified, the robot pulls pertinent data from its built-in database to provide a customized and personalized experience.
Infobot Functionality:
An "infobot" version of the robot appears when it recognizes a face. It retrieves previously saved data on the identified user, such as preferences, exchanges from the past, and specific details. By giving the robot this ability, it may offer recommendations, information, and support that are specifically tailored to the needs of each user, which improves engagement and satisfaction.
Unknown Face Identification and Visitor Tracking:
The innovation also addresses the difficulty of following and communicating with guests by recognizing unidentified faces. When a new face is spotted, the robot strikes up a conversation to learn crucial information about the visitor and their visit's objectives. This data is safely kept in the system and contributes to a thorough visitor tracking system.
Hardware Components:
The Raspberry Pi controls data processing, communication, and control for the robot. A microphone array records user speech, while the integrated camera module records visual input. Clear and audible answers are ensured with high-quality speakers. Additionally, the Raspberry Pi's storage capabilities enable the robot's memory storage system, which securely stores user data, recognized faces, and visitor information. , Claims:The scope of the invention is defined by the following claims:
Claims:
1. A chatbot robot with advanced capabilities, comprising:
a) A friendly chat interface for engaging in natural and personalized conversations with users and an object recognition system to identify and categorize various objects based on visual input.
b) A face recognition module capable of recognizing specific users and extracting relevant information about them and a memory storage system to retain information about known users, their preferences, and relevant data.
c) The unknown face identification feature to gather information about visitors and determine the purpose of their visit and the intended person they wish to meet.
2. According to Claim 1, the AI-powered system is installed for natural language processing, enabling seamless and human-like interactions. A robust object and face recognition algorithm integrated into the chatbot robot of claim 1, utilizing cutting-edge AI and computer vision technologies for accurate and efficient recognition.
3. According to Claim 1, the privacy safeguards and data encryption mechanisms is incorporated to protect sensitive user information. The adaptive learning capability integrated into the chatbot enabling continuous improvement of user interactions and recognition accuracy over time.
4. According to Claim 1, the comprehensive user management system allows seamless identification and differentiation between various users, known and unknown, for personalized and informative interactions.