Description:.

METHOD FOR REAL-TIME EMOTION DETECTION AND ANALYSIS IN HUMAN-COMPUTER INTERACTION

Field of the Invention

The Generally, the present disclosure relates to human-computer interaction. Particularly, the present disclosure relates to real-time detection of human emotions through facial expression analysis.
Background
The background description includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art.
In the domain of human-computer interaction, the significance of real-time feedback mechanisms is increasingly recognized. Techniques for emotion detection via facial expression analysis have emerged as crucial tools for enhancing user experience. Live video feeds are captured and processed to detect human faces. Subsequently, feature extraction techniques are employed to identify key facial features. Emotional states are classified using convolutional neural networks (CNNs), based on these features. Despite advancements, the process faces challenges in accurately detecting and classifying emotional states due to variability in facial expressions and environmental conditions.
Alternatively, emotion detection is also executed by analyzing physiological signals. This approach involves monitoring signals such as heart rate and skin conductance to infer emotional states. While this method offers a direct measure of physiological responses, it is often intrusive and less practical for real-time applications.
Problems associated with current techniques include inaccuracies in emotion classification, especially in dynamic or complex environments. There is also a lack of real-time processing capabilities, which hampers the immediate translation of emotional analysis into actionable feedback.
In light of the above discussion, there exists an urgent need for solutions that overcome the problems associated with conventional techniques for real-time sentiment detection via facial expression analysis.
Summary
The following presents a simplified summary of various aspects of this disclosure in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements nor delineate the scope of such aspects. Its purpose is to present some concepts of this disclosure in a simplified form as a prelude to the more detailed description that is presented later.
The following paragraphs provide additional support for the claims of the subject application.
In an aspect, the present disclosure provides a method for real-time sentiment detection through facial expression analysis. The disclosed method encompasses capturing live video feeds, preprocessing the captured video data to detect human faces, identifying key facial features using feature extraction techniques, and classifying the emotional state of detected faces with a trained convolutional neural network (CNN). Furthermore, the method includes translating the classification results into feedback for enhancing user experience and presenting the classified emotional state to improve user interaction. The method aims to provide a sophisticated approach to real-time emotion detection and analysis, enhancing human-computer interaction by accurately and promptly identifying emotional states from facial expressions.
Moreover, the present disclosure includes a system for real-time sentiment detection comprising image capture devices, a data preprocessing module, a feature extraction module, an emotion classification module with a CNN, a feedback translation module, and a presentation module. The system aims to utilize the trained CNN for emotion classification based on extracted facial features, thereby offering an effective solution for real-time sentiment analysis.
Further, the method improves upon the basic process by including steps for normalizing image data, performing image segmentation, applying grayscale conversion, and resizing images to meet CNN input requirements. The feature extraction technique employs methods such as edge detection, texture analysis, landmark identification, and contour mapping to identify key facial features accurately. The emotion classification considers expressions indicative of emotions like happiness, sadness, surprise, anger, disgust, fear, and neutral. The method also evaluates the confidence level of the detected emotional state, prompting adjustments if necessary and deciding on the adaptation of the detection model based on this confidence level. Additionally, the CNN is trained on a diverse dataset, considering variations in human demographics to ensure broad applicability.
The system and method described herein offer significant improvements in the accuracy and efficiency of real-time sentiment detection, enabling enhanced user experience in human-computer interactions.

Brief Description of the Drawings

The features and advantages of the present disclosure would be more clearly understood from the following description taken in conjunction with the accompanying drawings in which:
FIG. 1 illustrates a method (100) for real-time sentiment detection via facial expression analysis, in accordance with the embodiments of the present disclosure.
FIG. 2 illustrates a block diagram of a system (200) for real-time sentiment detection via facial expression analysis, in accordance with the embodiments of the present disclosure.
FIG. 3 illustrates a flowchart detailing a process for facial recognition and emotional classification based on live video feeds, in accordance with the embodiments of the present disclosure.
FIG. 4 illustrates a tabular representations of a comparative analysis of different classifiers and feature extraction methods used in facial expression analysis, in accordance with the embodiments of the present disclosure.

Detailed Description
In the following detailed description of the invention, reference is made to the accompanying drawings that form a part hereof, and in which is shown, by way of illustration, specific embodiments in which the invention may be practiced. In the drawings, like numerals describe substantially similar components throughout the several views. These embodiments are described in sufficient detail to claim those skilled in the art to practice the invention. Other embodiments may be utilized and structural, logical, and electrical changes may be made without departing from the scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims and equivalents thereof.
The use of the terms “a” and “an” and “the” and “at least one” and similar referents in the context of describing the invention (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The use of the term “at least one” followed by a list of one or more items (for example, “at least one of A and B”) is to be construed to mean one item selected from the listed items (A or B) or any combination of two or more of the listed items (A and B), unless otherwise indicated herein or clearly contradicted by context. The terms “comprising,” “having,” “including,” and “containing” are to be construed as open-ended terms (i.e., meaning “including, but not limited to,”) unless otherwise noted. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.
Pursuant to the "Detailed Description" section herein, whenever an element is explicitly associated with a specific numeral for the first time, such association shall be deemed consistent and applicable throughout the entirety of the "Detailed Description" section, unless otherwise expressly stated or contradicted by the context.
FIG. 1 illustrates a method (100) for real-time sentiment detection via facial expression analysis, in accordance with the embodiments of the present disclosure. The method (100) for real-time sentiment detection via facial expression analysis comprises: Step (102) capturing live video feeds through one or more cameras; the cameras employed for capturing live video feeds are capable of high-definition recording under varied lighting conditions. This feature ensures the quality and clarity of video data, which is critical for accurate face detection and subsequent analysis. Optionally, the cameras may include infrared capabilities to enhance detection in low-light conditions. Working examples include surveillance cameras and webcams integrated into computing devices. In step (104), preprocessing the captured video data to detect human faces; preprocessing involves several stages aimed at preparing the video data for effective face detection. These stages include normalizing the image data, performing image segmentation to isolate facial regions, applying grayscale conversion to reduce computational complexity, and resizing the images to fit the input requirements of the convolutional neural network (CNN). The preprocessing module's ability to refine and optimize video data significantly improves the accuracy of face detection. Optionally, noise reduction algorithms can be applied to enhance image quality further. In step (106) identifying key facial features from the preprocessed data using feature extraction techniques; the feature extraction techniques employed may include edge detection, texture analysis, landmark identification, and contour mapping. These techniques allow for the precise identification of facial features critical for emotion classification, such as the eyes, eyebrows, mouth, and nose. The feature extraction module's effectiveness lies in its ability to discern subtle facial expressions accurately. Optionally, deep learning algorithms may be utilized for more complex feature identification. In step (108), classifying the emotional state of the detected faces based on the identified facial features using a trained convolutional neural network (CNN); the CNN is trained on a diverse dataset comprising facial images representing various human demographics, including age, gender, ethnicity, and facial hair. This training enables the emotion classification module to accurately classify emotional states by analyzing facial features for expressions indicative of happiness, sadness, surprise, anger, disgust, fear, and neutral. The module's capability to recognize a wide range of emotions enhances the system's applicability across different user groups. Optionally, the system may evaluate the confidence level of the detected emotional state and, if below a predetermined threshold, prompt the user to adjust their position or lighting. In step (110), translating the classification results into feedback for user experience enhancement; the step (110) involves converting the emotional state classification results into actionable feedback that can be used to enhance user experience. For instance, if a user's emotional state is detected as frustrated, the system could suggest content or changes aimed at alleviating this state. Presenting the classified emotional state to enhance user interaction experience. In step (112), the method involves presenting the classified emotional state in a manner that enhances the user interaction experience. This could be through visual indicators, messages, or adjustments in the user interface to reflect the detected emotional state.
In an embodiment, the method for real-time sentiment detection via facial expression analysis includes an enhanced preprocessing step for the captured video data. This step involves normalizing the image data to ensure consistency across different lighting conditions and camera settings, which is pivotal for accurate facial detection. Image segmentation is performed to isolate facial regions from the background, enabling focused analysis on the features critical for emotion detection. Grayscale conversion is applied to the segmented images, significantly reducing the computational complexity without compromising the detection accuracy. Finally, the images are resized to meet the input requirements of the convolutional neural network (CNN). This resizing is essential for maintaining the efficiency of the system while ensuring the data is in the correct format for the CNN. Such enhancements in the preprocessing stage are vital for improving the overall accuracy and performance of the system in real-time sentiment detection.
In another embodiment, the method employs a sophisticated feature extraction technique comprising edge detection, texture analysis, landmark identification, and contour mapping. These techniques collectively enable the precise identification of key facial features that are indicative of emotional states. Edge detection helps in outlining the shapes of facial components, while texture analysis provides details about the skin surface and expressions. Landmark identification focuses on specific points like the corners of the mouth and eyes, crucial for emotion recognition. Contour mapping further aids in defining facial expressions by highlighting the contours of the face. Together, these techniques form a robust feature extraction framework that significantly enhances the system's ability to accurately classify emotional states from facial expressions.
In a further embodiment, the method determines the classification of the emotional state by analyzing facial features for expressions indicative of emotions such as happiness, sadness, surprise, anger, disgust, fear, and neutral. This classification is crucial for understanding the user's emotional feedback in real-time, thereby enabling the system to provide appropriate responses or adjustments to enhance the user interaction experience. The ability to recognize a broad spectrum of emotions ensures that the system is versatile and can cater to diverse user responses, making it more effective in various human-computer interaction scenarios.
In an embodiment, the method includes a step for evaluating the confidence level of the detected emotional state. If the confidence level is found to be below a predetermined threshold, the user is prompted to adjust their position or lighting conditions. This step ensures that the system maintains a high accuracy rate in emotion detection by mitigating factors that could potentially lower the confidence of the detection. Such adaptability in the method enhances user experience by ensuring that the feedback provided is based on accurate and reliable emotion detection.
In another embodiment, the method incorporates a step for deciding whether adaptation of the detection model is needed based on the confidence level of the detected emotional state. If adaptation is deemed necessary, adjustments are made to improve the model's accuracy. This feature ensures that the system remains effective over time, even as it encounters varied expressions and conditions across different users. The adaptability of the detection model is crucial for maintaining the system's reliability and effectiveness in real-time sentiment analysis.
In yet another embodiment, the convolutional neural network (CNN) employed in the method is trained on a dataset comprising facial images representing a diverse range of human demographics, including variations in age, gender, ethnicity, and facial hair. This comprehensive training ensures that the system is capable of accurately recognizing and classifying emotional states across a broad spectrum of the population. The inclusion of such diversity in the training dataset is essential for the system's applicability and effectiveness in real-world scenarios, where it encounters users from varied demographic backgrounds.
The term "system for real-time sentiment detection" as used throughout the present disclosure relates to an integrated assembly of hardware and software components designed to detect, analyze, and classify human emotions in real-time through facial expression analysis. This system captures live video feeds using one or more image capture devices, preprocesses the captured video data to detect human faces, and employs feature extraction techniques to identify key facial features indicative of emotional states. Utilizing a trained convolutional neural network (CNN), the system classifies these facial features into predefined emotional states. It then translates the classification results into actionable feedback for enhancing user experience and displays the classified emotional state to improve user interaction experience. This system is pivotal in applications ranging from enhancing user interface design to providing emotional insights in various fields such as marketing, entertainment, and mental health monitoring, offering a sophisticated approach to understanding and responding to user emotions in real-time.
The term "image capture devices (202)" as used throughout the present disclosure relates to apparatuses capable of recording live video feeds. These devices are instrumental in capturing real-time video data, which serves as the foundational input for the system's sentiment detection process. The image capture devices may include digital cameras, webcams, or integrated camera systems in mobile and computing devices, equipped with capabilities to record high-quality video under various lighting conditions.
The term "data preprocessing module (204)" as used throughout the present disclosure relates to a software or hardware component designed to prepare captured video data for further analysis. This module processes the incoming video feeds to detect human faces by applying a series of operations such as normalization, segmentation, grayscale conversion, and resizing. The preprocessing steps are critical for enhancing the accuracy of subsequent facial feature detection and emotion classification.
The term "feature extraction module (206)" as used throughout the present disclosure relates to a system component responsible for identifying key facial features from the preprocessed video data. Utilizing advanced feature extraction techniques such as edge detection, texture analysis, landmark identification, and contour mapping, this module accurately identifies features essential for determining the user's emotional state.
The term "emotion classification module (208)" as used throughout the present disclosure relates to a component comprising a convolutional neural network (CNN) trained to classify emotional states based on extracted facial features. The CNN employs deep learning algorithms to analyze identified facial features and classify them into predefined emotional states such as happiness, sadness, surprise, anger, disgust, fear, and neutral. This module is pivotal for the accurate and real-time detection of the user's sentiment.
The term "feedback translation module (210)" as used throughout the present disclosure relates to a system component that converts the classification results obtained from the emotion classification module into actionable user feedback. This module interprets the classified emotional states and generates corresponding feedback or suggestions to enhance the user experience, making the system interactive and responsive to the user's emotional cues.
The term "presentation module (212)" as used throughout the present disclosure relates to a component responsible for displaying the classified emotional state in a manner that enhances user interaction experience. This module presents the emotional analysis results through user interfaces, such as graphical displays or auditory signals, allowing users to receive immediate and intuitive feedback on their emotional state.
The system (200) for real-time sentiment detection via facial expression analysis integrates these components to provide a sophisticated solution for analyzing and responding to user emotions in real-time. The system's capability to accurately detect, classify, and respond to human emotions offers significant improvements in human-computer interaction, making it invaluable in applications where understanding and adapting to user sentiment is crucial.
FIG. 2 illustrates a block diagram of a system (200) for real-time sentiment detection via facial expression analysis, in accordance with the embodiments of the present disclosure. Said system (200) includes various interconnected modules essential for the detection and classification of human emotions. An image capture devices module (202) is responsible for recording live video feeds, serving as the initial point of data collection. A data preprocessing module (204) is provided to prepare the captured video data for subsequent analysis. Said data preprocessing module (204) is tasked with detecting human faces from the video feeds, ensuring that only relevant facial data is forwarded for further processing. Adjacent to the data preprocessing module (204) is a feature extraction module (206). Said feature extraction module (206) employs advanced techniques to identify key facial features from the preprocessed data, which are indicative of the user's emotional state. Following feature extraction, an emotion classification module (208) is deployed, comprising a trained convolutional neural network (CNN) for classifying the emotional states based on the identified facial features. This classification is fundamental for interpreting the emotional sentiment conveyed through facial expressions. The system (200) further includes a feedback translation module (210). Said feedback translation module (210) converts the emotion classification results into meaningful feedback, aimed at enhancing the user experience. Additionally, a presentation module (212) is incorporated into the system (200) for displaying the classified emotional state. Such presentation module (212) ensures that the detected sentiment is communicated effectively to enhance user interaction experience. The systematic arrangement of these modules facilitates a comprehensive solution for real-time sentiment analysis in various applications.
In an embodiment, the system for real-time sentiment detection via facial expression analysis incorporates an emotion classification module (208) that utilizes a convolutional neural network (CNN) trained on a dataset containing labeled facial expressions corresponding to a plurality of emotional states. This dataset comprises a comprehensive collection of facial images, each labeled with an emotional state that the facial expression represents, such as happiness, sadness, surprise, anger, disgust, fear, and neutral. The diversity and breadth of the dataset are crucial for training the CNN to accurately recognize and classify a wide range of human emotions based on facial expressions. The training process involves feeding the CNN with these labeled images, allowing it to learn the intricate patterns and features associated with different emotional states. As a result, the emotion classification module is equipped with advanced capabilities to analyze extracted facial features from the system's feature extraction module (206) and classify them into the corresponding emotional states with high accuracy. This ability enhances the system's application in various scenarios, from interactive computing environments to psychological analysis and social media analytics, where understanding and responding to user emotions in real-time is essential. The use of a CNN trained on a diverse and comprehensive dataset ensures that the system is robust against variations in facial expressions, lighting conditions, and demographic differences among users, thereby improving the reliability and effectiveness of real-time sentiment detection.
FIG. 3 illustrates a flowchart detailing a process for facial recognition and emotional classification based on live video feeds, in accordance with the embodiments of the present disclosure. The process begins with capturing live video feed or receiving pre-recorded video. The video is preprocessed, which involves cleaning and formatting the video feed for further analysis. The next step involves identifying key facial features, which involves detecting specific points on the face that are unique to individuals. Once the facial features have been identified, the process classifies emotional state by analyzing the expressions corresponding to various emotions. The results from this classification are then Translated into feedback, which suggests that the system to provide an interactive response or output based on the emotional state detected. The flowchart also represents nodes to check quality and accuracy in the recognition process. When the confidence level is deemed adequate, the system evaluates whether an adaptation decision is needed.
The proposed sentiment detection system overcome limitations of traditional emotion analysis methods by integrating computer vision and deep learning techniques tailored for real-time facial expression analysis. This system improves accurately identifying a wide range of emotions, significantly reducing false positives and ensuring reliable emotional analysis. It includes real-time processing capabilities essential for interactive media and customer feedback systems, and features continuous adaptation to remain effective against evolving human expressions. Additionally, its modular design allows for scalable implementations across various platforms, and its automated processes reduce costs, making sophisticated emotion recognition technology more accessible. The system's deployment involves steps such as data collection, feature extraction, model training, performance testing, and ongoing updates to enhance accuracy and adaptability.
FIG. 4 illustrates a tabular representations of a comparative analysis of different classifiers and feature extraction methods used in facial expression analysis, in accordance with the embodiments of the present disclosure. The Convolutional Neural Network (CNN) has a higher false positive rate at 12.50% but maintains a robust true negative rate of 97.25%. In contrast, the Long Short-Term Memory (LSTM) network exhibits the lowest false positive rate of 10.00% and outperforms other models with a true positive rate of 92.00%. The Support Vector Machine (SVM) strikes a balance with the lowest false negative rate at 0.85% and a respectable true positive rate of 91.50%. When examining precision, recall, and F1 scores, LSTM slightly outperforms the other classifiers, signifying a balanced precision and recall. Further analysis on feature extraction methods reveals that CNN-based features enhance precision most significantly by 92.00%, indicating a superior performance over Histogram of Oriented Gradients and Haar Cascades, which show 88.00% and 85.50% improvements, respectively. The data overall suggests that LSTM and CNN-based feature extraction methods yield the most effective results for the facial expression analysis task at hand.
Example embodiments herein have been described above with reference to block diagrams and flowchart illustrations of methods and apparatuses. It will be understood that each block of the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, respectively, can be implemented by various means including hardware, software, firmware, and a combination thereof. For example, in one embodiment, each block of the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations can be implemented by computer program instructions. These computer program instructions may be loaded onto a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions which execute on the computer or other programmable data processing apparatus create means for implementing the functions specified in the flowchart block or blocks.
Throughout the present disclosure, the term ‘processing means’ or ‘microprocessor’ or ‘processor’ or ‘processors’ includes, but is not limited to, a general purpose processor (such as, for example, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor implementing other types of instruction sets, or a microprocessor implementing a combination of types of instruction sets) or a specialized processor (such as, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).
The term “non-transitory storage device” or “storage” or “memory,” as used herein relates to a random access memory, read only memory and variants thereof, in which a computer can store data or software for any duration.
Operations in accordance with a variety of aspects of the disclosure is described above would not have to be performed in the precise order described. Rather, various steps can be handled in reverse order or simultaneously or not at all.
While several implementations have been described and illustrated herein, a variety of other means and/or structures for performing the function and/or obtaining the results and/or one or more of the advantages described herein may be utilized, and each of such variations and/or modifications is deemed to be within the scope of the implementations described herein. More generally, all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the teachings is/are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific implementations described herein. It is, therefore, to be understood that the foregoing implementations are presented by way of example only and that, within the scope of the appended claims and equivalents thereto, implementations may be practiced otherwise than as specifically described and claimed. Implementations of the present disclosure are directed to each individual feature, system, article, material, kit, and/or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and/or methods, if such features, systems, articles, materials, kits, and/or methods are not mutually inconsistent, is included within the scope of the present disclosure.

Claims

I/We claim:

A method (100) for real-time sentiment detection via facial expression analysis comprising the steps of:
(a) capturing live video feeds through one or more cameras;
(b) preprocessing the captured video data to detect human faces;
(c) identifying key facial features from the preprocessed data using feature extraction techniques;
(d) classifying the emotional state of the detected faces based on the identified facial features using a trained convolutional neural network (CNN);
(e) translating the classification results into feedback for user experience enhancement; and
(f) presenting the classified emotional state to enhance user interaction experience.
The method (100) of claim 1, wherein the preprocessing of captured video data further comprises:
(a) normalizing the image data;
(b) performing image segmentation to isolate facial regions;
(c) applying grayscale conversion to reduce computational complexity; and
(d) resizing the images to fit the input requirements of the CNN.
The method (100) of claim 1, wherein the feature extraction technique is selected from the group consisting of:
(a) edge detection;
(b) texture analysis;
(c) landmark identification; and
(d) contour mapping.
The method (100) of claim 1, wherein the classification of the emotional state is determined by analyzing the facial features for expressions indicative of at least one of the following emotions selected from happiness, sadness, surprise, anger, disgust, fear and neutral.
The method (100) of claim 1, further comprising the step of evaluating the confidence level of the detected emotional state, and if the confidence level is below a predetermined threshold, prompting the user to adjust their position or lighting.
The method (100) of claim 5, further comprising the step of deciding whether adaptation of the detection model is needed based on the confidence level, and if so:
The method (100) of claim 1, wherein the CNN is trained on a dataset comprising facial images representing a diverse range of human demographics, including variations in age, gender, ethnicity, and facial hair.
A system (200) for real-time sentiment detection via facial expression analysis comprising:
(a) one or more image capture devices (202) for recording live video feeds;
(b) a data preprocessing module (204) for preparing captured video data for analysis;
(c) a feature extraction module (206) for identifying key facial features from the preprocessed data;
(d) an emotion classification module (208) comprising a CNN trained to classify emotional states based on extracted facial features;
(e) a feedback translation module (210) for converting classification results into user feedback; and
(f) a presentation module (212) for displaying the classified emotional state.
The system (200) of claim 8, wherein the emotion classification module (208) utilizes a CNN trained on a dataset containing labeled facial expressions corresponding to a plurality of emotional states.
METHOD FOR REAL-TIME EMOTION DETECTION AND ANALYSIS IN HUMAN-COMPUTER INTERACTION

A method is disclosed for real-time sentiment detection via facial expression analysis. The method includes capturing live video feeds through one or more cameras and preprocessing the captured video data to detect human faces. Key facial features are identified from the preprocessed data using feature extraction techniques. The emotional state of the detected faces is classified based on the identified facial features using a trained convolutional neural network (CNN). The classification results are translated into feedback for user experience enhancement. The classified emotional state is presented to enhance user interaction experience. This method provides a sophisticated approach to understanding and responding to user emotions in real-time, offering a significant improvement in human-computer interaction.
Drawings
/
FIG. 1
/
FIG. 2
/
FIG. 3
/
FIG. 4
, Claims:I/We claim:

A method (100) for real-time sentiment detection via facial expression analysis comprising the steps of:
(a) capturing live video feeds through one or more cameras;
(b) preprocessing the captured video data to detect human faces;
(c) identifying key facial features from the preprocessed data using feature extraction techniques;
(d) classifying the emotional state of the detected faces based on the identified facial features using a trained convolutional neural network (CNN);
(e) translating the classification results into feedback for user experience enhancement; and
(f) presenting the classified emotional state to enhance user interaction experience.
The method (100) of claim 1, wherein the preprocessing of captured video data further comprises:
(a) normalizing the image data;
(b) performing image segmentation to isolate facial regions;
(c) applying grayscale conversion to reduce computational complexity; and
(d) resizing the images to fit the input requirements of the CNN.
The method (100) of claim 1, wherein the feature extraction technique is selected from the group consisting of:
(a) edge detection;
(b) texture analysis;
(c) landmark identification; and
(d) contour mapping.
The method (100) of claim 1, wherein the classification of the emotional state is determined by analyzing the facial features for expressions indicative of at least one of the following emotions selected from happiness, sadness, surprise, anger, disgust, fear and neutral.
The method (100) of claim 1, further comprising the step of evaluating the confidence level of the detected emotional state, and if the confidence level is below a predetermined threshold, prompting the user to adjust their position or lighting.
The method (100) of claim 5, further comprising the step of deciding whether adaptation of the detection model is needed based on the confidence level, and if so:
The method (100) of claim 1, wherein the CNN is trained on a dataset comprising facial images representing a diverse range of human demographics, including variations in age, gender, ethnicity, and facial hair.
A system (200) for real-time sentiment detection via facial expression analysis comprising:
(a) one or more image capture devices (202) for recording live video feeds;
(b) a data preprocessing module (204) for preparing captured video data for analysis;
(c) a feature extraction module (206) for identifying key facial features from the preprocessed data;
(d) an emotion classification module (208) comprising a CNN trained to classify emotional states based on extracted facial features;
(e) a feedback translation module (210) for converting classification results into user feedback; and
(f) a presentation module (212) for displaying the classified emotional state.
The system (200) of claim 8, wherein the emotion classification module (208) utilizes a CNN trained on a dataset containing labeled facial expressions corresponding to a plurality of emotional states.
METHOD FOR REAL-TIME EMOTION DETECTION AND ANALYSIS IN HUMAN-COMPUTER INTERACTION