Claims:We Claim:
1. An ER-GAN framework based on GAN architecture for improving performance in human emotion recognition and classification.
2. A generator component that is based on CNN architecture with deep learning to have provision for making new samples towards data augmentation.
3. A discriminator component made up of CNN architecture with deep learning to validate samples given by generator and give feedback to arrive at new training samples.
4. A classifier component based on neural networks for efficient classification of human emotions based on the training given through machine learning.
5. An algorithm known as Deep Learning based GAN for Human Emotion Recognition (DLGAN-HER) which is crucial to realize the current invention ER-GAN.
6. An integration module that brings about seamless integration of components such as generator, discriminator and NN classifier.
7. A solution to the problem of human emotion recognition from surveillance videos or any such source.
, Description:FIELD OF INVENTION
Human emotion recognition is one of the computer vision applications that assumes significance in the contemporary era. There are plenty of human emotions such as happy, sad, angry, surprise and neutral. The current invention is meant for building an Artificial Intelligence (AI) enabled framework for human emotion recognition. Deep learning models such as Convolutional Neural Network (CNN) are used to implement the proposed framework based on the Generative Adversarial Network (GAN) architecture. The proposed GAN is known as ER-GAN which has two important components known as Generator (G) and Discriminator (D). Both the components work as players in a non-cooperative game. The idea is to enhance and augment training quality so as to improve the performance in human emotion recognition. An algorithm named Deep Learning based GAN for Human Emotion Recognition (DLGAN-HER) is proposed and implemented. The algorithm helps in realization of ER-GAN with higher level of accuracy.

It has both generator and discriminator components built using CNN architectures. Generator is meant for making new training samples and discriminator will validate them and give its feedback. Thus new samples are generated that satisfy the discriminator. There is non-cooperative game between the two components with adversarial settings. Hence it is known as Generative Adversarial Network (GAN) based architecture. The generator takes noise inputs and possible real class labels in order to generate new samples. Discriminator takes the newly generated samples and training samples to discriminate new samples and give feedback to generator. The multi-class classifier is another important component that is made up of Neural Network (NN) classifier. This component is to recognise human emotions into different categories.

BACKGROUND OF THE INVENTION
Human emotion recognition is one of the computer vision applications that assumes significance in the contemporary era. There are plenty of human emotions such as happy, sad, angry, surprise and neutral. In many computer vision applications, it is essential to deal with pre-recorded videos or live surveillance videos. In such cases, recognition of human emotions has its utility in real time applications. It is of very significance to organizations dealing with computer vision applications, investigation agencies, patient monitoring systems, researchers and academia. However, human emotion recognition is a tedious task as it is so complex phenomenon. Often there is the problem of lack of adequate training samples in case of supervised learning approaches with machine learning.
The problem aforementioned is overcome in this invention as it is based on the GAN architecture that supports data augmentation. The proposed GAN is known as ER-GAN which has two important components known as Generator (G) and Discriminator (D). Both the components work as players in a non-cooperative game. The idea is to enhance and augment training quality so as to improve the performance in human emotion recognition. An algorithm named Deep Learning based GAN for Human Emotion Recognition (DLGAN-HER) is proposed and implemented. The algorithm helps in realization of ER-GAN with higher level of accuracy. The baseline GAN has provision for generating new samples and validating them. There is an iterative process between generator and discriminator with feedback and improvement of samples. This will help in data augmentation and improve quality of training and thereby leveraging prediction performance. The generator provides new samples while discriminator is strict in validating and giving feedback. This kind of adversarial game brings about good quality in samples thus leading to more training samples. There is NN classifier that actually takes testing samples and classify them into various categories of human emotions.

OBJECTS OF THE INVENTION
1] Therefore, the object of the present invention is to provide an ER-GAN framework based on GAN architecture for improving performance in human emotion recognition and classification.

2] Another object of the present invention to have a generator component that is based on CNN architecture with deep learning to have provision for making new samples towards data augmentation.

3] Another object of the present invention is to have a discriminator component made up of CNN architecture with deep learning to validate samples given by generator and give feedback to arrive at new training samples.

4] Another important object of the present invention is to have a classifier component based on neural networks for efficient classification of human emotions based on the training given through machine learning.

5] Another object of the present invention is to have an algorithm known as Deep Learning based GAN for Human Emotion Recognition (DLGAN-HER) which is crucial to realize the current invention ER-GAN.

6] Another object of the present invention is to have an integration module that brings about seamless integration of components such as generator, discriminator and NN classifier.

7] Yet another object of the present invention is to have a solution to the problem of human emotion recognition from surveillance videos or any such source.

STATEMENT OF THE INVENTION
The present invention known as “An Artificial Intelligence (AI) Enabled Framework for Human Emotion Recognition” is meant for building an Artificial Intelligence (AI) enabled framework for human emotion recognition. Deep learning models such as Convolutional Neural Network (CNN) are used to implement the proposed framework based on the Generative Adversarial Network (GAN) architecture. The proposed GAN is known as ER-GAN which has two important components known as Generator (G) and Discriminator (D). Both the components work as players in a non-cooperative game. The idea is to enhance and augment training quality so as to improve the performance in human emotion recognition. An algorithm named Deep Learning based GAN for Human Emotion Recognition (DLGAN-HER) is proposed and implemented. The algorithm helps in realization of ER-GAN with higher level of accuracy. Here are the details provided for the drawings given in the preceding section.

An algorithm known as Deep Learning based GAN for Human Emotion Recognition (DLGAN-HER) is defined to realize the proposed ER-GAN. It takes Noise Input z, labels C, test samples T1, training samples T2 as inputs and produces human emotion classification results. Step 2 through Step 6, there is an iterative process where generator produces new samples and discriminator validates them and give necessary feedback. Then CNN classifier is used in order to have final classification of human emotions. In the process, there is provision for computing error rate, optimization of generator and discriminator.

BRIEF DESCRIPTION OF THE DRAWING
Human emotion recognition is one of the computer vision applications that assumes significance in the contemporary era. There are plenty of human emotions such as happy, sad, angry, surprise and neutral. The current invention is meant for building an Artificial Intelligence (AI) enabled framework for human emotion recognition. Deep learning models such as Convolutional Neural Network (CNN) are used to implement the proposed framework based on the Generative Adversarial Network (GAN) architecture. The proposed GAN is known as ER-GAN which has two important components known as Generator (G) and Discriminator (D). Both the components work as players in a non-cooperative game. The idea is to enhance and augment training quality so as to improve the performance in human emotion recognition. An algorithm named Deep Learning based GAN for Human Emotion Recognition (DLGAN-HER) is proposed and implemented. The algorithm helps in realization of ER-GAN with higher level of accuracy. The current invention is illustrated with the help of many drawings given below.

Figure 1: Illustrates the architecture of the proposed invention known as ER-GAN for human emotion recognition

Figure 2: Illustrates a generic or baseline GAN architecture

Algorithm: Deep Learning based GAN for Human Emotion Recognition (DLGAN-HER) Inputs: Noise Input z, labels C, test samples T1, training samples T2
Output: Human emotion classification results
1. For each input noise z
2. Repeat:
3. newSamplesCreateNewSamples(G, C, z) //using CNN
4. (fake, not fake)Discriminator(newSamples, T2) //using CNN
5. feedbackGetFeedbackFromDiscriminator()
6. Until z becomes true
7. RUseNNClassifier(T1)
8. Computer error rate
9. Generator optimization
10. Discriminator optimization
11. Return R
12. End for

Figure 3: Provides the modus operandi of Deep Learning based GAN for Human Emotion Recognition (DLGAN-HER) algorithm

Figure 4: Illustrates the flow of activities involved in the process of human emotion recognition

Figure 5: Illustrates stakeholders for which the invention is beneficial
DETAILED DESCRIPTION OF DRAWINGS
Human emotion recognition is one of the computer vision applications that assumes significance in the contemporary era. There are plenty of human emotions such as happy, sad, angry, surprise and neutral. The current invention is meant for building an Artificial Intelligence (AI) enabled framework for human emotion recognition. Deep learning models such as Convolutional Neural Network (CNN) are used to implement the proposed framework based on the Generative Adversarial Network (GAN) architecture. The proposed GAN is known as ER-GAN which has two important components known as Generator (G) and Discriminator (D). Both the components work as players in a non-cooperative game. The idea is to enhance and augment training quality so as to improve the performance in human emotion recognition. An algorithm named Deep Learning based GAN for Human Emotion Recognition (DLGAN-HER) is proposed and implemented. The algorithm helps in realization of ER-GAN with higher level of accuracy. Here are the details provided for the drawings given in the preceding section.

Referring to Figure 1, it illustrates the architecture of the proposed invention known as ER-GAN for human emotion recognition. It has both generator and discriminator components built using CNN architectures. Generator is meant for making new training samples and discriminator will validate them and give its feedback. Thus new samples are generated that satisfy the discriminator. There is non-cooperative game between the two components with adversarial settings. Hence it is known as Generative Adversarial Network (GAN) based architecture. The generator takes noise inputs and possible real class labels in order to generate new samples. Discriminator takes the newly generated samples and training samples to discriminate new samples and give feedback to generator. The multi-class classifier is another important component that is made up of Neural Network (NN) classifier. This component is to recognise human emotions into different categories.

Referring to Figure 2, it illustrates the typical GAN architecture that has components like generator and discriminator. This is known as baseline GAN architecture. The current invention known as ER-GAN is based on this GAN model. The baseline GAN has provision for generating new samples and validating them. There is an iterative process between generator and discriminator with feedback and improvement of samples. This will help in data augmentation and improve quality of training and thereby leveraging prediction performance.

Referring to Figure 3, it illustrates the underlying algorithm of the current invention. The algorithm is named as Deep Learning based GAN for Human Emotion Recognition (DLGAN-HER). It takes Noise Input z, labels C, test samples T1, training samples T2 as inputs and produces human emotion classification results. Step 2 through Step 6, there is an iterative process where generator produces new samples and discriminator validates them and give necessary feedback. Then CNN classifier is used in order to have final classification of human emotions. In the process, there is provision for computing error rate, optimization of generator and discriminator.

Referring to Figure 4, it illustrates the flow of activities involved in the process of human emotion recognition. After taking inputs such as Noise Input z, labels C, test samples T1, training samples T2, there is continuous communication between generator and discriminator. The generator provides new samples while discriminator is strict in validating and giving feedback. This kind of adversarial game brings about good quality in samples thus leading to more training samples. There is NN classifier that actually takes testing samples and classify them into various categories of human emotions.

Referring to Figure 5, it illustrates that the current invention has benefits to many stakeholders. The stakeholders include organizations dealing with computer vision applications, investigation agencies, patient monitoring systems, researchers and academia.