Description:BACKGROUND OF THE INVENTION
Due to social isolation, academic pressure, digital overload and lack of timely psychological intervention now a days there are several mental health issues raising among the students at an alarming rate. If we see there are several traditional methods to assessing mental health issues but as they are manual, reactive and often fails in providing early detection or real-time monitoring. Also, students often feel shy in expressing their emotional struggles openly which leads to underreporting and delayed care. Hence, there is a need of automated, continuous and non-invasive mental health monitoring system which integrates behavioural and psychological data. For tracking vital signs wearable devices can be used, while for analysing the facial expression can offer insights into emotional states, but these systems typically function in isolation. If there is no unified, real-time analysis framework there will be lack of accuracy and practicality for prediction of mental health. Therefore, there is requirement of such a device which can collaborate wearable sensors and edge vision with cloud-based machine learning for enabling early detection and proactive mental health support for students.
Monitoring of mental health among the students has gained a significant attention in last few years due to the increasing rates of depression, stress, anxiety and issues related to the psychological conditions of the students. Although, there are numerous study which have explored the digital mental health and self-report assessments but these methods often lacks of real-time monitoring and also depends upon the inputs given by the users, which limits the effectiveness in management of proactive mental health. The tradisiional clinical methods for detection of the mental health conditions are labour-intensive and there is requirement of face-to-face interaction with mental health professionals and are often inaccessible to the large student populations.
At present there exist several wearable technologies which have been introduced to monitor physiological parameter such as skin temperature, activity levels and heart rate. However, most of the existing solutions are limited to only tracking about the fitness of the human being and they do not specifically target or integrates the prediction of mental health. These wearables often fail to incorporate comprehensive bio signals such as photoplethysmography or galvanic skin response which are highly indicative of emotional states. Moreover, these devices usually function independently without leveraging additional behavioural data that can improve the accuracy of diagnostics.
On the other hand, integration of the vision-based emotion recognition has emerged as a valuable approach for understanding the psychological states by using the facial expressions. While some of the researches has utilised the data of facial analysis in surveillance as well as educational contexts, these systems typically rely upon cloud processing which is leading to privacy concerns, latency and high bandwidth consumption. Most importantly, they lacks of integration with physiological data by reducing their ability for detecting nuanced mental health conditions accurately.
There is lack of systems which uses both the physiological as well as behavioural data analysis using machine learning and edge computing. Today’s literature and commercial solutions don’t explore a multimodal fusion approach involving both wearable sensor data and edge vision-based emotional detection for predicting mental health. Furthermore, there are very rare systems or studies which addresses the needs of privacy-preserving, Wi-Fi enabled infrastructure that can scale across institutions and classrooms while maintaining real-time prediction capability.
This research aims on bridging these gaps by developing an intelligent system which combined both edge vision devices for emotion recognition and sensor-rich device wearable bands for behavioural analysis. Both the devices transmit the processed data to a cloud-based machine learning model. Such a solution addresses the problem of single-source data collection by enhancing the prediction accuracy through multimodal input and promotes early intervention in a scalable, ethical and user-friendly manner. To the best of our knowledge, there exists no work which provides a comprehensive, real-time, classroom-ready framework for monitoring mental health based on this dual-layer data fusion architecture.
CN107463790A A kind of mental health medical system, the present invention is given conveniently using network to patient and doctor, it has been also convenient for some patients and has may be because the problem of the problems such as introverted delays treatment etc., the system includes on-line consulting, Mental health test, online intervention module, it is more convenient for the treatment level of Gao doctor and the treatment of patient, whenever and wherever possible, convenient treatment, data storage includes psychologist's profile module, test and appraisal case module, psychotherapy method module, Psychological Evaluation evaluation criteria module, the information of storage is comprehensive, and therapeutic scheme can be provided, the present invention is for preventing mental disease, psychogenic disorder, realize the self-service intervention in part, specialty is provided and intervenes guidance, and the present invention can remotely carry out medical treatment, solves patient's busy the problems such as seeing a doctor.
RESEARCH GAP: The proposed system is different as it uses real-time physiological and emotional data from wearable sensors and edge AI, unlike the cited system which relies on online consultations and stored psychological data.
US20200107767A1 A system of hardware and software that captures and processes in real time biometric data of patient responses and reactions to tablet-based valenced pictorial stimuli to facilitate diagnosis of mental health conditions of the patient including Autism Spectrum Disorder (ASD), Attention-Deficit Hyperactivity Disorder (ADHD), Traumatic Brain Injury (TBI), and Post-Traumatic Stress Disorder (PTSD) is provided. Via transparent, non-invasive biometrics induced by pictorial stimuli, the system removes potentially threatening testing instrument characteristics that can invalidate authentic assessment. The system will mitigate or remove other threats to assessment validity including subjective judgments as well as bias on the part of the examiners who are completing subjective surveys. The biometric assessment is incorporated within a variety of engaging game formats that appeal to males and females of nearly any age that speak a variety of languages and encompass a wide spectrum of demographics and ethnicities.
RESEARCH GAP: The proposed system is different as it monitors students’ mental health continuously in real-time using wearable sensors and edge-based facial emotion recognition, unlike the cited system which relies on short-term biometric responses to pictorial stimuli for diagnostic assessment.
None of the prior art indicate above either alone or in combination with one another disclose what the present invention has disclosed. This invention relates to Real-Time Student Mental Health Monitoring Using Wearable Sensors and Edge-Based Emotion Recognition.
SUMMARY OF THE INVENTION
This summary is provided to introduce a selection of concepts, in a simplified format, that are further described in the detailed description of the invention.
This summary is neither intended to identify key or essential inventive concepts of the invention and nor is it intended for determining the scope of the invention.
To further clarify advantages and features of the present invention, a more particular description of the invention will be rendered by reference to specific embodiments thereof, which is illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail with the accompanying drawings.
The proposed system is a comprehensive and intelligent framework which is designed to predict and monitor the status of the mental health of the students using a combination of p number edge vision devices deployed in the classrooms settings with n number of wearable physiological sensors devices. At the heart of the system there is wearable smart band that each student wears throughout the day. This band integrates several critical biosensors such as photoplethysmography (PPG) sensor for capturing heart rate and heart rate variability (HRV), a pulse oximeter (SpO₂) which measures the blood oxygen levels, a galvanic skin response (GSR) sensor to detect electrodermal activity, a skin temperature sensor and an internal measurement unit to monitor physical movement and sleep pattern. These sensors collectively provide a rich stream of real-time physiological data that correlates strongly with anxiety, fatigue, stress and other physiological states.
The invention relates to a real-time mental health monitoring system for students. The system integrates wearable sensor devices that are configured to measure various physiological parameters together with edge vision devices that capture and process facial expressions. These devices are communicatively coupled via Wi-Fi to a cloud server, where a machine learning model resides. The machine learning model analyzes the combined physiological and emotional data to predict the mental health status of students and displays the results on an interactive dashboard. The wearable device incorporates multiple biosensors, including a photoplethysmography (PPG) sensor, a pulse oximeter (SpO₂), a galvanic skin response (GSR) sensor, a skin temperature sensor, and an inertial measurement unit (IMU), all of which contribute to comprehensive data collection. A microcontroller embedded within the wearable device is responsible for local preprocessing tasks such as normalization, noise filtering, and temporary data storage before the physiological data is transmitted to the cloud.
The edge vision device is equipped with a high-definition (HD) camera coupled with an embedded processor that converts captured image frames into numerical emotion scores without transmitting raw images, thereby ensuring privacy. An AI-accelerated processor within the device is further configured to classify emotional states such as happiness, sadness, anger, stress, fear, and confusion. On the cloud side, the machine learning model is trained using pre-labeled physiological and emotional datasets and is capable of classifying student mental health status into different stress levels or probability scores of specific conditions such as anxiety or fatigue. The model is adaptive in nature and can be periodically retrained with new data to improve prediction accuracy.
The wearable sensor device is powered by a rechargeable battery, while the edge vision device derives power from a solar panel-charged battery system, ensuring sustainable operation in classroom environments. The results of the analysis are presented on a dashboard accessible to authorized counsellors, administrators, or caregivers, providing insights at both individual and classroom levels and enabling early intervention through alerts. Importantly, the system is designed with privacy as a priority, as the cloud server processes only numerical data and does not transmit identifiable content such as raw images or audio, thereby maintaining confidentiality while enabling effective real-time monitoring of student mental health.
BRIEF DESCRIPTION OF THE DRAWINGS
The illustrated embodiments of the subject matter will be understood by reference to the drawings, wherein like parts are designated by like numerals throughout. The following description is intended only by way of example, and simply illustrates certain selected embodiments of devices, systems, and methods that are consistent with the subject matter as claimed herein, wherein:
Fig. 1 Overall Architecture
Fig.2 Smart Wearable Sensory Data Collector Device
Fig 3 Edge Vision Devices
The figures depict embodiments of the present subject matter for the purposes of illustration only. A person skilled in the art will easily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the disclosure described herein.
DETAILED DESCRIPTION OF THE INVENTION
The detailed description of various exemplary embodiments of the disclosure is described herein with reference to the accompanying drawings. It should be noted that the embodiments are described herein in such details as to clearly communicate the disclosure. However, the amount of details provided herein is not intended to limit the anticipated variations of embodiments; on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the scope of the present disclosure as defined by the appended claims.
It is also to be understood that various arrangements may be devised that, although not explicitly described or shown herein, embody the principles of the present disclosure. Moreover, all statements herein reciting principles, aspects, and embodiments of the present disclosure, as well as specific examples, are intended to encompass equivalents thereof.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms “a",” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes” and/or “including,” when used herein, specify the presence of stated features, integers, steps, operations, elements and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or groups thereof.
It should also be noted that in some alternative implementations, the functions/acts noted may occur out of the order noted in the figures. For example, two figures shown in succession may, in fact, be executed concurrently or may sometimes be executed in the reverse order, depending upon the functionality/acts involved.
In addition, the descriptions of "first", "second", “third”, and the like in the present invention are used for the purpose of description only, and are not to be construed as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" and "second" may include at least one of the features, either explicitly or implicitly.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which example embodiments belong. It will be further understood that terms, e.g., those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The proposed system is a comprehensive and intelligent framework which is designed to predict and monitor the status of the mental health of the students using a combination of p number edge vision devices deployed in the classrooms settings with n number of wearable physiological sensors devices. At the heart of the system there is wearable smart band that each student wears throughout the day. This band integrates several critical biosensors such as photoplethysmography (PPG) sensor for capturing heart rate and heart rate variability (HRV), a pulse oximeter (SpO₂) which measures the blood oxygen levels, a galvanic skin response (GSR) sensor to detect electrodermal activity, a skin temperature sensor and an internal measurement unit to monitor physical movement and sleep pattern. These sensors collectively provide a rich stream of real-time physiological data that correlates strongly with anxiety, fatigue, stress and other physiological states.
There is a microcontroller which is embedded in the smart band which handles preprocessing tasks on-device such as normalization, noise filtering and temporary data storage. This device is powered by a rechargeable battery which ensures its long-term use without frequent interruptions. The band uses Wi-Fi as its primary communication protocol to send the encrypted and structured sensor data to a cloud server by the means of Wi-Fi. The data is transferred regular intervals or based on predefined event triggers such as sudden changes in GSR or heart rate. Most importantly, the system is designed such that to preserve the privacy of the student by transmitting the numerical data only and no identifiable information.
Complementing the wearable device there are p number of edge vision devices installed in the classrooms strategically. The system comprises of a HD camera which is connected to Raspberry Pi which is equipped with AI acceleration capabilities. These devices continuously monitor the facial expressions of the students. These models have capability to convert the image data in numerical emotion score data by predicting the various emotional states such as happiness, sadness, anger, fear, stress and confusion. Then, the models send the real-time data to the cloud server by the means of Wi-Fi. For ensuring the privacy and reducing the bandwidth usage no raw images or video streams are uploaded.
Both physiological as well as the emotional data streams are synchronized cleaned and transformed in cloud server into structured feature as per suitability for machine learning. There exists a pre-trained machine learning model from existing data of same manner and the data pre-processed by the cloud server is sent to this model for processing. The machine learning model on the basis of data received assess each student’s mental health status. The output is a risk- level classification- such as low, moderate or high-stress or a probability score indicating the likelihood of specific mental health conditions.
Edge vision device gets energy from a battery which is charged by the solar panels placed on rooftop of the institutions or on the walls outside and the and wearable sensor device have inbuilt rechargeable battery on which the device run.
The final result predicted by the machine learning model is now displayed on the dashboard which may be available to the school counsellors, administrators or authorized caregivers. This dashboard shows individual as well as the class-level trends, early warning signals and also provides suggestions for interventions. Optionally, students also can access a personalised interface with mental wellness tips, trend analysis and feedbacks. By combining all the factors such as real-time physiological monitoring, analysis of emotional behaviour and AI-based prediction this system offers a privacy-conscious, scalable and impactful approach for promoting the mental well-being among students.
A machine learning algorithm is used by the system to analyse the multimodal data collected from both wearable sensor devices as well as the edge vision devices for predicting the mental health status of the students. At initial stage raw physiological signals as well as facial scores are pre-processed for extracting meaningful features of the student such as heart rate variability, emotion probabilities and stress indicators. These features are then given as input into a pre-trained classification model Support Vector Machine (SVM), which has been trained on labelled mental health data. The model learns complex patterns and correlations between emotional states and physiological responses to identify potential mental health risks. It gives outputs predictions in the form of the form of stress levels or categorised mental health conditions such as fatigue and anxiety. Also, the algorithm is designed such that it is adoptive in nature and may be periodically retained using new data to improve the accuracy. The intelligent decision-making process enables personalised and real-time mental health monitoring for each and every student of the class.
There are four main steps involved in the algorithm for soil health prediction. First step is data collection by gathering inputs from the virous sensor used i.e. NPK sensor, pH sensor, temperature & humidity sensor, soil conductivity sensor, GPS sensor and an HD camera attached to the raspberry pi. Second step is preprocessing in which normalization of the sensor data is done and also preparation of images for analysis is made. Third step is feature extraction in which identification of the key characteristics is made from both the sensor data i.e. nutrient levels, humidity & temperature etc. and images data like soil texture, surface anomalies. In fourth step, finally a Machine learning model is trained by using the combined features for predicting soil health which offes real-time insights for precision agriculture. The algorithm used is as follows:
Algorithm Used
BEGIN
// Step 1: Initialize Devices
Initialize WearableSensorModule()
Initialize EdgeVisionDevice()
ConnectToWiFi()
ConnectToCloudServer()
// Step 2: Continuous Data Collection
WHILE system_is_active DO
// Step 2.1: Collect physiological data from wearable band
SpO2 ← ReadSpO2()
PPG ← ReadPPG()
GSR ← ReadGSR()
SkinTemp ← ReadSkinTemperature()
IMU ← ReadMotionAndSleepData()
// Step 2.2: Collect facial emotion data from edge vision device
ImageFrame ← CaptureImageFromCamera()
EmotionScores ← ProcessImageForEmotions(ImageFrame)
// Step 3: Preprocessing
PhysiologicalFeatures ← PreprocessSensorData(SpO2, PPG, GSR, SkinTemp, IMU)
EmotionFeatures ← ConvertEmotionScoresToNumerical(EmotionScores)
// Step 4: Merge data streams
CombinedFeatures ← Merge(PhysiologicalFeatures, EmotionFeatures)
// Step 5: Send features to cloud server
TransmitToCloud(CombinedFeatures)
// Step 6: Cloud-side ML inference
IF NewDataReceivedOnCloud THEN
CleanedData ← CleanAndNormalize(CombinedFeatures)
Prediction ← MLModel.Predict(CleanedData)
StorePrediction(Prediction)
// Step 7: Notify relevant dashboards
UpdateCounselorDashboard(Prediction)
IF HighRiskDetected(Prediction) THEN
TriggerAlertNotification()
ENDIF
ENDIF
WAIT for next interval
END WHILE
END
ADVANTAGES OF THE INVENTION:
This system offers several key advantages that make it a valuable tool for mental health detection of the students:
• The behavioral cues from facial expressions with physiological signals from wearable devices. This multimodal fusion promotes accuracy and reliability of mental health predictions. It provides a more holistic view physical as well as emotional state of students.
• Real-time tracking of the mental health indicator is enabled by continuous data collection. The system is created such that it can detect abnormal patterns instantly and send alerts to caregivers or counsellors. Hence, it allows for early intervention before problem escalates.
• The cloud server only receives the numerical data from the system. There is no exchange of raw data like image, audio or identifiable content. Edge processing of facial expressions ensures sensitive information remains local. This ensures privacy to the students while still enabling effective monitoring.
• The proposed system can be deployed among multiple classrooms with minimal infrastructure. Wi-Fi based connectivity allows seamless integration of both edge and wearable devices.
• The intelligent dashboard provides the insights which are backed by the real-time data trends. Hence, counsellors receive actionable information to tailor interventions for each and every student. Over the times it supports mental wellness strategies and data-driven policy decisions for the institutions.
, Claims:1. A real-time mental health monitoring system for students, comprising:
a plurality of wearable sensor devices configured to measure physiological parameters;
a plurality of edge vision devices configured to capture and process facial expressions;
a cloud server communicatively coupled to said wearable sensor devices and said edge vision devices via Wi-Fi; and
a machine learning model residing on said cloud server, configured to analyze combined physiological and emotional data for predicting mental health status and displaying results on a dashboard.
2. The system as claimed in claim 1, wherein said wearable sensor device comprises a photoplethysmography (PPG) sensor, a pulse oximeter (SpO₂), a galvanic skin response (GSR) sensor, a skin temperature sensor, and an inertial measurement unit (IMU).
3. The system as claimed in claim 2, wherein said wearable sensor device further comprises a microcontroller configured to perform local preprocessing including normalization, noise filtering, and temporary data storage before transmitting said physiological data.
4. The system as claimed in claim 1, wherein said edge vision device comprises a high-definition (HD) camera coupled with an embedded processor configured to convert captured image frames into numerical emotion scores without transmitting raw images.
5. The system as claimed in claim 4, wherein said edge vision device further comprises an AI-accelerated processor configured to classify emotional states including happiness, sadness, anger, stress, fear, and confusion.
6. The system as claimed in claim 1, wherein said machine learning model is trained using pre-labeled physiological and emotional datasets and is configured to classify student mental health status into stress levels or probability scores of specific conditions such as anxiety or fatigue.
7. The system as claimed in claim 6, wherein said machine learning model is adaptive and periodically retrained with new data to improve prediction accuracy over time.
8. The system as claimed in claim 1, wherein said wearable sensor device is powered by a rechargeable battery and said edge vision device is powered by a solar panel-charged battery system.
9. The system as claimed in claim 1, wherein said dashboard is accessible to authorized counsellors, administrators, or caregivers, and configured to display individual as well as classroom-level trends with early intervention alerts.
10. The system as claimed in claim 1, wherein said cloud server is configured to process only numerical data, thereby ensuring privacy by preventing transmission of identifiable information such as raw images or audio data.