1. A system (100) for real-time multimodal classroom engagement monitoring and personalized learning intervention, comprising:  a plurality of high-resolution vision sensors (182) mounted overhead or front-facing along classroom walls so that each student’s face and upper torso lie within the field of view;  an edge computing node configured to receive video frames from the plurality of vision sensors (182) and to extract abstracted feature vectors including body keypoints, gaze vectors and motion blobs, the edge computing node transmitting only the abstracted feature vectors to downstream modules;  a computer vision processing pipeline (110) executed by one or more processors, the pipeline receiving the abstracted feature vectors and performing student detection and tracking using a one-stage anchor-based object detector, pose and gaze estimation via a Holistic model to extract skeletal keypoints and facial mesh landmarks and to compute gaze vectors, optical-flow computation to derive motion intensity and optical-flow entropy, posture classification into upright, leaning or slouching states via a multilayer perceptron, and temporal aggregation of gaze vectors over sliding windows to generate attention heatmaps, thereby producing time-stamped behavioral feature vectors;  a psychometric assessment module (130) configured to receive psychometric test results (186) and to generate a trait profile vector for each student;  a signal-level feature fusion module configured to receive the time-stamped behavioral feature vectors and the trait profile vector and to compute a normalized engagement index per student by applying a weighted fusion mechanism to continuous signals including gaze directionality, posture class probabilities and motion intensity, and to discrete signals including hand-raise counts and quiz response correctness, and by passing a weighted sum of the continuous and discrete signals through a sigmoid function;  an AI inference engine (120) configured to infer latent cognitive-state probabilities for each student by clustering the fused engagement features using a Gaussian Mixture Model and by modeling temporal state transitions using a Hidden Markov Model, and to detect when the engagement index remains below a configurable threshold for a predefined duration;  a generative intervention module configured to, upon detection of low engagement, invoke a transformer-based language model fine-tuned on educational dialogue using contextual inputs comprising the student’s trait profile vector, current lesson module identifier and recent response history to generate personalized pedagogical hints or prompts;  an adaptive orchestration controller configured as a reinforcement-learning agent to select and schedule interventions at individual, group or whole-class levels based on a learned policy that maximizes a reward signal combining post-intervention engagement recovery and quiz performance improvements;  a teacher dashboard (164) comprising a web-based interface that visualizes live engagement heatmaps overlaid on a classroom floor plan, individual student engagement timelines and clustering-based cognitive-state labels, and that provides one-click controls for intervention customization; and  a data and infrastructure layer (150) comprising a data lake for raw video frames, screen captures and psychometric records, a feature store for curated behavioral embeddings, and a containerized microservices architecture orchestrated by Kubernetes to enable scalable, fault-isolated and privacy-preserving operation.

2. The system as claimed in claim 1, wherein the one-stage anchor-based object detector of the computer vision processing pipeline (110) comprises a YOLO-v3 to YOLO-v5 network executing at approximately thirty frames per second.

3. The system as claimed in claim 1, wherein the student detection and tracking of the computer vision processing pipeline (110) further comprises a lightweight tracker that fuses Kalman-filter predictions with appearance descriptors to maintain persistent student identifiers.

4. The system as claimed in claim 1, wherein the posture classification submodule of the computer vision processing pipeline (110) receives angle-based features comprising torso lean angle and head-torso offset and classifies posture via a multilayer perceptron trained to output upright, leaning or slouching probabilities.

5. The system as claimed in claim 1, wherein the weighted fusion mechanism initializes weights heuristically based on domain knowledge and adaptively calibrates the weights by minimizing a logistic-regression loss on a validation set of expert-annotated engagement labels.

6. The system as claimed in claim 1, wherein the Hidden Markov Model of the AI inference engine (120) captures temporal cognitive-state transitions by modeling state dwell times and transition probabilities between focused, wandering and confused states.

7. The system as claimed in claim 1, wherein the generative intervention module invokes the transformer-based language model using inputs comprising the student’s trait profile vector, current lesson module identifier and recent quiz response history to generate scaffolded hints of adjustable difficulty.

8. The system as claimed in claim 1, wherein the adaptive orchestration controller employs a Deep Q-Network to learn an optimal intervention policy mapping student state representations to actions comprising individual hint delivery, group prompting and whole-class pacing adjustments.

9. The system as claimed in claim 1, further comprising a screen-capture module (184) configured to record on-screen student activities and to forward application usage features and digital interaction events to the AI inference engine (120).

10. A method for real-time multimodal classroom engagement monitoring and personalized learning intervention, comprising:  receiving, at an edge computing node, video frames from a plurality of high-resolution vision sensors (182) mounted in a classroom;  extracting, by a computer vision processing pipeline (110), abstracted feature vectors including student bounding boxes and identifiers via anchor-based object detection and tracking, skeletal keypoints and facial mesh landmarks via a Holistic model, gaze vectors via a PnP solve on three-dimensional landmarks, motion intensity and optical-flow entropy via Farneback computation, and posture probabilities via a multilayer perceptron;  aggregating the gaze vectors over sliding time windows to generate attention heatmaps;  receiving psychometric test results (186) and generating a trait profile vector for each student via a psychometric assessment module (130);  fusing the abstracted feature vectors and the trait profile vector into a normalized engagement index per student by applying a weighted fusion mechanism to continuous signals and discrete signals and by passing the weighted sum through a sigmoid function;  inferring latent cognitive-state probabilities by clustering recent engagement features using a Gaussian Mixture Model and by modeling temporal transitions with a Hidden Markov Model;  upon detection that the engagement index remains below a configurable threshold for a predefined duration, invoking a transformer-based language model fine-tuned on educational dialogue with contextual inputs comprising the student’s trait profile vector, current lesson module and recent response history to generate a personalized pedagogical intervention;  selecting and scheduling the personalized intervention at an individual, group or whole-class level via a reinforcement-learning-based orchestration controller trained to maximize a reward signal combining post-intervention engagement recovery and quiz performance improvements; and  displaying the personalized intervention and live engagement analytics via a teacher dashboard (164).

Description:FIELD OF THE INVENTION
The present invention relates to computer-implemented education technologies, and more particularly to a system and method for real-time AI-driven multimodal classroom engagement monitoring and personalized learning intervention.
BACKGROUND OF THE INVENTION
In heterogeneous and remote learning environments, instructors lack objective, real-time indicators of student attention and comprehension. Traditional lecture formats and manual observation fail to address the wide variation in learning speeds, styles and baseline competencies within a single cohort. In large-scale online courses and hybrid classrooms, high student-to-instructor ratios make continuous monitoring impractical, and disengagement often remains undetected until performance has already deteriorated. Institutions increasingly require data-driven tools to improve retention, completion rates and learning outcomes, yet available solutions struggle to balance scalability, privacy and richness of behavioural insight.
Current approaches to classroom engagement monitoring and personalised instruction rely largely on coarse metrics such as clickstream logs, quiz results and text-based interactions. Some systems employ specialised eye-tracking devices or wearable sensors to capture gaze and physiological signals, but these solutions are expensive, intrusive and difficult to deploy at scale. Analytic engines often operate in batch mode, generating insights only after delays, while fusion of multimodal signals is frequently rule-based and static, preventing adaptation to moment-to-moment shifts in student affect and cognitive state. Educators are left to interpret disparate dashboards and manually devise interventions without automated guidance, exacerbating workload and limiting responsiveness.
A prior art US11610500 discloses an AI-driven classroom engagement platform that derives student attention metrics from digital interaction records and performance data, adapting content delivery based on aggregated quiz results while overlooking real-time behavioural signals and dynamic personalisation.
A prior art WO2022214992A1 describes a multimodal engagement monitoring framework employing vision and audio sensors to infer cognitive states, using predefined fusion rules to combine gaze, posture and speech analytics, but relying on specialised hardware and offline processing that restrict scalability and responsiveness.
These existing solutions assume homogeneous instruction, focus predominantly on text-centric or clickstream data, and lack fine-grained nonverbal behavioural insight. High-cost sensing hardware and batch-oriented analytics introduce latency that prevents immediate intervention. Static fusion approaches diminish sensitivity to evolving classroom dynamics, and the absence of a closed-loop orchestration layer forces educators to generate corrective actions manually. Privacy and feasibility concerns further limit real-world adoption, leaving a clear gap between current capabilities and the needs of modern educational settings.
Accordingly, there remains a compelling need for a system and method for real-time multimodal classroom engagement monitoring and personalized learning intervention.
OBJECTS OF THE INVENTION
An object of the invention is to provide an end-to-end, real-time classroom monitoring and intervention system built on three interlocking pillars multimodal sensing, signal-level feature fusion, and closed-loop cognitive-state inference with adaptive orchestration that delivers personalized pedagogical support with sub-second responsiveness.
Another object of the invention is to provide a privacy-aware multimodal sensing architecture that deploys high-resolution overhead and frontal cameras in classrooms, processes raw video on an edge GPU to extract body keypoints, gaze vectors, and motion blobs, and forwards only abstracted feature vectors to downstream modules, thereby preserving student privacy and minimizing data transmission.
Another object of the invention is to provide a computer-vision processing pipeline that combines YOLO-based student detection and SORT tracking at \~30 FPS, MediaPipe holistic pose and gaze estimation, Farneback optical-flow motion analysis, and angle-based posture classification via a lightweight MLP to derive fine-grained behavioral cues indicative of engagement.
Yet another object of the invention is to provide a temporal attention-heatmap generation method that aggregates gaze-vector and head-pose data over sliding windows to visualize attention drift across instructional targets, peers, or off-screen regions in real time.
Another object of the invention is to provide a psychometric-vision fusion mechanism that constructs continuous (gaze, posture, motion intensity) and discrete (hand-raise counts, quiz correctness) feature vectors, applies heuristic weight initialization and adaptive calibration through logistic-regression loss minimization, and yields a normalized scalar engagement index in the range [0,1].
Another object of the invention is to provide an AI inference engine that employs Gaussian Mixture Models for soft-assignment clustering of engagement vectors, Hidden Markov Models to model temporal state transitions, and a transformer-based generative language model that produces context-aware, personalized instructional prompts when engagement dips below a predefined threshold.
Another object of the invention is to provide an adaptive orchestration controller based on reinforcement learning that learns optimal intervention policies choosing between individual, group, or whole-class prompts by maximizing a reward signal combining post-intervention engagement recovery and subsequent quiz performance.
Another object of the invention is to provide a real-time teacher dashboard that overlays live engagement heatmaps on a classroom floor plan, displays individual and cohort engagement timelines with cognitive-state labels, and enables one-click or customized intervention triggers to streamline in-class decision-making.
Another object of the invention is to provide a scalable microservices-based infrastructure in which the vision pipeline, fusion engine, inference engine, and dashboard API operate as containerized modules orchestrated by Kubernetes, with a time-series database logging all feature embeddings and inference outputs for later analysis.
Yet another object of the invention is to provide a continuous learning framework that refines fusion weights and intervention policies through feedback from expert-annotated ground-truth engagement labels and student performance metrics, thereby improving classification accuracy and personalization over successive classroom deployments.
SUMMARY OF THE INVENTION
The present invention is described in the following sections by various embodiments. However, it should be understood that the invention can be implemented in various forms and is not limited to the specific embodiment provided herein.In the context of the present disclosure, it should be understood that the described embodiments in this section are put forth for illustrative purposes only. Those skilled in the art will appreciate that various modifications, adaptations, and alternative designs may be employed without departing from the scope and spirit of the invention. Accordingly, the present invention should not be limited to the specific embodiments illustrated herein, but rather should be construed according to the claims and description that follow.
Embodiments of the present invention provide a system for real-time multimodal classroom engagement monitoring and personalized learning intervention, the system comprises, but not limited to, a plurality of high-resolution cameras mounted overhead or front-facing along classroom boundaries so that faces and upper torsos of all students are within fields of view; an edge computing node configured to receive video streams and to extract and transmit only abstracted behavioral feature vectors including body keypoints, gaze vectors and motion blobs; a computer-vision processing pipeline executed by one or more processors to detect and track students via an anchor-based object detector, to estimate skeletal keypoints and facial landmarks via a holistic model and compute gaze vectors, to compute motion intensity and entropy via optical flow, to classify posture into upright, leaning or slouching states via a multilayer perceptron, and to aggregate gaze over sliding windows to generate attention heatmaps thereby producing time-stamped behavioral feature vectors; a psychometric assessment module to ingest psychometric test results and to generate a trait profile vector for each student; a signal-level feature fusion module to receive the behavioral and trait profile vectors and to compute a normalized engagement index per student by applying a weighted fusion mechanism to continuous signals comprising gaze directionality, posture probabilities and motion intensity and to discrete signals comprising hand-raise counts and quiz response correctness and by passing the weighted sum through a sigmoid function; an AI inference engine to cluster fused features into latent cognitive-state probabilities using a Gaussian mixture model and to model temporal state transitions via a hidden Markov model and to detect when engagement remains below a threshold for a predefined duration; a generative intervention module to invoke a transformer-based language model fine-tuned on educational dialogue using contextual inputs comprising the trait profile, current lesson identifier and recent response history to generate personalized pedagogical prompts; an adaptive orchestration controller configured as a reinforcement-learning agent to select and schedule interventions at individual, group or whole-class levels based on a reward signal combining engagement recovery and performance improvements; a teacher dashboard providing real-time heatmaps, engagement timelines and one-click intervention controls; and a data and infrastructure layer comprising a data lake, feature store and containerized microservices orchestrated by Kubernetes for scalable, fault-isolated and privacy-preserving operation.
The system as defined above delivers multiple advantages by enabling continuous, sub-second detection of student engagement and cognitive states within the classroom while preserving privacy through on-site feature extraction, by fusing heterogeneous visual and psychometric signals into a robust engagement index that adapts to different pedagogical contexts, and by closing the loop with AI-driven, reinforcement-learning-guided interventions that are personalized to individual learners or groups—thereby improving learning outcomes, reducing teacher workload, and maintaining seamless instructional flow.
In accordance with an embodiment of the present invention the one-stage anchor-based object detector of the computer-vision processing pipeline comprises a YOLO-v3 to YOLO-v5 network configured to operate at approximately thirty frames per second, whereby rapid and accurate localization of student regions supports downstream tracking and feature extraction with minimal latency.
In accordance with an embodiment of the present invention the student detection and tracking further comprises a lightweight tracker that fuses Kalman-filter predictions with appearance descriptors to maintain persistent student identifiers across frames, enabling reliable temporal association of behavior features for each learner.
In accordance with an embodiment of the present invention the posture classification submodule receives angle-based features including torso lean angle and head-to-torso offset and classifies each posture instance via a multilayer perceptron trained to output probabilities for upright, leaning or slouching classes, thereby capturing subtle changes in body language that reflect attentiveness.
In accordance with an embodiment of the present invention the weighted fusion mechanism initializes modality weights based on pedagogical domain knowledge and adaptively calibrates these weights by minimizing a logistic-regression loss on a validation set of expert-annotated engagement labels, ensuring that the composite engagement index remains aligned with human-rated ground truth and adapts to varying classroom populations.
In accordance with an embodiment of the present invention the hidden Markov model captures temporal cognitive-state transitions by modeling state dwell times and transition probabilities among focused, wandering and confused states, thereby enhancing the inference engine with sequential dynamics for more nuanced mental-state predictions.
In accordance with an embodiment of the present invention the generative intervention module invokes a transformer-based language model using contextual inputs comprising each student’s trait profile vector, the current lesson module identifier and recent quiz response history to generate scaffolded hints of adjustable difficulty, facilitating personalized prompts that match the learner’s current needs.
In accordance with an embodiment of the present invention the adaptive orchestration controller employs a deep Q-network to learn an optimal intervention policy that maps student state representations to actions including individual hint delivery, group prompting and whole-class pacing adjustments, thus balancing immediacy of feedback with overall classroom harmony.
In accordance with an embodiment of the present invention the system further comprises a screen-capture module configured to record on-screen student activities and to forward application usage features and digital interaction events to the AI inference engine, enriching the engagement assessment with direct evidence of student–software interactions.
In accordance with an embodiment of the present invention the AI inference engine integrates application usage features with video-derived behavioral features by mapping on-screen interactions onto nodes and edges of a domain-specific knowledge graph and performing real-time mastery estimation, thus combining digital learning analytics with physical behavior cues for a comprehensive understanding of learner progress.
Embodiments of the present invention further provide a method for real-time multimodal classroom engagement monitoring and personalized learning intervention, the method comprises, but not limited to, receiving video frames from a plurality of high-resolution vision sensors mounted in a classroom at an edge computing node; extracting abstracted feature vectors including student bounding boxes and identifiers via anchor-based object detection and tracking, skeletal keypoints and facial mesh landmarks via a holistic model, gaze vectors via a perspective-n-point solve on three-dimensional landmarks, motion intensity and optical-flow entropy via Farneback computation, and posture probabilities via a multilayer perceptron; aggregating gaze vectors over sliding time windows to generate attention heatmaps; receiving psychometric test results and generating a trait profile vector for each learner via a psychometric assessment module; fusing the abstracted feature vectors and the trait profile vector into a normalized engagement index per student by applying a weighted fusion mechanism to continuous and discrete signals and by passing the weighted sum through a sigmoid function; inferring latent cognitive-state probabilities by clustering recent engagement features using a Gaussian mixture model and modeling temporal transitions with a hidden Markov model; upon detecting that the engagement index remains below a configurable threshold for a predefined duration invoking a transformer-based language model fine-tuned on educational dialogue with contextual inputs comprising each student’s trait profile vector, current lesson module and recent response history to generate a personalized intervention; selecting and scheduling the personalized intervention at individual, group or whole-class level via a reinforcement-learning-based orchestration controller trained to maximize a reward signal combining post-intervention engagement recovery and quiz performance improvements; and displaying the personalized intervention and live engagement analytics via a teacher dashboard.
In accordance with an embodiment of the present invention the anchor-based object detection operates at at least thirty frames per second using a YOLO-v3 or YOLO-v5 network, ensuring that capture-to-analysis latency remains low enough to support responsive classroom feedback loops.
In accordance with an embodiment of the present invention the method further comprises calibrating gaze estimation per classroom by instructing a sample student to fixate on known reference points to refine camera intrinsic and extrinsic parameters, thereby improving accuracy of projected gaze vectors.
In accordance with an embodiment of the present invention the weighted fusion mechanism adaptively calibrates signal weights by minimizing logistic-regression loss on a validation set of expert-annotated engagement labels, enabling the method to fine-tune modality contributions based on actual classroom evaluations.
In accordance with an embodiment of the present invention the reinforcement-learning-based orchestration controller employs a deep Q-network to learn an optimal policy mapping cognitive-state and engagement index inputs to intervention actions, dynamically adjusting educational strategies in response to observed student behavior.
In accordance with an embodiment of the present invention the method further comprises measuring heart-rate variability and galvanic skin response via wearable photoplethysmography bands and incorporating these physiological signals as additional continuous inputs to the fusion mechanism, thereby extending the multimodal engagement assessment with direct indicators of student arousal.
BRIEF DESCRIPTION OF DRAWINGS
In order to facilitate a comprehensive understanding of the detailed features of the present invention, a more specific description of the invention, briefly summarized above, may have been referenced through various embodiments, some of which are depicted in the accompanying drawings. It should be emphasized, however, that the provided drawings merely exemplify typical embodiments of the present invention and should not be construed as limiting its scope, as the invention may encompass other equally efficacious embodiments.
These and additional features, advantages, and benefits of the present invention will become apparent by consulting the following textual illustration, wherein similar reference numerals denote similar components throughout the various views.
Fig. 1 illustrates a schematic diagram of the proposed model.
DETAILED DESCRIPTION OF THE INVENTION
The present invention is subsequently described herein using various embodiments with reference to the accompanying drawing, wherein the reference numerals utilized in the accompanying drawing correspond to the similar elements throughout the description. While the present invention is illustratively described herein by way of example using embodiments and accompanying drawings, those skilled in the art will acknowledge that the invention is not limited to the described embodiments or drawings and is not intended to represent the scale of the different components. Furthermore, certain components that may constitute a part of the invention might not be depicted in specific figures for the purpose of simplified illustration, and such omissions do not restrict the outlined embodiments in any manner. It should be comprehended that the drawings and the detailed description provided are not intended to limit the invention to the particular disclosed form, but instead, the invention is intended to encompass all modifications, equivalents, and alternatives falling within the scope of the present invention as defined by the appended claim. Throughout this description, the term 'may' is used in a permissive sense, indicating the potential to, rather than in a mandatory sense, indicating a requirement. Additionally, the words 'a' or 'an' signify at least one, and the word 'plurality' signifies 'one or more' unless otherwise specified. Moreover, the terminology and phraseology employed herein are solely for descriptive purposes and should not be construed as limiting in scope. Terms such as 'including', 'comprising', 'having', 'containing', or 'involving', and their variations, are intended to be broad and encompass the listed subject matter thereafter, as well as equivalents and additional subject matter not explicitly mentioned, and should not be interpreted as excluding other additives, components, integers, or steps. Similarly, the term 'comprising' is considered synonymous with the terms 'including' or 'containing' for applicable legal purposes.
The invention provides a real‐time, end‐to‐end system for monitoring classroom engagement and delivering personalized learning interventions. At its core the system comprises but is not limited to a multimodal sensing architecture that captures visual, audio and physiological signals, a signal‐level fusion module that computes a unified engagement index, and a closed‐loop inference and orchestration engine that generates and schedules adaptive pedagogical prompts. The invention may continuously process data on an edge computing node to preserve student privacy by transmitting only abstracted feature vectors, while a centralized AI inference engine interprets these features to infer latent cognitive states and trigger timely interventions.
The multimodal sensing architecture comprises but is not limited to high‐resolution cameras positioned overhead or along classroom walls to cover each student’s face and upper torso, optional wearable photoplethysmography bands for heart‐rate variability and galvanic skin response, and directional microphone arrays for voice activity detection. Video streams may be processed by a computer vision pipeline employing one‐stage, anchor‐based detection for student localization, MediaPipe‐style pose and gaze estimation for skeletal keypoints and eye‐gaze vectors, optical‐flow analysis for motion intensity, and a multilayer perceptron for posture classification. Temporal aggregation of gaze vectors over sliding windows may yield attention heatmaps that feed into downstream fusion.
A psychometric assessment module may supply trait profile vectors which, together with time‐stamped behavioral feature vectors, are input to a weighted fusion mechanism to produce a normalized engagement index. The fusion mechanism may initialize weights heuristically and adaptively recalibrate via loss minimization on expert‐annotated labels. An AI inference engine may apply Gaussian Mixture Models and Hidden Markov Models to cluster engagement features into cognitive states and model temporal transitions. Upon detecting sustained low engagement, a generative language model may generate tailored hints using contextual inputs, and an adaptive orchestration controller based on reinforcement learning may select individual, group or whole‐class interventions. A teacher dashboard may visualize live engagement heatmaps, student timelines and recommended actions, while a microservices‐based infrastructure layer ensures scalability, privacy and fault isolation.
Various modifications to these embodiments are evident to those skilled in the art based on the description. The principles associated with the various embodiments described herein can be applied to additional embodiments. Consequently, the description is not intended to be limited to the embodiments but aims to provide the broadest scope consistent with the principles and the innovative and inventive features disclosed or suggested herein. Therefore, the invention is expected to encompass all other such alternatives, modifications, and variations falling within the scope of the present invention.
Figure 1 illustrates a high‐level block diagram of the proposed end‐to‐end, real-time classroom engagement and personalized learning platform, showing the flow of raw inputs through successive processing modules to the final delivery channels. Input & data sources such as overhead or side-wall mounted HD cameras capture student facial expressions and upper-torso movements at 30 FPS and 1080p resolution, while screen-capture agents record on-screen activity at up to 60 Hz with 24-bit color fidelity. Encrypted video and activity streams traverse a secured Ethernet or Wi-Fi network to reach the computer vision module (1). Module (1) resides on a rack-mounted blade server housing one or more GPU cards within a sealed metal enclosure and is implemented as a Dockerized microservice. Time-stamped behavioral feature vectors output by module (1) are transmitted via RESTful API calls to the reasoning engine (2).
Psychometric assessment module (3) receives standardized test results, learning-style inventories and cognitive-ability scores stored in a relational database subsystem of the data & infrastructure layer (5). Module (3) transforms these trait measures into a fixed-length vector representation via a feature store implemented on a distributed NoSQL cluster. Reasoning engine (2) fuses trait vectors from module (3) with behavioral vectors from module (1) by applying knowledge-graph embeddings and rule-based recommendation logic within a Kubernetes-orchestrated inference service. Fused outputs propagate to student profile & learning orchestrator (4), which dynamically schedules personalized content delivery, adjusts difficulty levels and issues context-aware feedback prompts according to predefined orchestration policies.
In one embodiment of the invention the computer vision module (1) comprises but is not limited to a YOLO-based one-stage object detector operating at 30 FPS and a MediaPipe-inspired pose and gaze estimator that yields 3D skeletal keypoints and head-pose vectors. Module (1) is encapsulated in a Docker container configured with CUDA-accelerated libraries and communicates with adjacent modules via gRPC over a secured network fabric. Calibration parameters for each camera—including focal length, distortion coefficients and mounting coordinates—are stored in the infrastructure layer (5) to ensure sub-second end-to-end latency. An edge-deployed variant of module (1) may execute on NVIDIA Jetson Nano devices enclosed in IP64-rated housings to preserve on-premise privacy.
Student profile & learning orchestrator (4) aggregates fused engagement scores and psychometric profiles to construct a unified, continuously updated student model. Orchestrator (4) implements reinforcement-learning policies within a TensorFlow Serving container that invokes a generative-AI intervention module via an internal LLM API. Tailored prompts are formatted as JSON payloads and dispatched to delivery channels—including a web application, a native mobile application and a teacher dashboard—through a message broker. Data & infrastructure layer (5) persists engagement indices, cognitive-state transitions and intervention histories in a time-series database, while monitoring and logging services collect system health metrics. An embodiment of the present invention may further include optional wearable PPG sensors interfaced via Bluetooth Low Energy to enrich physiological inputs without altering the core microservices architecture.
Various modifications to these embodiments are evident to those skilled in the art based on the description and accompanying drawings. The principles associated with the various embodiments described herein can be applied to additional embodiments. Consequently, the description is not intended to be limited to the embodiments shown in conjunction with the accompanying drawings but aims to provide the broadest scope consistent with the principles and the innovative and inventive features disclosed or suggested herein. Therefore, the invention is expected to encompass all other such alternatives, modifications, and variations falling within the scope of the present invenion and the appended claims.
, C , Claims:I/we claim:
1. A system (100) for real-time multimodal classroom engagement monitoring and personalized learning intervention, comprising:
 a plurality of high-resolution vision sensors (182) mounted overhead or front-facing along classroom walls so that each student’s face and upper torso lie within the field of view;
 an edge computing node configured to receive video frames from the plurality of vision sensors (182) and to extract abstracted feature vectors including body keypoints, gaze vectors and motion blobs, the edge computing node transmitting only the abstracted feature vectors to downstream modules;
 a computer vision processing pipeline (110) executed by one or more processors, the pipeline receiving the abstracted feature vectors and performing student detection and tracking using a one-stage anchor-based object detector, pose and gaze estimation via a Holistic model to extract skeletal keypoints and facial mesh landmarks and to compute gaze vectors, optical-flow computation to derive motion intensity and optical-flow entropy, posture classification into upright, leaning or slouching states via a multilayer perceptron, and temporal aggregation of gaze vectors over sliding windows to generate attention heatmaps, thereby producing time-stamped behavioral feature vectors;
 a psychometric assessment module (130) configured to receive psychometric test results (186) and to generate a trait profile vector for each student;
 a signal-level feature fusion module configured to receive the time-stamped behavioral feature vectors and the trait profile vector and to compute a normalized engagement index per student by applying a weighted fusion mechanism to continuous signals including gaze directionality, posture class probabilities and motion intensity, and to discrete signals including hand-raise counts and quiz response correctness, and by passing a weighted sum of the continuous and discrete signals through a sigmoid function;
 an AI inference engine (120) configured to infer latent cognitive-state probabilities for each student by clustering the fused engagement features using a Gaussian Mixture Model and by modeling temporal state transitions using a Hidden Markov Model, and to detect when the engagement index remains below a configurable threshold for a predefined duration;
 a generative intervention module configured to, upon detection of low engagement, invoke a transformer-based language model fine-tuned on educational dialogue using contextual inputs comprising the student’s trait profile vector, current lesson module identifier and recent response history to generate personalized pedagogical hints or prompts;
 an adaptive orchestration controller configured as a reinforcement-learning agent to select and schedule interventions at individual, group or whole-class levels based on a learned policy that maximizes a reward signal combining post-intervention engagement recovery and quiz performance improvements;
 a teacher dashboard (164) comprising a web-based interface that visualizes live engagement heatmaps overlaid on a classroom floor plan, individual student engagement timelines and clustering-based cognitive-state labels, and that provides one-click controls for intervention customization; and
 a data and infrastructure layer (150) comprising a data lake for raw video frames, screen captures and psychometric records, a feature store for curated behavioral embeddings, and a containerized microservices architecture orchestrated by Kubernetes to enable scalable, fault-isolated and privacy-preserving operation.
2. The system as claimed in claim 1, wherein the one-stage anchor-based object detector of the computer vision processing pipeline (110) comprises a YOLO-v3 to YOLO-v5 network executing at approximately thirty frames per second.
3. The system as claimed in claim 1, wherein the student detection and tracking of the computer vision processing pipeline (110) further comprises a lightweight tracker that fuses Kalman-filter predictions with appearance descriptors to maintain persistent student identifiers.
4. The system as claimed in claim 1, wherein the posture classification submodule of the computer vision processing pipeline (110) receives angle-based features comprising torso lean angle and head-torso offset and classifies posture via a multilayer perceptron trained to output upright, leaning or slouching probabilities.
5. The system as claimed in claim 1, wherein the weighted fusion mechanism initializes weights heuristically based on domain knowledge and adaptively calibrates the weights by minimizing a logistic-regression loss on a validation set of expert-annotated engagement labels.
6. The system as claimed in claim 1, wherein the Hidden Markov Model of the AI inference engine (120) captures temporal cognitive-state transitions by modeling state dwell times and transition probabilities between focused, wandering and confused states.
7. The system as claimed in claim 1, wherein the generative intervention module invokes the transformer-based language model using inputs comprising the student’s trait profile vector, current lesson module identifier and recent quiz response history to generate scaffolded hints of adjustable difficulty.
8. The system as claimed in claim 1, wherein the adaptive orchestration controller employs a Deep Q-Network to learn an optimal intervention policy mapping student state representations to actions comprising individual hint delivery, group prompting and whole-class pacing adjustments.
9. The system as claimed in claim 1, further comprising a screen-capture module (184) configured to record on-screen student activities and to forward application usage features and digital interaction events to the AI inference engine (120).
10. A method for real-time multimodal classroom engagement monitoring and personalized learning intervention, comprising:
 receiving, at an edge computing node, video frames from a plurality of high-resolution vision sensors (182) mounted in a classroom;
 extracting, by a computer vision processing pipeline (110), abstracted feature vectors including student bounding boxes and identifiers via anchor-based object detection and tracking, skeletal keypoints and facial mesh landmarks via a Holistic model, gaze vectors via a PnP solve on three-dimensional landmarks, motion intensity and optical-flow entropy via Farneback computation, and posture probabilities via a multilayer perceptron;
 aggregating the gaze vectors over sliding time windows to generate attention heatmaps;
 receiving psychometric test results (186) and generating a trait profile vector for each student via a psychometric assessment module (130);
 fusing the abstracted feature vectors and the trait profile vector into a normalized engagement index per student by applying a weighted fusion mechanism to continuous signals and discrete signals and by passing the weighted sum through a sigmoid function;
 inferring latent cognitive-state probabilities by clustering recent engagement features using a Gaussian Mixture Model and by modeling temporal transitions with a Hidden Markov Model;
 upon detection that the engagement index remains below a configurable threshold for a predefined duration, invoking a transformer-based language model fine-tuned on educational dialogue with contextual inputs comprising the student’s trait profile vector, current lesson module and recent response history to generate a personalized pedagogical intervention;
 selecting and scheduling the personalized intervention at an individual, group or whole-class level via a reinforcement-learning-based orchestration controller trained to maximize a reward signal combining post-intervention engagement recovery and quiz performance improvements; and
 displaying the personalized intervention and live engagement analytics via a teacher dashboard (164).