Description:FORM 2

THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003

COMPLETE SPECIFICATION
(See Section 10 and Rule 13)

Title of invention:

METHOD AND SYSTEM FOR ASSESSING MENTAL WORKLOAD USING DIFFERENTIAL DERMAL POTENTIAL RECORDED FROM EAR

Applicant

Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th floor,
Nariman point, Mumbai 400021,
Maharashtra, India

Preamble to the description:
The following specification particularly describes the invention and the manner in which it is to be performed.
CROSS-REFERENCE TO RELATED APPLICATIONS AND PRIORITY
[001] The present application is a patent of addition of Indian Patent Application No. 202021025764, having priority date June 18, 2020, the entire content of which is hereby incorporated herein by way of reference.

TECHNICAL FIELD
[002] The disclosure herein generally relates to automated assessment of mental workload, and, more particularly, to a method and a system for assessing mental workload using differential dermal potential recorded from ear.

BACKGROUND
[003] Mental workload or cognitive load refers to the load imposed on working memory of a human while holding or processing meaningful information while engaged in a cognitive task. It is a well-known fact that the human working memory is limited in capacity and time. So, higher cognitive load indicates overloading of the human working memory that sometimes might lead to reduction in performance, increase in error rates which in turn may lead to some serious consequences in some mission critical use-cases. Thus, it becomes important to monitor and measure the working memory load for some real-life and mission critical applications.
[004] Various state of the art techniques have used physiological signals like brain signals, heart rate and heart rate variability, respiratory signal and the like for continuous assessment of mental workload in real life applications. Few state of the art techniques have widely relied on electrodermal activity (EDA) for measuring cognitive load as EDA measures continuous variations occurring in characteristics of human skin due to sweat gland activity. In particular, arousing of a sympathetic branch of an autonomic nervous system (ANS) in response to any stimulus, increases activity of the sweat gland, which in turn increases skin conductance. So, these changes occurring in the skin of a human being are measured using methods such as skin conductance response (SCR), skin potential (SP), skin conductance level (SCL), and skin potential response (SPR) for assessing mental workload as these changes are completely modulated by ANS which also controls human behavior, cognition and emotional states on a subconscious level. Hence, research involving EDA has been applied widely in almost all areas of psychology, psychiatry and psychophysiology.
[005] Further, few state of the art techniques have used exosomatic approach like Galvanic Skin Resistance (GSR) for measuring EDA which further helps in assessment of cognitive load experienced by a subject, such as human. The exosomatic approach measure skin conductance and resistance between two electrodes by passing a very weak Alternating Current (AC)/Direct current (DC). In particular, electrodes are typically attached on active sites of palmar and plantar surfaces for measuring skin conductance and resistance. However, exosomatic approach does not perform well for individuals having excessively dry or sweaty hands.
[006] Additionally, few state of the art techniques have used endosomatic or passive methods in which voltage is measured between an active site and a comparatively inactive site on a palmar and plantar skin surface. Although, endosomatic approaches are considered to be more physical, the signals obtained are difficult to analyze and interpret, thus accuracy of the measurement cannot be guaranteed.
[007] The concerns and limitations of convention art and approaches are addressed in Applicant’s Indian patent applicant No. 202021025764, filed on 09 January 2021 by providing assessment of cognitive load using bio-potential signals to implement endosomatic approach. The Indian patent applicant No. 202021025764 utilizes a multichannel wearable endosomatic device capable of acquiring and combining multiple bio-potentials, which are biomarkers of cognitive load experienced by a subject performing a cognitive task. Thereafter, the extracted information is utilized for classification of the cognitive load, from the acquired bio-signals using a set of statistical and a set of spectral features. Furthermore, a feature selection approach is utilized to identify a set of optimum features to train a Machine Learning (ML) based task classifier to classify the cognitive load experienced by a subject into high load task and low load task. However, further additions and refinements to the approach are not discussed that can improve the accuracy of classification without causing discomfort to the subject by eliminating the need to wear the recording device for long time at places which are prone to movements, such a hands and legs.

SUMMARY
[008] Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems. For example, in one embodiment, a method for assessing mental workload using differential dermal potential (DDP) recorded from ear is provided. The method includes receiving, by a mental workload assessment system (MWAS) via one or more hardware processors, a DDP data associated with a plurality of subjects involved in a cognitive task for one or more sessions, wherein the DDP data associated with each subject is obtained from at least one of: one or more ear canals, and one or more ear lobes of a respective subject, wherein the DDP data obtained from the ear canal comprises a plurality of channel data, and wherein the DDP data received for each of the plurality of sessions of each subject is timestamped; performing, for the DDP data received for each session of one or more sessions performed by each subject of the plurality of subjects: extracting, by the MWAS via the one or more hardware processors, a workable data from the DDP data based on timestamps; preprocessing, by the MWAS via the one or more hardware processors, the workable data using one or more preprocessing techniques to obtain a pre-processed workable data for a respective session; extracting, by the MWAS via the one or more hardware processors, a baseline data and a trial data from the pre-processed workable data of the respective session based on the time stamps using a predefined DDP data categorization technique; segmenting, by the MWAS via the one or more hardware processors, the baseline data into a plurality of baseline time windows and the trial data into a plurality of trial time windows based on a predefined time division criteria; computing, by the MWAS via the one or more hardware processors, a plurality of statistical time domain features for each baseline time window of the plurality of baseline time windows based on the baseline data present in a respective baseline time window, and for each trial time window of the plurality of trial time windows based on the trial data present in a respective trial time window; selecting, by the MWAS via the one or more hardware processors, one or more statistical time domain features from the plurality of statistical time domain features for each baseline time window and for each trial time window using a maximal information coefficient technique; identifying, by the MWAS via the one or more hardware processors, the selected one or more time domain features for each baseline time window as baseline features, and the selected one or more time domain features for each trial time window as trial features; and training, by the MWAS via the one or more hardware processors, a cognitive load prediction model to classify cognitive load of a subject into one of: a rest and a load based on the baseline features and the trial features, respectively.
[009] In another aspect, a mental workload assessment system for assessing mental workload using differential dermal potential (DDP) recorded from ear is provided. The mental workload assessment system includes a memory storing instructions; one or more communication interfaces; and one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to: receive a DDP data associated with a plurality of subjects involved in a cognitive task for one or more sessions, wherein the DDP data associated with each subject is obtained from at least one of: one or more ear canals, and one or more ear lobes of a respective subject, wherein the DDP data obtained from the ear canal comprises a plurality of channel data, and wherein the DDP data received for each of the plurality of sessions of each subject is timestamped; perform, for the DDP data received for each session of one or more sessions performed by each subject of the plurality of subjects: extract a workable data from the DDP data based on timestamps; preprocess the workable data using one or more preprocessing techniques to obtain a pre-processed workable data for a respective session; extract a baseline data and a trial data from the pre-processed workable data of the respective session based on the time stamps using a predefined DDP data categorization technique; segment the baseline data into a plurality of baseline time windows and the trial data into a plurality of trial time windows based on a predefined time division criteria; compute a plurality of statistical time domain features for each baseline time window of the plurality of baseline time windows based on the baseline data present in a respective baseline time window, and for each trial time window of the plurality of trial time windows based on the trial data present in a respective trial time window; select one or more statistical time domain features from the plurality of statistical time domain features for each baseline time window and for each trial time window using a maximal information coefficient technique; identify the selected one or more time domain features for each baseline time window as baseline features, and the selected one or more time domain features for each trial time window as trial features; and train a cognitive load prediction model to classify cognitive load of a subject into one of: a rest and a load based on the baseline features and the trial features, respectively.
[010] In yet another aspect, a non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause a method for assessing mental workload using differential dermal potential (DDP) recorded from ear is provided. The method includes receiving, by a mental workload assessment system (MWAS) via one or more hardware processors, a DDP data associated with a plurality of subjects involved in a cognitive task for one or more sessions, wherein the DDP data associated with each subject is obtained from at least one of: one or more ear canals, and one or more ear lobes of a respective subject, wherein the DDP data obtained from the ear canal comprises a plurality of channel data, and wherein the DDP data received for each session of the plurality of sessions of each subject is timestamped; performing, for the DDP data received for each session of one or more sessions performed by each subject of the plurality of subjects: extracting, by a MWAS, a workable data from the DDP data based on timestamps; preprocessing, by the MWAS, the workable data using one or more preprocessing techniques to obtain a pre-processed workable data for a respective session; extracting, by the MWAS, a baseline data and a trial data from the pre-processed workable data of the respective session based on the time stamps using a predefined DDP data categorization technique; segmenting, by the MWAS, the baseline data into a plurality of baseline time windows and the trial data into a plurality of trial time windows based on a predefined time division criteria; computing, by the MWAS, a plurality of statistical time domain features for each baseline time window of the plurality of baseline time windows based on the baseline data present in a respective baseline time window, and for each trial time window of the plurality of trial time windows based on the trial data present in a respective trial time window; selecting, by the MWAS, one or more statistical time domain features from the plurality of statistical time domain features for each baseline time window and for each trial time window using a maximal information coefficient technique; identifying, by the MWAS, the selected one or more time domain features for each baseline time window as baseline features, and the selected one or more time domain features for each trial time window as trial features; and training, by the MWAS, a cognitive load prediction model to classify cognitive load of a subject into one of: a rest and a load based on the baseline features and the trial features, respectively.
[011] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS
[012] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles:
[013] FIG. 1 is an example representation of an environment, related to at least some example embodiments of the present disclosure.
[014] FIG. 2 illustrates an exemplary block diagram of a system for assessing mental workload using differential dermal potential (DDP) recorded from ear, in accordance with an embodiment of the present disclosure.
[015] FIG. 3 illustrates a schematic block diagram representation of a process associated with the system of FIGS. 1 and 2 for training a cognitive load prediction model for classifying cognitive load of a subject, in accordance with some embodiments of the present disclosure.
[016] FIGS. 4A and 4B, with reference to FIGS. 1 through 3, collectively, represent an exemplary flow diagram of a method for assessing mental workload using DDP recorded from the ear of a subject, in accordance with some embodiments of the present disclosure.
[017] FIG. 5, with reference to FIGS, 1-4, illustrates an example representation of a session timeline, in accordance with some embodiments of the present disclosure.
[018] FIG. 6, with reference to FIGS, 1-5, illustrates a schematic representation showing placement of electrodes in an ear canal for capturing DDP data associated with a subject performing mental task, in accordance with some embodiments of the present disclosure.
[019] FIG. 7A, with reference to FIGS, 1-6, illustrates a schematic representation of a close clip like structure used for capturing DDP data associated with a subject performing mental task, in accordance with some embodiments of the present disclosure.
[020] FIG. 7B, with reference to FIGS, 1-6, illustrates a representation of a subject wearing the close clip like structure, in accordance with some embodiments of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS
[021] Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the scope of the disclosed embodiments.
[022] According to cognitive load theory, human working memory is limited in capacity and time. So, as per theory, if human try to process too much information at once, they start facing trouble in performing tasks and learning new things which further leads to less productivity, increase in error rates, etc. In particular, performance is hampered which might have some serious consequences for job positions requiring a lot of engagement and responsibility, e.g., air-traffic controllers or for people working on critical missions. Thus, it becomes important to measure mental workload or fatigue.
[023] As discussed earlier, some available techniques for assessing mental workload use physiological signals like brain signals, heart rate and heart rate variability, respiratory signa etc. Some have used electrodermal activity (EDA) measured using exosomatic and endosomatic approaches for measuring cognitive workload. However, exosomatic approaches requires external electrical current to be passed across the electrodes which makes it uncomfortable for the user, and endosomatic approaches generate signals that are difficult to analyze and interpret. Additionally, recording approaches used are obtrusive in nature and hence are not always suitable for continuous and long term monitoring in real life applications.
[024] As discussed earlier, the applicant has addressed concerns and limitations in the art in applicants Indian patent applicant No. 202021025764, filed on 09 January 2021 by providing assessment of cognitive load using bio-potential signals to implement endosomatic approach. However, in the application, the biopotentials are acquired from hands and legs which are very prone to movement artifacts.
[025] Embodiments of the present disclosure overcome the above-mentioned disadvantages by providing a method and a system for assessing mental workload using differential dermal potential (DDP) recorded from in and around ear. It should be noted that the embodiments of the present disclosure provide further additions and refinements to the approach that are not discussed in Indian patent applicant No. 202021025764 for assessment of cognitive load using bio-potentials.
[026] The system of the present disclosure uses DDP recorded from in and around ear for assessing mental workload of a subject involved in cognitive task. As DDP is a non-invasive bio-potential signal acquired from the skin surface, the requirement of applying external electrical current for measurement is eliminated. Further, the system ensures accurate assessment of mental workload as the DDP used for calculation is recorded from ears which are not prone to movements. The disclosed system is further explained with the method as described in conjunction with FIG.1 to FIG. 7B below.
[027] Referring now to the drawings, and more particularly to FIG. 1 through 7B, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and/or method.
[028] FIG. 1 illustrates an exemplary representation of an environment 100 related to at least some example embodiments of the present disclosure. Although the environment 100 is presented in one arrangement, other embodiments may include the parts of the environment 100 (or other parts) arranged otherwise depending on, for example, extracting workable data, segmenting workable data into plurality of time windows, etc. The environment 100 generally includes a mental workload assessment system (hereinafter referred as ‘MWAS’) 102, and one or more electronic devices, such as an electronic device 106, each coupled to, and in communication with (and/or with access to) a network 104. It should be noted that one electronic device is shown for the sake of explanation; there can be more number of electronic devices.
[029] The network 104 may include, without limitation, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, a virtual network, and/or another suitable public and/or private network capable of supporting communication among two or more of the parts or users illustrated in FIG. 1, or any combination thereof.
[030] Various entities in the environment 100 may connect to the network 104 in accordance with various wired and wireless communication protocols, such as Transmission Control Protocol and Internet Protocol (TCP/IP), User Datagram Protocol (UDP), 2nd Generation (2G), 3rd Generation (3G), 4th Generation (4G), 5th Generation (5G) communication protocols, Long Term Evolution (LTE) communication protocols, or any combination thereof.
[031] The electronic device 106 is associated with a subject (e.g., a user of the MWAS) who is performing mental workload assessment of a plurality of subjects involved in a cognitive task. Examples of the electronic device 106 include, but are not limited to, a personal computer (PC), a mobile phone, a tablet device, a Personal Digital Assistant (PDA), a server, a voice activated assistant, a smartphone, and a laptop.
[032] The mental workload assessment system (hereinafter referred as ‘MWAS’) 102 includes one or more hardware processors and a memory. The MWAS 102 is first configured to receive a DDP data associated with a plurality of subjects involved in a cognitive task for one or more sessions via the network 104 from the user device 106. The DDP data associated with each subject is obtained from at least one of one or more ear canals, and one or more ear lobes of a respective subject. It should be noted that the DDP data received for each session of each subject is timestamped.
[033] In an embodiment, three-dimensional (3D) printed customized earpieces are used to collect DDP data from the one or more ear canals of each subject. In an embodiment, each 3D printed customized earpiece includes a plurality of electrodes for recording DDP signals from a plurality of locations of an ear canal. In at least one example embodiment, 8 electrodes are used for recording DDP data from 8 different locations of the ear canal. A schematic representation showing placement of the electrodes in the ear canal for capturing the DDP data is shown with reference to FIG. 6. It should be noted that the DDP data obtained from the ear canal includes a plurality of channel data as the plurality of electrodes are used for recording.
[034] In an embodiment, a preexisting device is used for collecting the DDP data from the one or more ear lobes of each subject. In one embodiment, a new set of electrodes are designed and integrated in the preexisting device. Further, to hold the electrodes, a close clip like structure (shown with reference to FIG. 7A) is used. The close clip like structure makes it easy for a subject to wear the electrodes. An example representation of a subject wearing the preexisting device along with the close clip like structure is shown with reference to FIG. 7B.
[035] Thereafter, for the DDP data received for each session of one or more sessions performed by each subject of the plurality of subjects, the MWAS 102 performs sanity check and data extraction. Further, the MWAS 102 performs pre-processing of the extracted data. Once the pre-processed data is available, the MWAS 102 performs feature generation in which a plurality of statistical time domain features are computed based on the pre-processed data. The MWAS 102 then performs feature selection on the plurality of statistical time domain features to obtain one or more statistical time domain features which are then used to train a cognitive load prediction model. The cognitive load prediction model, once trained, can classify cognitive load of a subject into one of a rest condition and a load condition. The pre-processing, the feature generation and the feature selection part is explained in detail with reference to FIGS. 3 and 4A-4B.
[036] The number and arrangement of systems, devices, and/or networks shown in FIG. 1 are provided as an example. There may be additional systems, devices, and/or networks; fewer systems, devices, and/or networks; different systems, devices, and/or networks; and/or differently arranged systems, devices, and/or networks than those shown in FIG. 1. Furthermore, two or more systems or devices shown in FIG. 1 may be implemented within a single system or device, or a single system or device shown in FIG. 1 may be implemented as multiple, distributed systems or devices. Additionally, or alternatively, a set of systems (e.g., one or more systems) or a set of devices (e.g., one or more devices) of the environment 100 may perform one or more functions described as being performed by another set of systems or another set of devices of the environment 100 (e.g., refer scenarios described above).
[037] FIG. 2 illustrates an exemplary block diagram of a mental workload assessment system 102 for assessing mental workload using differential dermal potential (DDP) recorded from ear, in accordance with an embodiment of the present disclosure. In an embodiment, the mental workload assessment system 102 may also be referred as system 102 and may be interchangeably used herein. In some embodiments, the system 102 is embodied as a cloud-based and/or SaaS-based (software as a service) architecture. In some embodiments, the system 102 may be implemented in a server system. In some embodiments, the system 102 may be implemented in a variety of computing systems, such as laptop computers, notebooks, hand-held devices, workstations, mainframe computers, and the like.
[038] In an embodiment, the system 102 includes one or more processors 204, communication interface device(s) or input/output (I/O) interface(s) 206, and one or more data storage devices or memory 202 operatively coupled to the one or more processors 204. The one or more processors 204 may be one or more software processing modules and/or hardware processors. In an embodiment, the hardware processors can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor(s) is configured to fetch and execute computer-readable instructions stored in the memory. In an embodiment, the system 102 can be implemented in a variety of computing systems, such as laptop computers, notebooks, hand-held devices, workstations, mainframe computers, servers, a network cloud and the like.
[039] The I/O interface device(s) 206 can include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like and can facilitate multiple communications within a wide variety of networks N/W and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. In an embodiment, the I/O interface device(s) can include one or more ports for connecting a number of devices to one another or to another server.
[040] The memory 202 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. In an embodiment a database 208 can be stored in the memory 202, wherein the database 208 may comprise, but are not limited to, DDP data associated with each subject, statistical time domain features selected for each subject, and the like. The memory 202 further comprises (or may further comprise) information pertaining to input(s)/output(s) of each step performed by the systems and methods of the present disclosure. In other words, input(s) fed at each step and output(s) generated at each step are comprised in the memory 202 and can be utilized in further processing and analysis.
[041] It is noted that the system 102 as illustrated and hereinafter described is merely illustrative of an apparatus that could benefit from embodiments of the present disclosure and, therefore, should not be taken to limit the scope of the present disclosure. It is noted that the system 102 may include fewer or more components than those depicted in FIG. 2.
[042] FIG. 3, with reference to FIGS, 1-2, illustrates a schematic block diagram representation 300 of a process associated with the system 102 of FIGS. 1 and 2 for training a cognitive load prediction model for classifying cognitive load of a subject, in accordance with an embodiment of the present disclosure.
[043] As seen in FIG.3, the system 102 first receives the DDP data associated with a plurality of subjects involved in a cognitive task for one or more sessions. In an embodiment, the DDP data associated with each subject is obtained from one or more ear canals (also referred as EC-DS), and one or more ear lobes (also referred as EL-DS) of a respective subject.
[044] As already discussed, the DDP data received for each session of each subject is timestamped. So, a sanity check is performed on the DDP data received for each subject to ensure that a complete sensor data is available for a given timeline. In particular, length of DDP data files are cross checked in terms of sampling frequency and duration of the cognitive task. The DDP data which is found to be non-complaint is discarded at this stage.
[045] Thereafter, workable data is extracted from the DDP data received for each session of each subject based on timestamps. The above sentence can be better understood by the way of following description.
[046] In an embodiment, the DDP data of each session performed by a subject includes a rest period data, a baseline period data and a trial data. In at least one example embodiment, the trial data further includes the DDP data of 3 trials in which each trial started with a 45 seconds baseline period, followed by a trial of a given cognitive workload. Further, after each trial, the subjects is instructed to verbally tell the answer within 5 secs. Additionally, a preparation time of 5 seconds is given to get ready for the next trial. An example representation of a session timeline is shown with respect to FIG. 5.
[047] In particular, the rest period data is ignored from the DDP data and remaining data consisting of the baseline period data and the trial data is extracted as the workable data. Then, the workable data is pre-processed using one or more preprocessing techniques, such as down-sampling, filtering, normalization and channel selection to obtain a pre-processed workable data for a respective session.
[048] Further, a plurality of statistical time domain features are computed based on the pre-processed workable data during a feature engineering phase. In an embodiment, the plurality of statistical time domain features includes one or more of mean, median, standard deviation, skewness, kurtosis, minimum, maximum, area under a signal curve, zero crossing instant, instantaneous slope of a signal, signal moments and signal entropy.
[049] Once the statistical time domain features are available, a feature selection is performed in which one or more statistical time domain features are selected from the plurality of statistical time domain features using a maximal information coefficient (MIC) technique. The one or more statistical time domain features are then used to train the cognitive load prediction model that, once trained, can predict whether the subject is at rest or at load i.e. having higher mental workload. In particular, the cognitive load prediction model is trained and tested to obtain a feature subset which gives the maximum accuracy using a Leave one subject out (LOSO) validation.
[050] Additionally, a cross dataset validation is performed to analyze co-occurrence of the DDP signals recorded from two recording sites i.e., ear canal and ear lobe of a subject.
[051] FIGS. 4A and 4B, with reference to FIGS. 1 through 3, collectively, represent an exemplary flow diagram of a method 400 for assessing mental workload using differential dermal potential (DDP) recorded from ear of a subject, in accordance with an embodiment of the present disclosure. The method 400 may use the system 102 of FIGS. 1 and 2 for execution. In an embodiment, the system 102 comprises one or more data storage devices or the memory 202 operatively coupled to the one or more hardware processors 204 and is configured to store instructions for execution of steps of the method 400 by the one or more hardware processors 204. The sequence of steps of the flow diagram may not be necessarily executed in the same order as they are presented. Further, one or more steps may be grouped together and performed in form of a single step, or one step may have several sub-steps that may be performed in parallel or in sequential manner. The steps of the method of the present disclosure will now be explained with reference to the components of the system 102 of FIG. 1.
[052] At step 402 of the method of the present disclosure, the one or more hardware processors 204 of the system 102 receive a DDP data associated with a plurality of subjects involved in a cognitive task for one or more sessions. The DDP data associated with each subject is obtained from at least one of one or more ear canals, and one or more ear lobes of a respective subject. In an embodiment, the DDP data obtained from the ear canal includes a plurality of channel data. The DDP data received for each session of each subject is timestamped.
[053] At step 404 of the method of the present disclosure, the one or more hardware processors 204 of the system 102 perform a plurality of steps 404a through 404h for the DDP data received for each session performed by each subject so that a cognitive load prediction model can be trained.
[054] More specifically, at step 404a of the present disclosure, the one or more hardware processors 204 of the system 102 extract a workable data from the DDP data based on timestamps.
[055] At step 404b of the present disclosure, the one or more hardware processors 204 of the system 200 preprocess the workable data using one or more preprocessing techniques to obtain a pre-processed workable data for a respective session. The above step can be better understood by the way of following description.
[056] As the devices used for collecting the DDP data from ear canal and ear lobe of a subject are different. The collected DDP data includes different sampling rates. So, to ensure uniformity in sampling rate for unbiased analysis, the down-sampling of the workable data is performed to obtain a down-sampled workable data using a down-sampling technique. It should be noted that the down sampling technique can be any down-sampling technique known in the art. In particular, the sampling rate of the DDP datasets are matched during the down-sampling.
[057] Then, the down-sampled workable data is filtered using a low-pass filter to obtain a filtered workable data. In an embodiment, a 2nd order Butterworth low pass filter having a cut-of frequency of 1 Hz is applied to remove high frequency noisy components present in the down-sampled workable data to obtain the filtered workable data.
[058] Once the filtered workable data is available, the system 200 performs normalization of the filtered workable data using a normalization technique to obtain a normalized workable data. In particular, a z-score normalization is performed on the filtered workable data to obtain the normalized workable data.
[059] Additionally, as the DDP data obtained from the ear canal of a subject includes data from the plurality of channels whereas the DDP data obtained from the ear lobe includes data from two channels. So, to bring parity between the DDP data obtained from two different sources, a channel selection is performed on the normalized workable data using a channel selection technique to convert the plurality of channel data present in the normalized workable data into one or more channel data. So, to perform channel selection, first the system 102 determines whether the normalized workable data includes the plurality of channel data i.e., whether it includes the DDP data obtained from the ear canal. In case the normalized workable data includes the plurality of channel data, the system 200 performs channel selection using the channel selection technique.
[060] In an embodiment, the channel selection technique includes computing signal band-power for all the plurality of channels provided in the DDP dataset obtained from the one or more ear canals. Then, for each ear canal, a channel with maximum band-power is selected among the plurality of channels for further analysis. In particular, a single channel is selected for each ear canal of the subject.
[061] Finally, the normalized workable data obtained after channel selection is identified as the pre-processed workable data.
[062] In an embodiment, derived signals are generated using combination of both the channels. In particular, the DDP data for both channels (left and right ear) is taken to compute the derived signals, such as paired sum that is a sum of both channels, gap signal that is a difference of both channels and a ratio signal that is a ratio of both channels.
[063] At step 404c of the present disclosure, the one or more hardware processors 204 of the system 102 extract a baseline data and a trial data from the pre-processed workable data of the respective session based on the time stamps using a predefined DDP data categorization technique. In an embodiment, the DDP data categorization technique includes segmenting the first 45 seconds of the DDP data as the baseline data and remaining data is then used to compute the trial data. As discussed with reference to FIG. 3, the DDP data includes 3 trials data, and each trial data is of 45 seconds. So, using the same session timeline, the trail data is segmented from the remaining data.
[064] At step 404d of the present disclosure, the one or more hardware processors 204 of the system 102 segment the baseline data into a plurality of baseline time windows and the trial data into a plurality of trial time windows based on a predefined time division criteria. In particular, the baseline data is divided into plurality of baseline time windows using a particular window size as described in the predefined time division criteria. In an example embodiment, if the window size of 9 seconds is described in the time division criteria, then baseline data is divided into 5 baseline time windows of 9 seconds each. Similarly, the trial data is divided into the plurality of trial time windows based on the window size described in the time division criteria.
[065] At step 404e of the present disclosure, the one or more hardware processors 204 of the system 102 compute a plurality of statistical time domain features for each baseline time window of the plurality of baseline time windows based on the baseline data present in a respective baseline time window, and for each trial time window of the plurality of trial time windows based on the trial data present in a respective trial time window. The plurality of statistical time domain features comprises one or more of: mean, median, standard deviation, skewness, kurtosis, minimum, maximum, area under a signal curve, zero crossing instant, instantaneous slope of a signal, signal moments and signal entropy.
[066] In particular, for each 9 second baseline time window, a set of statistical time domain features are computed using the DDP data present in that particular baseline time window. In at least one example embodiment, a set of 81 features are computed from each baseline time window as well as each trial time window.
[067] In at least one example embodiment, the statistical time domain features are computed for each of the derived signals.
[068] In an embodiment, as baseline time window is of 45 seconds and the trial time window is of 45*3 i.e., 135 seconds, there is a class imbalance in terms of statistical time domain features. So, a synthetic minority over-sampling technique (SMOTE) is applied on the plurality of statistical time domain features of each of the baseline time window and the trial time window to obtain equal number of statistical time domain features for each of the baseline time window and the trial time window.
[069] At step 404f of the present disclosure, the one or more hardware processors 204 of the system 102 select one or more statistical time domain features from the plurality of statistical time domain features for each baseline time window and for each trial time window using a maximal information coefficient (MIC) technique. In particular, the MIC technique helps in selection of relevant features which show a strong relationship with ground truth labels. In an embodiment, the MIC technique includes giving score to each statistical time domain features of the plurality of statistical time domain features computed for each of the baseline time window and the trial time window which further help in deciding which one or more statistical time domain features to select from the plurality of statistical time domain features from each of the baseline time window and the trial time window. In particular, more the score, more the chances of getting selected.
[070] At step 404g of the present disclosure, the one or more hardware processors 204 of the system 102 identify the selected one or more statistical time domain features for each baseline time window as baseline features, and the selected one or more statistical time domain features for each trial time window as trial features.
[071] At step 404h of the present disclosure, the one or more hardware processors 204 of the system 102 train a cognitive load prediction model to classify cognitive load of a subject into one of: a rest and a load based on the baseline features and the trial features, respectively. In particular, the baseline features are used to train the cognitive load prediction model for identifying rest condition. Similarly, the trial features are used to train the cognitive load prediction model for identifying load conditions.
[072] In an embodiment, once the cognitive load prediction model is trained, the system 102, upon receiving a real-time DDP data associated with a real subject performing a real cognitive task, classifies a cognitive load of the real subject for the real cognitive task into one of rest and load based on the received real-time DDP data using the trained cognitive load prediction model. The system then displays the cognitive load of the real subject for the real cognitive task on a user device, such as the user device 106 via the network 104.
[073] FIG. 5, with reference to FIGS, 1-4, illustrates an example representation of a session timeline, in accordance with an embodiment of the present disclosure.
[074] FIG. 6, with reference to FIGS, 1-5, illustrates a schematic representation showing placement of electrodes in an ear canal for capturing DDP data associated with a subject performing mental task, in accordance with an embodiment of the present disclosure.
[075] As seen in FIG. 6, 4 electrodes are placed in a concha region of an ear and 4 electrodes are placed inside the ear canal. The electrodes are connected to eight channel OpenBCI Cyton board 1, which is capable to recording data at 250 Hz sampling rate via a Bluetooth receiver. An open source software present on the user device 106 is then used to control the board, collect data and save in a CSV maintained on the user device 106.
[076] FIG. 7A, with reference to FIGS, 1-5, illustrates a schematic representation of a close clip like structure used for capturing DDP data associated with a subject performing mental task, in accordance with an embodiment of the present disclosure.
[077] The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims and may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope of the claims if they have similar elements that do not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language of the claims.
[078] The system of the present disclosure herein addresses unresolved problem of assessing mental workload without applying any external electric current across the electrodes. The embodiment thus provides a system and a method for assessing mental workload using DDP obtained from ears of a subject. Moreover, the embodiments herein further provides solution to the problem of capturing signals from body parts which are prone to movement. The system and the method uses DDP signals collected from in and around ears which is less prone to movement, thereby improving the accuracy of the assessment.
[079] It is to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g., any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g., hardware means like e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g., an ASIC and an FPGA, or at least one microprocessor and at least one memory with software processing components located therein. Thus, the means can include both hardware means and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments may be implemented on different hardware devices, e.g., using a plurality of CPUs.
[080] The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various components described herein may be implemented in other components or combinations of other components. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
[081] The illustrated steps are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope of the disclosed embodiments. Also, the words “comprising,” “having,” “containing,” and “including,” and other similar forms are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.
[082] Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.
[083] It is intended that the disclosure and examples be considered as exemplary only, with a true scope of disclosed embodiments being indicated by the following claims.
, Claims:We Claim:
1. A processor implemented method for assessing mental workload (400), comprising:
receiving (402), by a mental workload assessment system (MWAS) via one or more hardware processors, a DDP data associated with a plurality of subjects involved in a cognitive task for one or more sessions, wherein the DDP data associated with each subject is obtained from at least one of: one or more ear canals, and one or more ear lobes of a respective subject, wherein the DDP data obtained from the ear canal comprises a plurality of channel data, and wherein the DDP data received for each of the one or more sessions of each subject is timestamped; and
performing (404), for the DDP data received for each session of one or more sessions performed by each subject of the plurality of subjects:
extracting (404a), by the MWAS via the one or more hardware processors, a workable data from the DDP data based on timestamps;
preprocessing (404b), by the MWAS via the one or more hardware processors, the workable data using one or more preprocessing techniques to obtain a pre-processed workable data for a respective session;
extracting (404c), by the MWAS via the one or more hardware processors, a baseline data and a trial data from the pre-processed workable data of the respective session based on the time stamps using a predefined DDP data categorization technique;
segmenting (404d), by the MWAS via the one or more hardware processors, the baseline data into a plurality of baseline time windows and the trial data into a plurality of trial time windows based on a predefined time division criteria;
computing (404e), by the MWAS via the one or more hardware processors, a plurality of statistical time domain features for each baseline time window of the plurality of baseline time windows based on the baseline data present in a respective baseline time window, and for each trial time window of the plurality of trial time windows based on the trial data present in a respective trial time window;
selecting (404f), by the MWAS via the one or more hardware processors, one or more statistical time domain features from the plurality of statistical time domain features for each baseline time window and for each trial time window using a maximal information coefficient technique;
identifying (404g), by the MWAS via the one or more hardware processors, the selected one or more statistical time domain features for each baseline time window as baseline features, and the selected one or more statistical time domain features for each trial time window as trial features; and
training (404h), by the MWAS via the one or more hardware processors, a cognitive load prediction model to classify cognitive load of a subject into one of: a rest and a load based on the baseline features and the trial features, respectively.

2. The method (400) as claimed in claim 1, wherein the step of processing, by the MWAS via the one or more hardware processors, the workable data using one or more preprocessing techniques to obtain a pre-processed workable data for the respective session comprises:
down-sampling the workable data to obtain a down-sampled workable data using a down sampling technique;
filtering the down-sampled workable data using a low-pass filter to obtain a filtered workable data;
performing normalization of the filtered workable data using a normalization technique to obtain a normalized workable data;
determining whether the normalized workable data comprises the plurality of channel data;
based on determination, performing channel selection on the normalized workable data using a channel selection technique to convert the plurality of channel data present in the normalized workable data into one or more channel data; and
identifying the normalized workable data obtained after channel selection as the pre-processed workable data.

3. The method (400) as claimed in claim 1, wherein the plurality of statistical time domain features comprises one or more of: mean, median, standard deviation, skewness, kurtosis, minimum, maximum, area under a signal curve, zero crossing instant, instantaneous slope of a signal, signal moments, and signal entropy.

4. The method (400) as claimed in claim 1, wherein the step of selecting, by the MWAS via the one or more hardware processors, the one or more statistical time domain features from the plurality of statistical time domain features for each baseline time window and for each trial time window using the maximal information coefficient technique is preceded by:
applying a synthetic minority over-sampling technique (SMOTE) on the plurality of statistical time domain features of each of the baseline time window and the trial time window to obtain equal number of statistical time domain features for each of the baseline time window and the trial time window.

5. The method (400) as claimed in claim 1, comprising:
receiving, by the MWAS via the one or more hardware processors, a real-time DDP data associated with a real subject performing a real cognitive task;
classifying, by the MWAS via the one or more hardware processors, a cognitive load of the real subject for the real cognitive task into one of rest and load based on the received real-time DDP data using the trained cognitive load prediction model; and
displaying, by the MWAS via the one or more hardware processors, the cognitive load of the real subject for the real cognitive task on a user device.

6. A mental workload assessment system (102), comprising:
a memory (202) storing instructions;
one or more communication interfaces (206); and
one or more hardware processors (204) coupled to the memory (202) via the one or more communication interfaces (206), wherein the one or more hardware processors (204) are configured by the instructions to:
receive a DDP data associated with a plurality of subjects involved in a cognitive task for one or more sessions, wherein the DDP data associated with each subject is obtained from at least one of: one or more ear canals, and one or more ear lobes of a respective subject, wherein the DDP data obtained from the ear canal comprises a plurality of channel data, and wherein the DDP data received for each of the one or more sessions of each subject is timestamped; and
performing, for the DDP data received for each session of one or more sessions performed by each subject of the plurality of subjects:
extract a workable data from the DDP data based on timestamps;
preprocess the workable data using one or more preprocessing techniques to obtain a pre-processed workable data for a respective session;
extract a baseline data and a trial data from the pre-processed workable data of the respective session based on the time stamps using a predefined DDP data categorization technique;
segment the baseline data into a plurality of baseline time windows and the trial data into a plurality of trial time windows based on a predefined time division criteria;
compute a plurality of statistical time domain features for each baseline time window of the plurality of baseline time windows based on the baseline data present in a respective baseline time window, and for each trial time window of the plurality of trial time windows based on the trial data present in a respective trial time window;
select one or more statistical time domain features from the plurality of statistical time domain features for each baseline time window and for each trial time window using a maximal information coefficient technique;
identify the selected one or more time domain features for each baseline time window as baseline features, and the selected one or more time domain features for each trial time window as trial features; and
train a cognitive load prediction model to classify cognitive load of a subject into one of: a rest and a load based on the baseline features and the trial features, respectively.

7. The mental workload assessment system (102) as claimed in claim 6, wherein for processing the workable data using one or more preprocessing techniques to obtain the pre-processed workable data for the respective session, the one or more hardware processors (204) are configured by the instructions to:
down-sample the workable data to obtain a down-sampled workable data using a down sampling technique;
filter the down-sampled workable data using a low-pass filter to obtain a filtered workable data;
perform normalization of the filtered workable data using a normalization technique to obtain a normalized workable data;
determine whether the normalized workable data comprises the plurality of channel data;
based on determination, perform channel selection on the normalized workable data using a channel selection technique to convert the plurality of channel data present in the normalized workable data into one or more channel data; and
identify the normalized workable data obtained after channel selection as the pre-processed workable data.

8. The mental workload assessment system (102) as claimed in claim 6, wherein the plurality of statistical time domain features comprises one or more of: mean, median, standard deviation, skewness, kurtosis, minimum, maximum, area under a signal curve, zero crossing instant, instantaneous slope of a signal, signal moments and signal entropy.

9. The mental workload assessment system (102) as claimed in claim 6, wherein prior to selecting the one or more statistical time domain features from the plurality of statistical time domain features for each baseline time window and for each trial time window using the maximal information coefficient technique, the one or more hardware processors (204) are configured by the instructions to:
apply a synthetic minority over-sampling technique (SMOTE) on the plurality of statistical time domain features of each of the baseline time window and the trial time window to obtain equal number of statistical time domain features for each of the baseline time window and the trial time window.

10. The mental workload assessment system (102) as claimed in claim 6, wherein the one or more hardware processors (204) are configured by the instructions to:
receive a real-time DDP data associated with a real subject performing a real cognitive task;
classify a cognitive load of the real subject for the real cognitive task into one of rest and load based on the received real-time DDP data using the trained cognitive load prediction model; and
display the cognitive load of the real subject for the real cognitive task on a user device.

Dated this 23rd Day of June 2023

Tata Consultancy Services Limited
By their Agent & Attorney

(Adheesh Nargolkar)
of Khaitan & Co
Reg No IN-PA-1086