Claims:1. A method (200) for determining cognitive load and confidence level of a person, the method comprising a processor implemented steps of:

providing a stimulus to the person using a display screen (102) present in front of the person, wherein stimulus results in generation of spontaneous pupillary fluctuations (SPF) (202);
capturing the spontaneous pupillary fluctuations as a pupillary signal of the person in response to the stimulus using an eye tracker (104) placed in front of the person (204);
decomposing the captured pupillary signal using empirical mode decomposition (EMD) and ensemble EMD into a finite set of intrinsic mode functions (IMFs) (206);
decomposing the captured pupillary signal using variation mode decomposition (VMD) in to a plurality of VMD components (208);
isolating a source of interest of the captured pupillary signal using independent component analysis (ICA) (210);
reconstructing the isolated source of interest in the finite set of IMFs by multiplying the isolated source of interest with a mixing matrix (212);
reconstructing the pupillary signal using the finite set of IMFs and the plurality of VMD components (214);
extracting a plurality of features from the captured pupillary signal, the reconstructed pupillary signal and the plurality of VMD components, wherein the plurality of features include a relative mean-frequency power and a percentage change in pupil dilation (216);
detecting the cognitive load of the person using the plurality of features (218); and
determining the confidence level of the person using the plurality of features (220).

2. The method of claim 1 further comprising the step of evaluating the reconstructed pupillary signal by calculating a root mean squared error.

3. The method of claim 1 further comprising the step of determining the accuracy of detected cognitive load and confidence level of the person using Fscore.

4. The method of claim 1, wherein the step of providing stimulus includes providing an addition task and an anagram task.

5. The method of claim 4, wherein the stimulus is provided in order to achieve variations in the cognitive load and the confidence level.

6. The method of claim 1 further comprising the step of deriving the mixing matrix and an un-mixing matrix.

7. The method of claim 1 further comprising the calculation of a relative root mean squared error to evaluate the pupillary signal reconstruction.

8. The method of claim 1, wherein the finite set of intrinsic mode functions (IMFs) are 11 in number extracted from the spontaneous pupillary functions.

9. A system (100) for determining cognitive load and confidence level of a person, the system comprises

a display screen (102) present in front of the person to provide a stimulus, wherein stimulus results in generation of spontaneous pupillary fluctuations (SPF);
an eye tracker (104) present in front of the person configured to capture the spontaneous pupillary fluctuations as a pupillary signal of the person in response to the stimulus;
a memory (106); and
a processor (108) in communication with the memory, wherein the processor further comprises:
a decomposition module (110) configured to
decompose the captured pupillary signal using empirical mode decomposition (EMD) and ensemble EMD into a finite set of intrinsic mode functions (IMFs), and
decompose the captured pupillary signal using variation mode decomposition (VMD) in to a plurality of VMD components;
an isolation module (112) configured to isolate a source of interest of the captured pupillary signal using independent component analysis (ICA);
a first reconstruction module (114) configured to reconstruct the isolated source of interest in the finite set of IMFs by multiplying the isolated source of interest with a mixing matrix;
a second reconstruction module (116) configured to reconstruct the pupillary signal using the finite set of IMFs and the plurality of VMD components;
an extraction module (118) configured to extract a plurality of features from the captured pupillary signal, the reconstructed pupillary signal and the plurality of VMD components, wherein the plurality of features include a relative mean-frequency power and a percentage change in pupil dilation;
a cognitive load detection module (120) for detecting the cognitive load of the person using the plurality of features; and
a confidence level determining module (122) for determining the confidence level of the person using the plurality of features.
, Description:FORM 2

THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003

COMPLETE SPECIFICATION

(See Section 10 and Rule 13)

Title of invention:

METHOD AND SYSTEM FOR DETERMINING COGNITIVE LOAD AND CONFIDENCE OF A PERSON USING PUPILLARY RESPONSE

Applicant

Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th floor,
Nariman point, Mumbai 400021,
Maharashtra, India

The following specification particularly describes the invention and the manner in which it is to be performed.

TECHNICAL FIELD

The embodiments herein generally relates to the field of cognitive load measurement. More particularly, but not specifically, the invention provides a system and method for measuring cognitive load of a person using

BACKGROUND

The study of emotions in human-computer interaction has increased in recent years for variety of purposes. The study of emotions has led to a growing need for computer applications to detect cognitive load experienced by an individual. The cognitive load has been used in a variety of fields that deal with the human mind interacting with some external stimulants. Cognitive load can be defined as the mental effort required for a particular person to comprehend or learn some material, or complete some task. Cognitive load is relative to both the user and the task being completed, at any single point in time.
Cognitive load (CL) or mental workload indicates how an individual is able to manage his/her mental resources while performing a task. This is associated with the amount of memory resources used to accomplish the task. An optimum amount of CL need to be maintained to get best performance from an individual. Very low CL is basically under-utilization of one's abilities whereas very high CL results in mental stress, fatigue, etc. which ultimately degrades the overall performance. Thus the measurement of CL is very crucial.
Majority of the day-to-day decisions are associated with a sense of confidence. Even in the absence of explicit feedback, we possess an awareness of the goodness of the decisions made. Assessment of confidence is crucial as it is a major indicator of cognitive impairments like obsessive-compulsive disorder (OCD) and anxiety. A “checking" behavior is seen in subjects with reduced confidence level in their own memory. Thus confidence or the capability of being aware of the goodness of the self-performance is vital for guiding adaptive behavior in cases which lack direct feedback from the surroundings. Therefore, both CL and confidence level are indispensable part of analyzing user's behavioral and psychological states during assessments, training or in rehabilitation applications.
The measurement of CL can be done by indirect scales like the NASA-TLX. Another approach for measuring CL is by using various physiological sensing like Electro-encephalogram (EEG), Heart rate variability (HRV), Electrocardiogram (ECG), Electro dermal activity (EDA) and so on. However, these wearable sensors make the user uncomfortable and conscious throughout the experiment. Nearable sensors are better, as they monitor the physiological changes remotely. Eye tracking is an attractive means for remotely monitoring a user.
Assessment of CL based on eye tracking is mainly done on the spontaneous pupillary fluctuations (SPF) happening while doing a task. However, the raw SPF is composed of various inherent components as it is the outcome of decision making, confidence levels and the CL. The SPF is also influenced by aspects of the visual stimulus like the changes in spatial frequency or luminance responses. This adds to the complication towards the extraction of CL from SPF. The degree to which the pupillary signal corresponds to cognitive process or the reflex actions is still unknown. This can be done by identifying each of the hidden sources that constitute the overall SPF.

SUMMARY

The following presents a simplified summary of some embodiments of the disclosure in order to provide a basic understanding of the embodiments. This summary is not an extensive overview of the embodiments. It is not intended to identify key/critical elements of the embodiments or to delineate the scope of the embodiments. Its sole purpose is to present some embodiments in a simplified form as a prelude to the more detailed description that is presented below.
In view of the foregoing, an embodiment herein provides a system for determining cognitive load and confidence level of a person. The system comprises a display screen, an eye tracker, a memory and a processor in communication with the memory. The display screen present in front of the person to provide a stimulus, wherein stimulus results in generation of spontaneous pupillary fluctuations (SPF). The eye tracker present in front of the person configured to capture the spontaneous pupillary fluctuations as a pupillary signal of the person in response to the stimulus. The processor further comprises a decomposition module, an isolation module, a first reconstruction module, a second reconstruction module, an extraction module, a cognitive load detection module and a confidence level determining module. The decomposition module decomposes the captured pupillary signal using empirical mode decomposition (EMD) and ensemble EMD into a finite set of intrinsic mode functions (IMFs). The decomposition module further decomposes the captured pupillary signal using variation mode decomposition (VMD) in to a plurality of VMD components. The isolation module isolates a source of interest of the captured pupillary signal using independent component analysis (ICA). The first reconstruction module reconstructs the isolated source of interest in the finite set of IMFs by multiplying the isolated source of interest with a mixing matrix. The second reconstruction module reconstructs the pupillary signal using the finite set of IMFs and the plurality of VMD components. The extraction module extracts a plurality of features from the captured pupillary signal, the reconstructed pupillary signal and the plurality of VMD components, wherein the plurality of features include a relative mean-frequency power and a percentage change in pupil dilation. The cognitive load detection module detects the cognitive load of the person using the plurality of features. The confidence level determining module determines the confidence level of the person using the plurality of features.
In another aspect the embodiment here provides a method for determining cognitive load and confidence level of a person. Initially, a stimulus is provided to the person using a display screen present in front of the person, wherein stimulus results in generation of spontaneous pupillary fluctuations (SPF). In the next step, the spontaneous pupillary fluctuations are captured as a pupillary signal of the person in response to the stimulus using an eye tracker placed in front of the person. Further, the captured pupillary signal is decomposed using empirical mode decomposition (EMD) and ensemble EMD into a finite set of intrinsic mode functions (IMFs). At the same time, the captured pupillary signal is also decomposed using variation mode decomposition (VMD) in to a plurality of VMD components. In the next step, a source of interest of the captured pupillary signal is isolated using independent component analysis (ICA). The isolated source of interest in the finite set of IMFs is then reconstructed by multiplying the isolated source of interest with a mixing matrix. In the next step, the pupillary signal is reconstructed using the finite set of IMFs and the plurality of VMD components. In the next step, a plurality of features are extracted from the captured pupillary signal, the reconstructed pupillary signal and the plurality of VMD components, wherein the plurality of features include a relative mean-frequency power and a percentage change in pupil dilation. In the next step, the cognitive load of the person is detected using the plurality of features. And finally, the confidence level of the person is determined using the plurality of features.
It should be appreciated by those skilled in the art that any block diagram herein represent conceptual views of illustrative systems embodying the principles of the present subject matter. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computing device or processor, whether or not such computing device or processor is explicitly shown.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles.
Fig. 1 illustrates a block diagram of a system for determining cognitive load and confidence level of a person according to an embodiment of the present disclosure;
Fig. 2 shows an experimental setup of the system for determining cognitive load and confidence level of the person according to an embodiment of the disclosure;
Fig. 3 shows VMD components of a pupil dilation signal according to an embodiment of the present disclosure;
Fig. 4A-4B is a flowchart illustrating the steps involved in determining cognitive load and confidence level of a person according to an embodiment of the present disclosure;
Fig. 5 shows a schema of an addition task stimulus according to an embodiment of the disclosure;
Fig. 6 shows a schema of an anagram task according to an embodiment of the disclosure; and
Fig. 7 shows an averaged pupil size data for different tasks according to an embodiment of the disclosure.

DETAILED DESCRIPTION

Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the spirit and scope of the disclosed embodiments. It is intended that the following detailed description be considered as exemplary only, with the true scope and spirit being indicated by the following claims.
Referring now to the drawings, and more particularly to Fig. 1 through Fig. 7, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and/or method.
According to an embodiment of the disclosure, a system 100 for determining cognitive load and confidence level of a person is shown in the block diagram of Fig. 1. The system 100 is configured to determine the cognitive load of the person while a person is performing a task and also measures the confidence with which the task is performed. The system 100 determines the cognitive load by measuring a spontaneous pupillary fluctuations (SPF) generated by the eyes of the person in response to the performed task. The system 100 uses variational mode decomposition (VMD) in order to overcome the limitations imposed by empirical mode decomposition. The variational mode decomposition outperforms existing state-of-the-art methods. The hidden components of pupillary response were identified during cognitive tasks like mental addition and the anagram test.
According to an embodiment of the disclosure, the system 100 further comprises a display screen 102 or an input / output module 102, an eye tracker 104, a memory 106 and a processor 108 as shown in the block diagram of Fig. 1. The processor 108 works in communication with the memory 106. The processor 108 further comprises a plurality of modules. The plurality of modules accesses the set of algorithms stored in the memory 106 to perform a certain functions. The processor 108 further comprises a decomposition module 110, an isolation module 112, a first reconstruction module 114, a second reconstruction module 116, an extraction module 118 a cognitive load detection module 120 and a confidence level determining module 122.
According to an embodiment of the disclosure the input/output module 102 or a display screen 102 is configured to provide a stimulus to the persons. The input/output module 102 is configured to provide one or more task to the person. In an example of the disclosure, the input/output module 102 is a computer screen 124 as shown in the experimental setup of Fig. 2. In the present example, the input/output module 102 is configured to provide an addition task and an anagram task to the person for analyzing the cognitive load and the confidence level of the person. The same has been explained in detail in the later part of this disclosure. The input/output module 102 can include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like and can facilitate multiple communications within a wide variety of networks N/W and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite.
According to an embodiment of the disclosure, the system 100 comprises the eye tracker 104 placed in front of the user. The experimental setup of the system 100 is shown in Fig. 2. A low cost eye tracker from Eye Tribe Company with a sampling rate of 30 Hz has been used. The use of any other eye tracker is well within the scope of this disclosure. An initial calibration is done in order to ensure minimum systematic error. A chin rest 126 is used to minimize the head movements during the experiment. The display screen 102 is placed at a distance of 60 cm from the participant is used to present the stimulus. The eye tracker 104 is configured to capture the spontaneous pupillary fluctuations (SPF) as a pupillary signal of the person in response to the stimulus.
According to an embodiment of the disclosure, the processor 108 comprises the decomposition module 110. The decomposition module 110 is configured to decompose the captured pupillary signal using empirical mode decomposition (EMD) and ensemble EMD into a finite set of intrinsic mode functions (IMFs). EMD basically decomposes a time series signal into finite set of intrinsic mode functions (IMFs). IMFs are zero-mean oscillatory functions that are orthogonal to each other. EMD is selected as it decomposes the signal without any prior knowledge of the signal of interest that is embedded in the data. Since, EMD is highly sensitive to noise, an enhanced version called ensemble EMD (EEMD) is used. EEMD produces the IMF set for an ensemble of trials, wherein each IMF is produced by applying EMD to the signal of interest thereby cancelling the noisy components. Total 11 IMFs are extracted as it is the maximum limit observed from the morphology of SPF.
The decomposition module 110 is further configured to decompose the captured pupillary signal using variation mode decomposition (VMD) in to a plurality of VMD components. VMD decomposes a signal x(t) into K discrete modes uk. Here, each mode is compact along its center frequency wk. The technique is to solve a constrained variational function to search for wk and uk which is given by equation (1).
……………………. (1)
Fig. 3 shows an SPF signal decomposed into 3 VMD components. The number of components were empirically selected to be 3, above which no meaningful component is found.
According to an embodiment of the disclosure, the processor 108 further comprises the isolation module 112. The isolation module 112 is configured to isolate a source of interest of the captured pupillary signal using independent component analysis (ICA). Independent component analysis (ICA) is used to separate hidden components of a multivariate signal. ICA is applied on the derived IMFs.
According to an embodiment of the disclosure, the processor 108 also comprises a first reconstruction module 114 and a second reconstruction module 116. The first reconstruction module 114 is configured to reconstruct the isolated source of interest in the finite set of IMFs by multiplying the isolated source of interest with a mixing matrix. The corresponding mixing and un-mixing matrix can be derived using any known technique in the art. The independent components of interest are selected and multiplied with M to reconstruct its appearance in the IMF.
Further, the second reconstruction module 116 is configured to reconstruct the pupillary signal using the finite set of IMFs and the plurality of VMD components. The newly derived IMFs are used to reconstruct the source of interest of the original signal. EEMD+ICA and EMD+ ICA was selected for comparison with the proposed approach as it is reported to be the best one in the art. This is explained in detail in the later part of the disclosure.
According to an embodiment of the disclosure, the processor 108 also comprises the extraction module 118. The extraction module 118 is configured to extract a plurality of features from the captured pupillary signal, the reconstructed pupillary signal and the plurality of VMD components. In an embodiment of the disclosure, the plurality of features include a relative mean-frequency power, f1 and a percentage change in pupil dilation f2.
The relative mean-frequency power, f1 can be calculated as follows. Feature f1(t) per trial t gives frequency information of the signal as shown in Equation (2).
f1(t) = mf(?)t * p(?)t ………………………. (2)
where p is the power of the signal corresponding to the mean frequency mf of the signal for the power band ? (0-4 Hz), given by Equation (3).
mf(?) = (?_(i=0)^(n-1)¦I?if?i)/(?_(i=0)^(n-1)¦?I?(i)?)………………………. (3)
where n is the number of frequency bins, ? is the frequency band, f?(i) is the frequency and I?(i) is the energy density of ? at frequency bin i. It was further extended in order to get the changes w.r.t the baseline period as provided in Equation (4)
f1(t) = mf(w)t * p(?)t - mf(w)b * p(?)b ………………. (4)
where mf (?)t and mf (?)b are the minimum mean frequencies in the trial and baseline duration respectively, over a window of 1 sec; and p(?)t, p(?)b are the powers corresponding to mf(?)t and mf(?)b, respectively. The subtraction of baseline is done to compensate for the initial condition of an individual during the task, thereby reducing the subject-specific variability.
Further, the percentage change in pupil dilation feature, f2, is also calculated. The most widely used metric to assess mental states from SPF is by measuring the percentage change in the pupil dilation (PCPD) w.r.t baseline. PCPD increases with the increase in the cognitive load. Features, f1 and f2 are computed on raw SPF data, reconstructed data using EMD+ICA, EEMD+ICA, VMD+ICA and also on the plurality of VMD components derived using VMD.
According to an embodiment of the disclosure, the processor 108 also comprises the cognitive load detection module 120 and the confidence level determining module 122. The cognitive load detection module 120 is configured to detect the cognitive load of the person using the plurality of features, f1 and f2. The confidence level determining module 122 is configured to determine the confidence level of the person using the plurality of features.
In operation, a flowchart 200 illustrating a method for determining cognitive load and confidence level of the person while performing certain task in Fig. 4A-4B. Initially at step 202, a stimulus is provided to the person using the display screen 102 present in front of the person. The stimulus is provided in the form of task. In an example there are two types of stimulus which can be provided to the person, the addition task and the anagram task. The stimulus results in generation of spontaneous pupillary fluctuations (SPF). At step 204, the spontaneous pupillary fluctuations are captured as the pupillary signal of the person in response to the stimulus using the eye tracker 104 placed in front of the person. In an example, the low cost eye tracker such as Eye Tribe is used.
In the next step 206, the captured pupillary signal is decomposed using empirical mode decomposition (EMD) and ensemble EMD into a finite set of intrinsic mode functions (IMFs). The IMFs are zero-mean oscillatory functions that are orthogonal to each other. While simultaneously at step 208, the captured pupillary signal is decomposed using variation mode decomposition (VMD) in to a plurality of VMD components.
In the next step 210 which is following the step 206, the source of interest of the captured pupillary signal is isolated using independent component analysis (ICA). At step 212, the isolated source of interest in the finite set of IMFs is reconstructed by multiplying the isolated source of interest with the mixing matrix. Further at step 214, the pupillary signal is reconstructed using the finite set of IMFs. In the next step 216, the plurality of features are extracted from the captured pupillary signal, the reconstructed pupillary signal and the plurality of VMD components. The plurality of features include a relative mean-frequency power and a percentage change in pupil dilation. The relative mean-frequency power and the percentage change in pupil dilation are computed on raw SPF data, reconstructed data using EMD+ICA, EEMD+ICA, VMD+ICA and also on the plurality of VMD components derived using VMD. In the next step, the cognitive load of the person is detected using the plurality of features. And finally, the confidence level of the person is determined using the plurality of features.
According to an embodiment of the disclosure, the system 100 can also be explained with the help experimental procedures and results. As shown in Fig. 2, the experimental setup includes a low cost eye tracker from Eye Tribe, the chin rest 126 and a computer screen 124. The chin rest 126 is used to minimize the head movements during the experiment.
For the purpose of experiment, the addition task (A1) and the anagram task (A2) have been used for analyzing the cognitive load (CL) and the confidence level. Both the tasks have been modified in order to achieve variations in cognitive load (low and high CL) as well as confidence level. For the low cognitive load version of addition task, 10 single digit numbers appears on the computer screen one after another. For high cognitive load version, 10 such double digit numbers are shown on the screen. Participants are instructed to add those numbers mentally and report at the end of a session. Each session started with a 45 second baseline as shown in Fig. 5. The numbers used are as suggested in 2 sessions of low CL and 2 sessions of high CL are administered for each individual. This addition task is treated as the low confidence task as the correct answer is unknown and participants does not get any sort of feedback about the correctness.
The second task is a modified version of the standard anagram task so that the low and high load variants can be obtained as shown in Fig. 6. A list of frequently used 4 letter words are selected for the low load variant. On the other hand, 5 letter words are selected for the high load version. The 5 letter words were specifically selected from the article “The teacher’s wordbook of 30000 words, by Thorndike, E.L., Lorge, I., 1994, Newyork, Columbia university, teachers college”. The variations in cognitive load (low and high) is achieved due to (i) Number of letters in the words, (ii) Shuffling of the letters in the word, and iii) word frequency counts. The parameters considered while selection are summarized in Table 1. The shuffling of the letters in the words are done carefully so as to induce low and high cognitive loads when the participants unscrambled them. The frequency of occurrence is kept constant in both the cases as English is the second language for all of the participants under test.
Easy Difficult
# of letters 4 5
Shuffling 1324 14253, 25341, 52413, 31425, 42513
Word Frequency Very Frequent
(> 100 times / million)

TABLE 1 – Criterion used in anagram task

An initial pilot study was carried out to finalize the words to be used, the font size, task time and so on, for which 32 participants from our research lab (mean age +/- SD: 30.06 +/- 5.41) volunteered to provide feedback on a 5-point Likert scale. However, these parameters are configurable and hence can be changed as required. Sample questions from the list are: (i) Rate the difficulty level of the task. (ii) The time given to complete the task was sufficient; and so on. The question used to assess the difficulty levels of 4 and 5 letter words shows fairly good agreement across participants (p <0.05; effect size: d=0.71). Each test session started with a 45 second baseline period during which the white screen with a black fixation cross `+' at the center is shown. After 45 seconds the jumbled words are shown one after the other on the screen. The protocol of presentation of this task is as shown in Fig. 6. The participants are instructed to click the mouse button as soon as they could unscramble the words. If a participant fails to unscramble the word, then a new word appears after a duration of 30 seconds. 10 such words are presented in a particular session. Like A1 task, data was collected for two low and two high load versions of the A2 task also. The confidence level associated with this task (both low and high load variant) is fairly high as the participant is certain that he got the correct answer.
Data collection protocol
Twenty one healthy participants (11 females; mean age +/- SD: 28.59 +/- 4.9 years) from our research lab volunteered for the present study. They have normal or corrected to normal vision with glasses and also have similar educational and cultural backgrounds. The data collection procedure is carried out in a closed quiet room where constant environmental lighting is maintained. These factors ensured minimum variances across all participants. All the participants executed both addition and anagram task but half of them performed addition task first whereas remaining half performed anagram task first.
Results and discussions
The Eye Tribe eye tracker device returns the pupil size in arbitrary units. The ANOVA test shows significant differences in the average pupil size obtained from the eye tracker Eye Tribe (23.89 +/- 0.26 for single digit and 26.79 +/- 0.22 for double digit addition task) with p <0.05 and effect size: d=0.97. Similarly, in case of anagram task, the average pupil size for the 4-letter anagram task is 25.42 (+/- 0.52) while it is 25.92 (+/- 0.27) for the 5-letter anagram task (p <0.05; effect size: d=0.26). Fig. 7 shows the average pupil responses for 4 tasks averaged over all trials across all the participants. In case of A1, each trial lasted for 3 seconds and hence, Fig. 7 shows the averaged out trials of 3 second duration. For A2, the length of the trial is subject dependent as it lasted till the subject performed the mouse click event. Hence, Fig. 7 shows the averaged out data taken from 2 seconds before the click and 1 second after the click event.
The reaction times varied significantly for 2 variants of A2 (p <0.05; effect size: d=0.2547). The Average (+/- SD) reaction times are 2.7 (+/- 3.24) and 8.16 (+/- 5.9) seconds respectively for the 4 and 5 letter word versions.
Performance on simulated data
To test the effectiveness of the algorithms, first their performance was assessed on the simulated data. The relative root mean squared error (RRMSE) is computed to evaluate the signal reconstruction technique as provided in Equation (5).
RRMSE = (RMS(a(t)-a^(t)))/(RMS(a(t))) 100[%]……………………. (5)
where a(t) is the signal that was intended to extract and a^(t) is the estimation of the signal of interest. Lower the RRMSE values, better is the performance. Table 2 gives the RRMSE results generated on the simulated data by adding noise to the raw pupil size data of 21 participants and the parameters are set in such a way that the noise power is around 5% of the signal power. After applying the techniques like EMD+ICA, EEMD+ICA and VMD+ICA, 3 components are derived from each combinations and they are further used solely or in combination to reconstruct the signal. Hence, components 1 through 3 is used to derive signals S1, S2 and S3, respectively. Components 1 and 2 are used together to derive signal S12. Similarly, components 1 and 3 gives S13 and components 2 and 3 gives S23. The performance of VMD is also tested without using ICA, as the first component resembles the de-noised version of the original signal. The 3 components obtained from VMD are named as C1 through C3 and hence, there is no S1 through S6 in this case (indicated
as NA in Table 2). It is observed from the Table 2 that S13 from VMD+ICA and the C1 from VMD render better reconstruction of the signal.
S1 S2 S3 S12 S13 S23 C1 C2 C3
EMD + ICA 8.84 8.66 8.59 8.25 7.52 7.97 NA NA NA
EEMD + ICA 8.5 8.7 8.5 7.9 7.05 7.92 NA NA NA
VMD + ICA 8.45 7.17 8.35 6.2 5.01 6.02 NA NA NA
VMD NA NA NA NA NA NA 5.68 99.93 99.99

TABLE 2 – RRMSE computed for each algorithmic output

FScore of cognitive load computation
The classification of low and high CL or low or high confidence states is done by computing the Fscore in an unsupervised manner for all the participants. The component-wise FCM (CFCM) performs better, but since one feature was used at a time (f1 or f2), the functionality of FCM and CFCM become identical and hence, FCM was used for clustering. The Fmeasure is defined as the harmonic mean of the precision (P) and recall (R) as shown in Equation (6).
Fmeasure = (2 ? P ? R)/(P+R)…………………………. (6)
Wherein
P=TruePositive/(TruePositive+FalsePositive) and R=TruePositive/(TruePositive+FalseNegative)
The F-score is computed for each component derived using the various algorithmic chains and the component corresponding to cognitive load and to confidence is identified. Table 3 gives the average Fmeasure for cognitive load for all the participants. The feedback obtained from the participants and the nature of the stimulus itself, have been used as the ground truth here. For A1, the full 3 second trial duration is taken for the analysis; whereas for A2, the data from the start of the trial to the mouse click event has been analyzed. In Table 3, it was reported that the highest F-score value for all the components. It is clear that with feature f1, component C2 for VMD performs better for A1 task while for A2 task, the performance of C1 and that of the state-of-the-art method is almost identical. With feature f2, signal S1 for VMD+ICA outperforms the rest for addition (A1), and in case of the anagram (A2) task, signal S3 from VMD+ICA and S1 from EMD+ICA provide similar results. Next, the addition task was selected with 2 digit addition and the anagram task with 4 letter word as the former consists of least confidence while the latter is associated with high confidence.
Raw Data EMD + ICA EEMD + ICA VMD + ICA VMD
A1 A2 A1 A2 A1 A2 A1 A2 A1 A2
f1 0.64 0.74 0.56
(S12) 0.67
(S23) 0.67
(S23) 0.68
(S23) 0.61
(S1) 0.69
(S13) 0.73
(C2) 0.73
(C1)
f2 0.73 0.64 0.81
(S1) 0.82
(S1) 0.79
(S2) 0.77
(S1) 0.87
(S1) 0.81
(S3) 0.75
(C1) 0.66
(C1)
TABLE 3 – Performance comparison of different approaches in detecting cognitive load using Fmeasure
F-score of confidence level computation
For the addition task, the whole 3 second trial is taken; whereas, in case of the anagram task, a window of 2 second duration from the mouse click event is selected as it is the window when the participant senses confidence. If the trial is lesser than 2 second window, then the complete trial data is taken for the analysis. The trials with no clicks are not considered. Though cognitive load is also a distinguishing factor among the 2 stimuli, the component other than the one used for cognitive load detection should render the computation of confidence levels. The results are presented in Table 4. It is to be noted that the components giving highest F-score other than the ones reported in Table 3 are selected and is presented in Table 4, as the former is solely based on cognitive load while the latter corresponds to the confidence levels. Results show that the signal S12 in EEMD+ICA performs better for feature f1; and signal S2 in VMD+ICA gives better results for feature f2.

EMD + ICA EEMD + ICA VMD + ICA VMD
f1 0.58 (S23) 0.66 (S12) 0.59 (S3) 0.60 (C3)
f2 0.76 (S2) 0.76 (S23) 0.78 (S2) 0.61 (C2)
TABLE 4 – Performance comparison of different approaches in detecting confidence using Fmeasure for 2 digit addition and 4 letter anagram task

The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims and may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope of the claims if they have similar elements that do not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language of the claims.
The embodiments of present disclosure herein solves the difficulty of detection of cognitive load and confidence level of the person using pupillary fluctuations. The disclosure provides a method and system for determining cognitive load and confidence level using the pupillary fluctuations generated by the eye of the person in response to a stimulus. Moreover, the system identifies hidden sources of SPF using EMD+ICA, EEMD+ICA, VMD+ICA and VMD
It is to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g. any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g. hardware means like e.g. an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g. an ASIC and an FPGA, or at least one microprocessor and at least one memory with software modules located therein. Thus, the means can include both hardware means and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments may be implemented on different hardware devices, e.g. using a plurality of CPUs.
The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various modules described herein may be implemented in other modules or combinations of other modules. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
The illustrated steps are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope and spirit of the disclosed embodiments. Also, the words “comprising,” “having,” “containing,” and “including,” and other similar forms are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.
Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.
It is intended that the disclosure and examples be considered as exemplary only, with a true scope and spirit of disclosed embodiments being indicated by the following claims.