WO2018042261A1 - Method and system for monitoring of mental effort - Google Patents

Method and system for monitoring of mental effort Download PDF

Info

Publication number
WO2018042261A1
WO2018042261A1 PCT/IB2017/050677 IB2017050677W WO2018042261A1 WO 2018042261 A1 WO2018042261 A1 WO 2018042261A1 IB 2017050677 W IB2017050677 W IB 2017050677W WO 2018042261 A1 WO2018042261 A1 WO 2018042261A1
Authority
WO
WIPO (PCT)
Prior art keywords
gsr
data
task
feature
tasks
Prior art date
Application number
PCT/IB2017/050677
Other languages
French (fr)
Inventor
Pratyusha DAS
Debatri CHATTERJEE
Aniruddha Sinha
Avik GHOSE
Original Assignee
Tata Consultancy Services Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tata Consultancy Services Limited filed Critical Tata Consultancy Services Limited
Publication of WO2018042261A1 publication Critical patent/WO2018042261A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/16Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/05Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves 
    • A61B5/053Measuring electrical impedance or conductance of a portion of the body
    • A61B5/0531Measuring skin impedance
    • A61B5/0533Measuring galvanic skin response
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/48Other medical applications
    • A61B5/486Bio-feedback
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/48Other medical applications
    • A61B5/4884Other medical applications inducing physiological or psychological stress, e.g. applications for stress testing

Definitions

  • the present application generally relates to remote monitoring of mental efforts. Particularly, the application provides a method and system for end to end system based on low cost physiological sensor for measurement of cognitive load.
  • Measurement of mental workload or cognitive load can be done through a number of approaches. Most widely used measure of work overload is primary task measures like accuracy, time required to complete the task etc. These measures might give an overall idea about the workload or difficulty level of the task but it fails to account for individual differences in intellectual performances in terms of availability of mental resources. [004] Subjective measures like questionnaires or user feedback based approaches are highly biased and subject dependent. Physiological changes give a much more reliable and direct measure of the mental workload. The problems associated with the physiological measure of cognitive load are (i) the costly physiological sensors which makes mass deployment effectively impossible and (ii) correct interpretation of sensor data.
  • the present application provides a system (102).
  • the system (102) may comprise a processor (202), a memory (204), and a GSR (Galvanic Skin Resistance) sensor (224) operatively coupled with said processor.
  • the system (102) further comprises a GSR acquisition module (210) configured to acquire a Galvanic Skin Resistance (GSR) data from each of a one or more user performing a plurality of tasks of predefined varying cognitive load.
  • GSR acquisition module (210) receives GSR data from the GSR sensor (224).
  • the system disclosed herein further comprises a preprocessing module (212) configured to pre-process the acquired GSR data to remove one or more artifact from the acquired GSR data to generate a preprocessed GSR data for each of the one or more user and a feature extraction module (214) configured to extract a plurality of features for each of the one or more users from the preprocessed GSR data.
  • the system comprises a feature selection module (216) selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl).
  • the system further comprises a feature selection module (218) selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl) and an effort index generation module (220) computing a score window-wise for each of the one or more users and creating effort index (El) to measure cognitive load based on the selected most discriminative feature.
  • a feature selection module (218) selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl)
  • an effort index generation module (220) computing a score window-wise for each of the one or more users and creating effort index (El) to measure cognitive load based on the selected most discriminative feature.
  • the application discloses a method for measuring cognitive load; said method comprising steps of acquiring a Galvanic Skin Resistance (GSR) data from each of a one or more user performing a plurality of tasks of predefined varying cognitive load using a GSR acquisition module (210).
  • GSR acquisition module (210) receives GSR data from a GSR sensor (224).
  • the method further comprises pre-processing the acquired GSR data to remove an artifact from the acquired GSR data to generate a preprocessed GSR data for each of the one or more user using a preprocessing module (212).
  • the disclosed method comprises extracting a plurality of features for each of the one or more users from the preprocessed GSR data using a feature extraction module (214).
  • the method comprises the step of selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl) using a feature selection module (216); and finally a score is computed window- wise for each of the one or more users and effort index (El) is created to measure cognitive load based on the selected most discriminative feature using an effort index generation module (218).
  • the application discloses a non-transitory computer readable medium storing instructions which when executed by a possessor on a system, cause the processor to perform method for measuring cognitive load comprising steps of acquiring a Galvanic Skin Resistance (GSR) data from each of a one or more user performing a plurality of tasks of predefined varying cognitive load using a GSR acquisition module (210).
  • GSR acquisition module (210) receives GSR data from a GSR sensor (224).
  • the method further comprises preprocessing the acquired GSR data to remove an artifact from the acquired GSR data to generate a preprocessed GSR data for each of the one or more user using a preprocessing module (212).
  • the disclosed method comprises extracting a plurality of features for each of the one or more users from the preprocessed GSR data using a feature extraction module (214). Further the method comprises the step of selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl) using a feature selection module (216); and finally a score is computed window-wise for each of the one or more users and effort index (El) is created to measure cognitive load based on the selected most discriminative feature using an effort index generation module (218). [009] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
  • FIG. 1 illustrates a network implementation of a system for measuring cognitive load, in accordance with an embodiment of the present subject matter
  • FIG. 2 shows block diagrams illustrating the system for measuring cognitive load, in accordance with an embodiment of the present subject matter
  • FIG. 3 illustrates shows a flowchart illustrating the method for measuring cognitive load, in accordance with an embodiment of the present subject matter
  • FIG. 4 shows an exemplary architecture of the disclosed invention in accordance with an embodiment of the present subject matter
  • FIG. 5 shows the computed Discriminative Index (Dl) for all participants, in accordance with an embodiment of the present subject matter
  • FIG. 6 shows the Effort Index (El) for a participant varying over windows for low load task and high load task in accordance with an embodiment of the present subject matter
  • FIG. 7 shows the average El for a low load and high load tasks in accordance with an embodiment of the present subject matter
  • FIG. 8 shows the Dl of all participants for the arithmetic summation task (both low and high) for experiment 2 in accordance with an embodiment of the present subject matter
  • FIG. 9 shows El for a participant varying over windows for low load and high load task in accordance with an embodiment of the present subject matter.
  • Fig. 10 shows the average El for all the participant for the tonic mean feature in accordance with an embodiment of the present subject matter.
  • FIG. 1 a network implementation 100 of a system 102 for measuring cognitive load is illustrated, in accordance with an embodiment of the present subject matter.
  • the system 102 may also be implemented in a variety of computing systems, such as a laptop computer, a desktop computer, a notebook, a workstation, a mainframe computer, a server, a network server, and the like.
  • the system 102 may be implemented in a cloud-based environment. In another embodiment, it may be implemented as custom built hardware designed to efficiently perform the invention disclosed.
  • the system 102 may be accessed by multiple users through one or more user devices 104-1 , 104-2... 104-N, collectively referred to as user devices 104 hereinafter, or applications residing on the user devices 104.
  • user devices 104 may include, but are not limited to, a portable computer, a personal digital assistant, a handheld device, and a workstation.
  • the user devices 104 are communicatively coupled to the system 102 through a network 106.
  • the network 106 may be a wireless network, a wired network or a combination thereof.
  • the network 106 can be implemented as one of the different types of networks, such as intranet, local area network (LAN), wide area network (WAN), the internet, and the like.
  • the network 106 may either be a dedicated network or a shared network.
  • the shared network represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), and the like, to communicate with one another.
  • the network 106 may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, and the like.
  • the present invention referring to Fig. 2, a detailed working of the various components of the system 102 is described.
  • the system (102) is configured for extracting at least one metrics for measuring cognitive load while performing a plurality of tasks of varying cognitive load.
  • the system (102) may comprise a processor (202), a memory (204), and a GSR (Galvanic Skin Resistance) sensor (224) operatively coupled with said processor.
  • the system (102) further comprises a GSR acquisition module (210) configured to acquire a Galvanic Skin Resistance (GSR) data from each of a one or more user performing a plurality of tasks of predefined varying cognitive load.
  • GSR acquisition module (210) receives GSR data from the GSR sensor (224).
  • the GSR sensor (224) is a wearable sensor worn by the one or more users.
  • the system disclosed herein further comprises a preprocessing module (212) configured to pre-process the acquired GSR data to remove one or more artifact from the acquired GSR data to generate a preprocessed GSR data for each of the one or more user and a feature extraction module (214) configured to extract a plurality of features for each of the one or more users from the preprocessed GSR data.
  • the feature extraction may be based on one of Peak detection, Tonic power and fluctuation analysis.
  • the system comprises a feature selection module (216) selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl).
  • the system further comprises a feature selection module (218) selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl) and an effort index generation module (220) computing a score for each of the one or more users and creating effort index (El) to measure cognitive load based on the selected most discriminative feature.
  • the score computed by the effort index generation module (218) is stored on a server and used in combination with other known features to determine cognitive load.
  • the system is configured such that the plurality of tasks of varying cognitive load are performed after an optimal rest time between two consecutive tasks of the plurality of task has elapsed and wherein one of the two consecutive tasks is a high load task and other is a low load task.
  • the optimal rest time is calculated based on the difference between El of consecutive low load task and El for High load task such that the optimal rest time is calculated based maximum difference found in Effort Index in low load and high load for varying rest periods.
  • the scores for low load and high load may be statistically significant.
  • a flow chart illustrating the method for measuring cognitive is shown.
  • the process starts at step 302 where a Galvanic Skin Resistance (GSR) data is acquired from each of a one or more users performing a plurality of tasks of predefined varying cognitive load and using a GSR sensor.
  • GSR Galvanic Skin Resistance
  • the varying cognitive load of tasks may comprise tasks from a group of high cognitive load tasks and low cognitive load tasks.
  • the plurality of tasks of varying cognitive load are performed after an optimal rest time between two consecutive tasks of the plurality of task has elapsed and wherein one of the two consecutive tasks is a high load task and other is a low load task and wherein the optimal time is calculated based on the difference between El for consecutive High load task and Low load task.
  • the acquired GSR data is preprocessed to remove an artifact to generate a preprocessed GSR data for each of the one or more user.
  • a plurality of features for each of the one or more users are extracted from the preprocessed GSR data. In one embodiment feature extraction may be based on at least one of Peak Detection, Tonic power and Fluctuation analysis.
  • a most discriminative feature is selected for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl).
  • a score for each of the one or more users is computed.
  • an effort index (El) may also be created to measure cognitive load based on the selected most discriminative feature.
  • the mobile sub-system captures data from GSR sensor over Bluetooth to compute the cognitive score.
  • this score is transmitted over mobile internet to a backend server which performs comparative analytics on the score.
  • Such analysis may include but are not limited to cross-sectional and longitudinal analysis.
  • a low cost finger wearable GSR device having a sampling frequency of 5 Hz may be used for collecting GSR data.
  • the sensors are connected to the middle phalanges of the index and middle fingers of an individual while performing any task.
  • the acquired data may be passed through a low pass filter to remove high frequency artifacts.
  • the cleaned data is then analyzed in windows of duration 10 sec to derive three features namely, (i) fluctuation index (ii) tonic mean and (iii) peak detection.
  • the signal is divided in small windows of length say / and in each window, a least square line is fitted through the data points in each window to represent the local trend.
  • the time-series of these local trends over the entire length L is say ⁇ t) , then the Fluctuation index (FI) is calculated as given in (2).
  • Tonic power of GSR For any GSR signal there are two components of GSR signal: a slow varying component called tonic and a fast varying component called phasic. The components corresponding to the frequencies ⁇ 0.5 Hz are tonic components and the remaining are called phasic component. In an example for calculation, N point Fast Fourier Transform (FFT) of the complete signal is done.
  • FFT Fast Fourier Transform
  • Peak Detection For any GSR peak, if the peak height measured from its previous valley is more than o.os ⁇ then it is considered as a significant peak.
  • x (t) be the raw signal, sampled at a frequency of 5 Hz, and is passed through a 2Hz low- pass filter to remove high frequency artifacts. From two consecutive data points of x (t), y (t) is obtained using equation (5).
  • Dl is [0, 1 ]. Greater the Dl, more is the separation in the GSR feature values for two tasks with respect to rest.
  • duration may be of 1 min. This interval is treated as the baseline interval and both the features are derived window-wise from these intervals as well. For each of the features, the minimum feature value among all the window during rest period is extracted and used as the feature value at rest.
  • the next step after selection the most discriminative feature is that of Score computation.
  • Score computation in an aspect the following is implemented. While a participant is performing a task initially due to anticipation/excitement his/her effort level increases linearly. As the difficulty level of the task increases, the person tends to give more effort. If the difficulty level is increased further, the effort given reaches a saturation at some point of time. This trend is clearly reflected in GSR and hence defined score follows the rectangular hyperbolic characteristics and termed it as Effort Index (El) as shown in (8).
  • Effort Index (El) referring to equation (8) f(R) is feature value of the most inactive window in Rest period. The inactive window is that particular window of rest period that gives the minimum value of the feature and f( w ) is the feature value at the current window ( w). Then E/ is computed using
  • EI w) l .m (8) where f( w ) > o and f( w ) > f(R) .
  • the Dl is represented for a range of -0.5 to 1 .5 purposefully, to show the desired range of D/ [0, 1 ] with more clarity.
  • participants S1 , S3 and S14 mean tonic power overshoots and for S4, S1 1 , S12, S13, S19 it undershoots the preferred Dl range.
  • S4 S12 and S13, Dl is negative and for S10, S1 1 and S14, Dl is greater than 1 in Fl, which is not expected. It is clear from the plot that peak count always remains within the desired range of Dl, showing its superiority. From the computational complexity perspective also, complexity is more in computation of tonic power and Fl than that of peak count.
  • Fig. 7 shows the average E/ for a low load and high load tasks. It is clear from the figure that there is a clear separation in E/ for low load and high load task for all the participants. For the participants 4, 1 1 , 12 and 14 the average El is zero for low task hence the corresponding bar is not visible in Fig. 7.
  • the low and high task duration is not exactly same, it depends on how much time a participant takes to finish it. So, we compute each feature window-wise. Here, the results are shown considering a window-size of a 15 sec. The features in rest period are also computed window-wise leaving the first 30 sec rest data for the similar reason as mentioned in Experiment-1 . Next, we consider the most inactive window i.e.
  • Fig. 8 represents the Dl of all participants for the arithmetic summation task (both low and high) for the above mentioned features. While computing D/ for peak count, due to time normalization problem we consider maximum peak count feature across all the windows. For 9 participant, Dl is maximum for the tonic mean feature in this task. Fig. 9 shows El for a participant varying over windows for low load and high load task. The low and high load task duration are different here because participant takes less time to do the low task than the high task though the number of trials in low and high load are same i.e. 5.
  • Fig. 10 shows the average El for all the participant for the tonic mean feature. For 13 participants among 15 average El is low for low load than the high load, for S10 and S12, the El nature is reverse.
  • a computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored.
  • a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein.
  • the term "computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include random access memory (RAM), readonly memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.

Abstract

The present application provides a method and system for monitoring of mental effort is disclosed. The method and system disclosed herein comprise acquiring GSR data using a GSR sensor wherein the GSR data is collected while performing plurality of tasks of varying cognitive load, preprocessing the acquired data for artifact removal and generating a preprocessed data, extracting plurality of features from the preprocessed GSR data using feature extraction techniques including Peak Detection, Tonic power and Fluctuation analysis, selecting a most discriminative feature from the plurality of extracted feature based on a discriminative index, calculating a score and generating an effort index. The system and method also comprise determining an optimal rest period which is used as reference for computation of the effort index.

Description

METHOD AND SYSTEM FOR MONITORING OF MENTAL EFFORT
DESCRIPTION
Priority Claim
[001] This PCT application claims priority to: India Application No. 201621030176, filed on 02 September 2016. The entire contents of the aforementioned application are incorporated herein by reference.
Technical Field
[002] The present application generally relates to remote monitoring of mental efforts. Particularly, the application provides a method and system for end to end system based on low cost physiological sensor for measurement of cognitive load.
Background
[003] Measurement of mental workload or cognitive load can be done through a number of approaches. Most widely used measure of work overload is primary task measures like accuracy, time required to complete the task etc. These measures might give an overall idea about the workload or difficulty level of the task but it fails to account for individual differences in intellectual performances in terms of availability of mental resources. [004] Subjective measures like questionnaires or user feedback based approaches are highly biased and subject dependent. Physiological changes give a much more reliable and direct measure of the mental workload. The problems associated with the physiological measure of cognitive load are (i) the costly physiological sensors which makes mass deployment effectively impossible and (ii) correct interpretation of sensor data.
[005] Moreover majority of prior art teaches methods involving wearing a large number of physiological sensors which might make the participant uncomfortable and self-conscious. Also all these experiments are done using costly GSR setups with high sampling rate which is a hindrance for mass deployment.
SUMMARY
[006] Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems. For example, in one embodiment, the present application provides a system (102). The system (102) may comprise a processor (202), a memory (204), and a GSR (Galvanic Skin Resistance) sensor (224) operatively coupled with said processor. The system (102) further comprises a GSR acquisition module (210) configured to acquire a Galvanic Skin Resistance (GSR) data from each of a one or more user performing a plurality of tasks of predefined varying cognitive load. In one aspect GSR acquisition module (210) receives GSR data from the GSR sensor (224). The system disclosed herein further comprises a preprocessing module (212) configured to pre-process the acquired GSR data to remove one or more artifact from the acquired GSR data to generate a preprocessed GSR data for each of the one or more user and a feature extraction module (214) configured to extract a plurality of features for each of the one or more users from the preprocessed GSR data. The system comprises a feature selection module (216) selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl). The system further comprises a feature selection module (218) selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl) and an effort index generation module (220) computing a score window-wise for each of the one or more users and creating effort index (El) to measure cognitive load based on the selected most discriminative feature..
[007] In another embodiment, the application discloses a method for measuring cognitive load; said method comprising steps of acquiring a Galvanic Skin Resistance (GSR) data from each of a one or more user performing a plurality of tasks of predefined varying cognitive load using a GSR acquisition module (210). In one embodiment GSR acquisition module (210) receives GSR data from a GSR sensor (224). The method further comprises pre-processing the acquired GSR data to remove an artifact from the acquired GSR data to generate a preprocessed GSR data for each of the one or more user using a preprocessing module (212). Further the disclosed method comprises extracting a plurality of features for each of the one or more users from the preprocessed GSR data using a feature extraction module (214). Further the method comprises the step of selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl) using a feature selection module (216); and finally a score is computed window- wise for each of the one or more users and effort index (El) is created to measure cognitive load based on the selected most discriminative feature using an effort index generation module (218).
[008] In yet another embodiment, the application discloses a non-transitory computer readable medium storing instructions which when executed by a possessor on a system, cause the processor to perform method for measuring cognitive load comprising steps of acquiring a Galvanic Skin Resistance (GSR) data from each of a one or more user performing a plurality of tasks of predefined varying cognitive load using a GSR acquisition module (210). In one embodiment GSR acquisition module (210) receives GSR data from a GSR sensor (224). The method further comprises preprocessing the acquired GSR data to remove an artifact from the acquired GSR data to generate a preprocessed GSR data for each of the one or more user using a preprocessing module (212). Further the disclosed method comprises extracting a plurality of features for each of the one or more users from the preprocessed GSR data using a feature extraction module (214). Further the method comprises the step of selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl) using a feature selection module (216); and finally a score is computed window-wise for each of the one or more users and effort index (El) is created to measure cognitive load based on the selected most discriminative feature using an effort index generation module (218). [009] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
BRIEF DESCRIPTION OF THE DRAWINGS
[010] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles.
[01 1] FIG. 1 illustrates a network implementation of a system for measuring cognitive load, in accordance with an embodiment of the present subject matter;
[012] FIG. 2 shows block diagrams illustrating the system for measuring cognitive load, in accordance with an embodiment of the present subject matter;
[013] FIG. 3 illustrates shows a flowchart illustrating the method for measuring cognitive load, in accordance with an embodiment of the present subject matter;
[014] FIG. 4 shows an exemplary architecture of the disclosed invention in accordance with an embodiment of the present subject matter;
[015] FIG. 5 shows the computed Discriminative Index (Dl) for all participants, in accordance with an embodiment of the present subject matter;
[016] FIG. 6 shows the Effort Index (El) for a participant varying over windows for low load task and high load task in accordance with an embodiment of the present subject matter;
[017] FIG. 7 shows the average El for a low load and high load tasks in accordance with an embodiment of the present subject matter; [018] FIG. 8 shows the Dl of all participants for the arithmetic summation task (both low and high) for experiment 2 in accordance with an embodiment of the present subject matter;
[019] FIG. 9 shows El for a participant varying over windows for low load and high load task in accordance with an embodiment of the present subject matter; and
[020] Fig. 10 shows the average El for all the participant for the tonic mean feature in accordance with an embodiment of the present subject matter.
DETAILED DESCRIPTION
[021] Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the spirit and scope of the disclosed embodiments. It is intended that the following detailed description be considered as exemplary only, with the true scope and spirit being indicated by the following claims.
[022] The present application provides a computer implemented method and system for measuring cognitive load. Referring now to Fig. 1 , a network implementation 100 of a system 102 for measuring cognitive load is illustrated, in accordance with an embodiment of the present subject matter. Although the present subject matter is explained considering that the system 102 is implemented on a server, it may be understood that the system 102 may also be implemented in a variety of computing systems, such as a laptop computer, a desktop computer, a notebook, a workstation, a mainframe computer, a server, a network server, and the like. In one implementation, the system 102 may be implemented in a cloud-based environment. In another embodiment, it may be implemented as custom built hardware designed to efficiently perform the invention disclosed. It will be understood that the system 102 may be accessed by multiple users through one or more user devices 104-1 , 104-2... 104-N, collectively referred to as user devices 104 hereinafter, or applications residing on the user devices 104. Examples of the user devices 104 may include, but are not limited to, a portable computer, a personal digital assistant, a handheld device, and a workstation. The user devices 104 are communicatively coupled to the system 102 through a network 106.
[023] In one implementation, the network 106 may be a wireless network, a wired network or a combination thereof. The network 106 can be implemented as one of the different types of networks, such as intranet, local area network (LAN), wide area network (WAN), the internet, and the like. The network 106 may either be a dedicated network or a shared network. The shared network represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), and the like, to communicate with one another. Further the network 106 may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, and the like.
[024] In one embodiment the present invention, referring to Fig. 2, a detailed working of the various components of the system 102 is described. In an embodiment the system (102) is configured for extracting at least one metrics for measuring cognitive load while performing a plurality of tasks of varying cognitive load.
[025] The system (102) may comprise a processor (202), a memory (204), and a GSR (Galvanic Skin Resistance) sensor (224) operatively coupled with said processor. The system (102) further comprises a GSR acquisition module (210) configured to acquire a Galvanic Skin Resistance (GSR) data from each of a one or more user performing a plurality of tasks of predefined varying cognitive load. In one aspect GSR acquisition module (210) receives GSR data from the GSR sensor (224). In another aspect the GSR sensor (224) is a wearable sensor worn by the one or more users. The system disclosed herein further comprises a preprocessing module (212) configured to pre-process the acquired GSR data to remove one or more artifact from the acquired GSR data to generate a preprocessed GSR data for each of the one or more user and a feature extraction module (214) configured to extract a plurality of features for each of the one or more users from the preprocessed GSR data. In an embodiment the feature extraction may be based on one of Peak detection, Tonic power and fluctuation analysis. Further in another embodiment the system comprises a feature selection module (216) selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl).
[026] The system further comprises a feature selection module (218) selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl) and an effort index generation module (220) computing a score for each of the one or more users and creating effort index (El) to measure cognitive load based on the selected most discriminative feature. In an embodiment the score computed by the effort index generation module (218) is stored on a server and used in combination with other known features to determine cognitive load. In yet another embodiment the system is configured such that the plurality of tasks of varying cognitive load are performed after an optimal rest time between two consecutive tasks of the plurality of task has elapsed and wherein one of the two consecutive tasks is a high load task and other is a low load task. The optimal rest time is calculated based on the difference between El of consecutive low load task and El for High load task such that the optimal rest time is calculated based maximum difference found in Effort Index in low load and high load for varying rest periods. Also the scores for low load and high load may be statistically significant.
[027] In Referring now to Fig. 3 a flow chart illustrating the method for measuring cognitive is shown. The process starts at step 302 where a Galvanic Skin Resistance (GSR) data is acquired from each of a one or more users performing a plurality of tasks of predefined varying cognitive load and using a GSR sensor. In one embodiment the varying cognitive load of tasks may comprise tasks from a group of high cognitive load tasks and low cognitive load tasks.
[028] In an embodiment the plurality of tasks of varying cognitive load are performed after an optimal rest time between two consecutive tasks of the plurality of task has elapsed and wherein one of the two consecutive tasks is a high load task and other is a low load task and wherein the optimal time is calculated based on the difference between El for consecutive High load task and Low load task. [029] At the step 304 the acquired GSR data is preprocessed to remove an artifact to generate a preprocessed GSR data for each of the one or more user. At the step 306 a plurality of features for each of the one or more users are extracted from the preprocessed GSR data. In one embodiment feature extraction may be based on at least one of Peak Detection, Tonic power and Fluctuation analysis.
[030] At the step 308 a most discriminative feature is selected for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl).
[031] And finally at the step 310 a score for each of the one or more users is computed. In another embodiment an effort index (El) may also be created to measure cognitive load based on the selected most discriminative feature.
[032] The following paragraphs contain description of one implementation of the disclosed method and system is provided for better understanding of the claimed method and system. The following paragraphs shall not be understood as determining the scope of the instant invention which may be limited only by the claims.
[033] Referring to Fig 4, an architecture overview of the system disclosed herein is provided. The mobile sub-system captures data from GSR sensor over Bluetooth to compute the cognitive score. In an embodiment this score is transmitted over mobile internet to a backend server which performs comparative analytics on the score. Such analysis may include but are not limited to cross-sectional and longitudinal analysis. [034] In an embodiment a low cost finger wearable GSR device having a sampling frequency of 5 Hz may be used for collecting GSR data. The sensors are connected to the middle phalanges of the index and middle fingers of an individual while performing any task. In one example the acquired data may be passed through a low pass filter to remove high frequency artifacts. The cleaned data is then analyzed in windows of duration 10 sec to derive three features namely, (i) fluctuation index (ii) tonic mean and (iii) peak detection.
[035] Fluctuation Analysis: To analyze the fluctuations of a signal x(t) of length L, a zero mean signal is computed Xgn(t) by subtracting the mean Mx of the same signal as given by (1 )
Figure imgf000013_0001
Next, the signal is divided in small windows of length say / and in each window, a least square line is fitted through the data points in each window to represent the local trend. The time-series of these local trends over the entire length L is say ^t) , then the Fluctuation index (FI) is calculated as given in (2).
Figure imgf000013_0002
[036] Tonic power of GSR: For any GSR signal there are two components of GSR signal: a slow varying component called tonic and a fast varying component called phasic. The components corresponding to the frequencies < 0.5 Hz are tonic components and the remaining are called phasic component. In an example for calculation, N point Fast Fourier Transform (FFT) of the complete signal is done.
/ (3)
[037] In equation (3), which is well known in the art, / is signal frequency, s sampling frequency, k = 1,2,3,..., N-l Let at = 0.5Hz, k = kT.. The tonic power is computed by taking the inverse FFT (IFFT) of the first kT coefficients of the FFT of the GSR signal as given in equation (4) tonic component =
Figure imgf000014_0001
[038] Peak Detection: For any GSR peak, if the peak height measured from its previous valley is more than o.os^then it is considered as a significant peak. Let x(t) be the raw signal, sampled at a frequency of 5 Hz, and is passed through a 2Hz low- pass filter to remove high frequency artifacts. From two consecutive data points of x(t), y(t) is obtained using equation (5).
y(t) = x(t + l)-x(t) (5)
[039] If y(tp)<0 and y(tp -l)>0 , then it is considered as a peak at t = tp. The previous valley is detected from^). At t = tv , if y(t)>ovt = tv to(tp -i) , and y(tv -i)<o then we consider it as a valley a v . Finally the height h of the peak from the valley is computed using equation (6). In one example if /i > o.05 iS , then it is considered as a significant peak.
h = x(tp) - x(t (6)
[040] After feature extraction is concluded the next step is to select the most discriminative feature. To select the most discriminating feature from the extracted feature set, a new metric called Discriminative Index {Dl) is defined. Let f (T) S ith participant's fh feature value for the task T, where T = rest(R) / low(L) / high(H) task interval. So, D/ is defined as (7).
/ / A /< " > A,"-) (7)
'··' ./;., < // > ./;·.,· < *>
[041 ] It is expected that fi (R) < fi (L) < fi (H) and hence the desired range of
Dl is [0, 1 ]. Greater the Dl, more is the separation in the GSR feature values for two tasks with respect to rest.
[042] Next ( v/ ) is checked across all participants, for which feature (y), Dl is maximum in its desired range for maximum no. of participants. If this occurs for a feature, then we consider this as the most discriminating feature and we use that feature for score computation.
[043] Before starting the tasks, participants may be asked to relax for a duration. In an embodiment the duration may be of 1 min. This interval is treated as the baseline interval and both the features are derived window-wise from these intervals as well. For each of the features, the minimum feature value among all the window during rest period is extracted and used as the feature value at rest.
[044] The next step after selection the most discriminative feature is that of Score computation. For score calculation, in an aspect the following is implemented. While a participant is performing a task initially due to anticipation/excitement his/her effort level increases linearly. As the difficulty level of the task increases, the person tends to give more effort. If the difficulty level is increased further, the effort given reaches a saturation at some point of time. This trend is clearly reflected in GSR and hence defined score follows the rectangular hyperbolic characteristics and termed it as Effort Index (El) as shown in (8). Where referring to equation (8) f(R) is feature value of the most inactive window in Rest period. The inactive window is that particular window of rest period that gives the minimum value of the feature and f(w) is the feature value at the current window ( w). Then E/ is computed using
EI w) =l.m (8) where f(w) > o and f(w) > f(R) .
[045] This score in addition to other existing measures like performance score, completion time etc., gives an additional information more insight into the actual mental state of the participant/ user.
[046] At the beginning of a first experiment the GSR data of the participants while relaxing with closed eyes for 2 min was collected. In general, a person takes some time to start relaxing. So, first 30 sec of rest data is rejected. For comparison, next 80 sec rest data is considered as the duration of both tasks (high and low) are 80 sec. Tonic power is computed considering windows of length 1 sec and then the mean is taken as the overall tonic power for that particular task. Fl and peak count are calculated on full task duration. Next Dl is computed using equation (7) for all participants and are plotted in Fig. 5.
[047] The Dl is represented for a range of -0.5 to 1 .5 purposefully, to show the desired range of D/ [0, 1 ] with more clarity. For participants S1 , S3 and S14, mean tonic power overshoots and for S4, S1 1 , S12, S13, S19 it undershoots the preferred Dl range. For S4, S12 and S13, Dl is negative and for S10, S1 1 and S14, Dl is greater than 1 in Fl, which is not expected. It is clear from the plot that peak count always remains within the desired range of Dl, showing its superiority. From the computational complexity perspective also, complexity is more in computation of tonic power and Fl than that of peak count.
[048] To measure the mental effort of the participant Effort Index (El) is computed using selected feature for each window of duration 15 sec. Fig. 6 shows the E/ for a participant varying over windows for low load task and high load task. It is clear from the figure that the El more in case of high load task and less for the low load task. As was seen from Dl values, peak count performs well for this task, we compute El using peak count for all the participants.
[049] Fig. 7 shows the average E/ for a low load and high load tasks. It is clear from the figure that there is a clear separation in E/ for low load and high load task for all the participants. For the participants 4, 1 1 , 12 and 14 the average El is zero for low task hence the corresponding bar is not visible in Fig. 7. [050] For the Experiment 2, the low and high task duration is not exactly same, it depends on how much time a participant takes to finish it. So, we compute each feature window-wise. Here, the results are shown considering a window-size of a 15 sec. The features in rest period are also computed window-wise leaving the first 30 sec rest data for the similar reason as mentioned in Experiment-1 . Next, we consider the most inactive window i.e. the minimum feature value of all the windows as the actual rest feature for further processing. Fig. 8 represents the Dl of all participants for the arithmetic summation task (both low and high) for the above mentioned features. While computing D/ for peak count, due to time normalization problem we consider maximum peak count feature across all the windows. For 9 participant, Dl is maximum for the tonic mean feature in this task. Fig. 9 shows El for a participant varying over windows for low load and high load task. The low and high load task duration are different here because participant takes less time to do the low task than the high task though the number of trials in low and high load are same i.e. 5.
[051] Fig. 10 shows the average El for all the participant for the tonic mean feature. For 13 participants among 15 average El is low for low load than the high load, for S10 and S12, the El nature is reverse.
[052] The illustrated steps are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope and spirit of the disclosed embodiments. Also, the words "comprising," "having," "containing," and "including," and other similar forms are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise.
[053] Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term "computer-readable medium" should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include random access memory (RAM), readonly memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media. [054] It is intended that the disclosure and examples be considered as exemplary only, with a true scope and spirit of disclosed embodiments being indicated by the following claims.

Claims

CLAIMS WHAT IS CLAIMED IS:
1 . A method for measuring cognitive load; said method comprising processor
implemented steps of:
acquiring a Galvanic Skin Resistance (GSR) data from each of a one or more user performing a plurality of tasks of predefined varying cognitive load using a GSR acquisition module (210) wherein GSR acquisition module (210) receives GSR data from a GSR sensor (224);
pre-processing the acquired GSR data to remove an artifact from the acquired GSR data to generate a preprocessed GSR data for each of the one or more user using a preprocessing module (212);
extracting a plurality of features for each of the one or more users from the
preprocessed GSR data using a feature extraction module (214);
selecting a most discriminative feature for each of the one or more users from the
extracted plurality of features based on a discriminative index (Dl) using a feature selection module (216); and
computing a score for each of the one or more users and creating effort index (El) to measure cognitive load based on the selected most discriminative feature using an effort index generation module (218).
2. The method according to claim 1 wherein the GSR sensor (224) is a wearable sensor worn by the one or more users.
3. The method according to claim 1 wherein the feature extraction module (214) implements at least one of Peak Detection, Tonic power and Fluctuation analysis to extract the plurality of features.
4. The method according to claim 1 wherein the task of predefined varying cognitive load comprise one of High load tasks and Low load tasks.
5. The method of claim 1 wherein the score computed by the effort index generation module (218) is stored on a server and used in combination with other known features to determine cognitive load.
6. The method according to claim 4 wherein the plurality of tasks of varying
cognitive load are performed after an optimal rest time between two consecutive tasks of the plurality of task has elapsed and wherein one of the two consecutive tasks is a high load task and other is a low load task.
7. The method according to claim 6 wherein the optimal rest time is calculated
based on the difference between El for consecutive High load task and Low load task.
8. A system (102) for measuring cognitive load; comprising a processor (202), a memory (204), and a Galvanic Skin Resistance (GSR) sensor (224) operatively coupled with said processor, the system comprising:
a GSR acquisition module (210) configured to acquire a GSR data from each of a one or more user performing a plurality of tasks of predefined varying cognitive load wherein GSR acquisition module (210) receives GSR data from the GSR sensor (224);
a preprocessing module (212) configured to pre-process the acquired GSR data to remove one or more artifact from the acquired GSR data to generate a preprocessed GSR data for each of the one or more user;
a feature extraction module (214) configured to extract a plurality of features for each of the one or more users from the preprocessed GSR data;
a feature selection module (216) selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a
discriminative index (Dl);and
an effort index generation module (220) configure to compute a score for each of the one or more users and creating effort index (El) to measure cognitive load based on the selected most discriminative feature.
9. The system according to claim 8 wherein the GSR sensor (224) is a wearable sensor worn by the one or more users.
10. The system according to claim 8 wherein the task of varying cognitive load
comprise one of High load task and Low load task.
1 1 . The system according to claim 8 wherein the feature extraction module (214) is configured to implement at least one of Peak Detection and fluctuation analysis to extract the plurality of features.
12. The system according to claim 8 wherein the score computed by the effort index generation module (218) is stored in database and is used in combination with other known features to determine cognitive load.
13. The system according to claim 10 wherein the plurality of tasks of varying
cognitive load are performed after an optimal rest time between two consecutive tasks of the plurality of task has elapsed and wherein one of the two consecutive tasks is a high load task and other is a low load task.
14. The system according to claim 13 wherein the optimal rest time is calculated based on the difference between El of consecutive low load task and El for High load task.
15. A non-transitory computer readable medium storing instructions which when executed by a possessor on a system, cause the processor to perform method for measuring cognitive load comprising:
acquiring a Galvanic Skin Resistance (GSR) data from each of a one or more user performing a plurality of tasks of predefined varying cognitive load using a GSR acquisition module (210) wherein GSR acquisition module (210) receives GSR data from a GSR sensor (224);
pre-processing the acquired GSR data to remove an artifact from the acquired GSR data to generate a preprocessed GSR data for each of the one or more user using a preprocessing module (212);
extracting a plurality of features for each of the one or more users from the
preprocessed GSR data using a feature extraction module (214); selecting a most discriminative feature for each of the one or more users from the extracted plurality of features based on a discriminative index (Dl) using a feature selection module (216); and
computing a score for each of the one or more users and creating effort index (El) to measure cognitive load based on the selected most discriminative feature using an effort index generation module (218).
PCT/IB2017/050677 2016-09-02 2017-02-08 Method and system for monitoring of mental effort WO2018042261A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IN201621030176 2016-09-02
IN201621030176 2016-09-02

Publications (1)

Publication Number Publication Date
WO2018042261A1 true WO2018042261A1 (en) 2018-03-08

Family

ID=61301317

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2017/050677 WO2018042261A1 (en) 2016-09-02 2017-02-08 Method and system for monitoring of mental effort

Country Status (1)

Country Link
WO (1) WO2018042261A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110084501A (en) * 2019-04-19 2019-08-02 江苏大学 A kind of measurement method of construction risk cognitive load level
US20210393182A1 (en) * 2020-06-18 2021-12-23 Tata Consultancy Services Limited Method and system for assessment of cognitive load from bio-potentials measured using wearable endosomatic device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5724987A (en) * 1991-09-26 1998-03-10 Sam Technology, Inc. Neurocognitive adaptive computer-aided training method and system
WO2014146168A1 (en) * 2013-03-19 2014-09-25 National Ict Australia Limited Automatic detection of task transition
EP2247235B1 (en) * 2008-02-22 2016-04-13 Koninklijke Philips N.V. A system and kit for stress and relaxation management

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5724987A (en) * 1991-09-26 1998-03-10 Sam Technology, Inc. Neurocognitive adaptive computer-aided training method and system
EP2247235B1 (en) * 2008-02-22 2016-04-13 Koninklijke Philips N.V. A system and kit for stress and relaxation management
WO2014146168A1 (en) * 2013-03-19 2014-09-25 National Ict Australia Limited Automatic detection of task transition

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110084501A (en) * 2019-04-19 2019-08-02 江苏大学 A kind of measurement method of construction risk cognitive load level
US20210393182A1 (en) * 2020-06-18 2021-12-23 Tata Consultancy Services Limited Method and system for assessment of cognitive load from bio-potentials measured using wearable endosomatic device
US11872040B2 (en) 2020-06-18 2024-01-16 Tata Consultancy Services Limited Method and system for assessment of cognitive load from bio-potentials measured using wearable endosomatic device

Similar Documents

Publication Publication Date Title
Cecotti et al. A robust sensor-selection method for P300 brain–computer interfaces
Kirch et al. Detection of changes in multivariate time series with application to EEG data
CN105943013B (en) Heart rate detection method and device and intelligent wearable device
JP6849106B2 (en) Stress estimation device and stress estimation method using biological signals
EP3479758B1 (en) System and method for breathing pattern extraction from ppg signals
US20160353996A1 (en) Systems and methods for real-time signal processing and fitting
KR102043376B1 (en) Method for real time analyzing stress using deep neural network algorithm
AU2016204093B2 (en) System and method for detecting outliers in real-time for a univariate time-series signal
Cuomo et al. A novel O (n) numerical scheme for ECG signal denoising
US11354339B2 (en) Method and system for clustering users using cognitive stress report for classifying stress levels
DE112018005922T5 (en) DEVICE, METHOD AND PROGRAM FOR THE PREDICTION OF A DISEASE OUTBACK RISK
US20170086752A1 (en) Devices, systems, and methods for determining heart rate of a subject from noisy electrocardiogram data
Chandra et al. Feature extraction of ECG signal
Mohebbi et al. Structures of the recurrence plot of heart rate variability signal as a tool for predicting the onset of paroxysmal atrial fibrillation
WO2018042261A1 (en) Method and system for monitoring of mental effort
JP6243254B2 (en) Heart rate detection method and heart rate detection device
WO2018205007A1 (en) System and method for qrs complex detection in compressively sensed electrocardiogram data
Hag et al. A wearable single EEG channel analysis for mental stress state detection
EP4062827A1 (en) Patient invariant model for freezing of gait detection based on empirical wavelet decomposition
Saminu et al. Stationary wavelet transform and entropy-based features for ECG beat classification
EP3884863A1 (en) Method and system for tremor assessment using photoplethysmography (ppg)
Perinelli et al. Chasing chaos by improved identification of suitable embedding dimensions and lags
CN108201437B (en) Signal processing method and device
Pham et al. Unsupervised discrimination of motor unit action potentials using spectrograms
CN109567749B (en) Sleep data processing method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17845597

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17845597

Country of ref document: EP

Kind code of ref document: A1