KR20200038146A - The Recognition Method and Apparatus for Focus level using ECG - Google Patents

Abstract

Presented are a method and device for evaluating a concentration level using ECG. The method for evaluating a concentration level comprises the steps of: extracting an indicator of a plurality of time domains and an indicator of a plurality of frequency domains using ECG data detected from a subject; generating an integrated indicator using the indicator of the time domains and the indicator of the frequency domains; and evaluating high and low concentration of the subject by comparing a value of the integrated indicator with a rule base having an arbitrary threshold.

Description

The Recognition Method and Apparatus for Focus level using ECG}

The present disclosure relates to a method and apparatus for recognizing concentration using an electrocardiogram, and more particularly, to a method and apparatus for evaluating concentration for a specific task using a cardiac response.

Modern society is a competitive society that competes for limited resources, and the efficiency of achieving targeted results with minimal effort is an important factor. In order to increase efficiency in performing tasks, a high level of concentration is required. If all attention is focused on the task being performed, the perception of the surroundings and the passage of time are forgotten, and attention is actively generated due to interest and enjoyment of performance. Methods for measuring and evaluating concentration include subjective qualitative measures such as questionnaires and interviews, and objective quantitative measures using physiological responses. The subjective evaluation method is performed after the task is completed, so there is a time difference from the point at which concentration actually occurred, and it cannot evaluate the human emotion that changes with the passage of time. In addition, there is a disadvantage that it is difficult for the subject to accurately remember the moment when concentration occurs. Therefore, it is necessary to study how to objectively and quantitatively evaluate concentration through physiological responses. According to an existing study that evaluated the concentration of FPS games as a task, sympathetic nerves are activated and concentration and arousal are increased when concentration is achieved. It has been reported that the heart rate decreases when concentration is induced by presenting negative image stimuli. Studies have shown that the heart rate increases when concentration is caused by presenting stimuli that require cognitive load. In addition, it is reported that concentration is induced according to the level of the task. When the task level is low, indifference and relaxation are felt, and when the level is high, arousal and immersion are felt, causing concentration. Existing studies have reported only physiological pattern results of concentration in individual tasks without interaction. However, it is known that humans express their emotions in a community called society and empathize with their emotions to form relationships and communicate. At this time, the emotion caused by interaction with other people is called social emotion, and social emotion occurs in a complex way by interacting with other people as well as environmental factors such as context and context. Therefore, there is a need for research on how to objectively and quantitatively recognize concentration through pattern analysis of cardiac responses according to differences in concentration caused by interacting situations. Such research will be able to provide high-efficiency and interesting services by developing elements that can increase concentration in various industries including content.

[1] J. H. Connell, “Diversity and the coevolution of competitors, or the ghost of competition past”, Oikos, pp. 131-138, 1980. [2] B. O'regan, and M. Grfitzeli, “A low-cost, high-efficiency solar cell based on dye-sensitized”, nature, Vol. 353 No.6346, pp.737-740, 1991. [3] Kwon Seok-man. Positive Psychology: Scientific Inquiry of Happiness, Seoul: Hakjisa, 2008. [4] M. Csikszentmihalyi, Flow: The psychology of happiness, Random House, 2013. [5] Jeong Soon-cheol, Tak Gye-rae, Lee Jeong-han, Min Byeong-chan, “Integrated emotional evaluation system based on psychological and physiological evaluation,” Journal of the Korean Institute of Industrial Engineers, Vol.31, No.2, pp.127-134, 2005 . [6] SC Chung, BC Min, BW Min, YN Kim, MK Sim, and CJ Kim, “Real-time subjective sensibility assessment system using digitizer”, Journal of the Ergonomics Society of Korea, Vol. 20, No. 1, pp.1-13, 2001. [7] W. W. Wierwille, and F. T. Eggemeier, “Recommendations for mental workload measurement in a test and evaluation environment”, Human Factors, Vol. 35, No. 2, pp. 263-281, 1993. [8] BC Min, SC Chung, BW Min, MK Sim, HK Chung, and CJ Kim, “A study on positive and negative visual stimuli using the real-time human sensibility assessment system”, Journal of the Ergonomics Society of Korea, Vol. 20, No. 1, pp. 31-43, 2001. [9] D. T. Lykken, and P. H. Venables, “Direct measurement of skin conductance: A proposal for standardization”, Psychophysiology, Vol. 8, No. 5, pp. 656-672, 1971. [10] A. Lang, S. Zhou, N. Schwartz, PD Bolls, and RF Potter, “The effects of edits on arousal, attention, and memory for television messages: When an edit is an edit can an edit be too much ? ”, Journal of Broadcasting and Electronic Media, Vol. 44, No. 1, pp. 94-109, 2000. [11] PD Bolls, A. Lang, and RF Potter, “The effects of message valence and listener arousal on attention, memory, and facial muscular responses to radio advertisements”, Communication Research, Vol. 28, No. 5, pp. 627-651, 2001. [12] J. L. Lacey, “Some autonomic-central nervous system interrelationships”, Physiological correlates of emotion, pp.205-227, 1970. [13] D. Palomba, M. Sarlo, A. Angrilli, A. Mini, and L. Stegagno, “Cardiac responses associated with affective processing of unpleasant film stimuli”, International Journal of Psychophysiology, Vol. 36, No. 1, pp.45-57, 2000. [14] A. Perry, L. Stein, and S. Bentin, “Motor and attentional mechanisms involved in social interaction-Evidence from mu and alpha EEG suppression”, Neuroimage, Vol.58, No.3, pp.895-904 , 2011. [15] GD Ellis, JE Voelkl, and C. Morris, “Measurement and analysis issues with explanation of variance in daily experience using the flow model”, Journal of leisure research, Vol.26, No.4, pp.337, 1994 . [16] P. Salovey, and J. D. Mayer, “Emotional intelligence”, Imagination, cognition and personality, Vol. 9, No. 3, pp. 185-211, 1990. [17] R. J. Davidson, K. R. Sherer, and H. H. Goldsmith, Handbook of affective sciences, Oxford University Press, 2009. [18] S. Burnett, and S. J. Blakemore, “Functional connectivity during a social emotion task in adolescents and in adults”, european Journal of Neuroscience, Vol.29, No.6, pp.1294-1301, 2009. [19] KM Prkachin, RM Williams-Avery, C. Zwaal, and DE Mills, “Cardiovascular changes during induced emotion: An application of Lang's theory of emotional imagery”, Journal of Psychosomatic Research, Vol. 47, No. 3, pp .255-267, 1999. [20] J. Pan, and W. J. Tompkins, “A real-time QRS detection algorithm”. IEEE transactions on biomedical engineering, No. 3, pp. 230-236, 1985. [21] Choi Byung-moon, No Kyu-jeong, “Heart Rate Variability (HRV)”, Intravenous Anesthesia, Vol. 8, No. 2, 2004. [22] A. Malliani, M. Pagani, F. Lombardi, and S. Cerutti, “Cardiovascular neural regulation explored in the frequency domain”, Circulation, Vol. 84, No. 2, pp. 482-492, 1991. [23] M. V. Kamath, and E. L. Fallen, “Power spectral analysis of heart rate variability: a noninvasive signature of cardiac autonomic function”, Critical reviews in biomedical engineering, Vol. 21, No. 3, pp. 245-311, 1993.

According to an exemplary embodiment, a method and apparatus for objectively and quantitatively recognizing concentration through a pattern of a heart reaction according to a difference in concentration caused by an interaction situation are presented.

Method of evaluating concentration using electrocardiogram according to an exemplary embodiment:

Detecting ECG data from the subject using an ECG sensor;

Extracting a plurality of time domain indicators and a plurality of frequency domain indicators using the ECG data;

Generating an integrated indicator using the time domain indicator and the frequency domain indicator; And

And evaluating the high and low concentrations of the subject by comparing the values of the integrated indicators to a rule base having an arbitrary threshold.

According to a specific exemplary embodiment, the indicators of the plurality of time domains include SDNN, pNN50, and rMSSD, and the indicators of the plurality of frequency domains are Very Low Frequency (VLF) and Low Frequency (LF) extracted from the HRV spectrum. ), HF (high frequency) may include power values of multiple bands.

According to a specific exemplary embodiment, the HRV spectrum: may be extracted from RRI extracted from the ECG data.

According to a specific exemplary embodiment, the RRI is extracted from the ECG data by a QRS detection algorithm, and the HRV spectrum can be obtained by sampling data from the RRI and extracted by FFT analysis on the sampled data.

According to an exemplary specific embodiment, the frequency range of the HF is 0.15 ~ 04Hz,

The frequency range of the VLF may be 0.0033 to 0.04 Hz, and the frequency range of the LF may be 0.04 to 0.15 Hz.

According to a specific exemplary embodiment, the integrated indicator (Focus score) may be defined by the following equation.

According to a specific exemplary embodiment, the threshold may be set to 17.183.

An apparatus for evaluating concentration using an electrocardiogram according to an exemplary embodiment includes: an ECG sensor detecting an ECG signal from the subject; A pre-processor for pre-processing the ECG signal; And it may include; an analysis unit for evaluating the high and low concentration of the subject using the signal from the pre-processing unit.

1A is a concentrated content screen used in the experiment of the present invention.
1B is a photograph showing an experimental environment used in the experiment of the present invention.
2 is a flow chart showing an experimental procedure according to an exemplary embodiment.
Figure 3 illustrates an RRI (R-peak to R-peak Interval) extraction method according to an exemplary embodiment.
4 is a chart showing statistical analysis results for a time domain indicator.
5 is a chart showing statistical analysis results for frequency domain indicator variables.
6 is a chart showing statistical analysis results for VLF / HF and LF / HF in the frequency domain indicator variable.
7 is a chart showing the statistical analysis results of natural logarithm values (lnLF, lnHF, and lnVHF) taken from the frequency domain indicator variable.
8 is a chart showing statistical analysis results for an integrated indicator according to an exemplary embodiment.
9 shows a rule base and data patterns derived through the above process.
Fig. 10 shows the integrated rule base verification of concentration, and shows the distribution pattern of subject data.
11 is a block diagram showing a schematic configuration of an analysis apparatus according to the present invention.

Hereinafter, a method and apparatus for recognizing concentration according to exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

According to an exemplary embodiment, it is possible to implement a method and apparatus for objectively and quantitatively recognizing concentration through a pattern of a heart reaction according to a difference in concentration caused by an interaction situation. According to this exemplary embodiment of the present invention, it is possible to provide a service of high efficiency and interest by developing elements capable of increasing concentration in various industries including content.

The present invention extracts raw data for evaluating a subject's concentration using an ECG sensor, and the data is processed by a processing device. The processing device has an analysis tool or software and a hardware system on which it is executed. Such a processing device may be a computer-based device, a general purpose computer including a software containing algorithms and hardware capable of driving the software, or a dedicated device.

The processing result from the above processing device may be displayed by the display device, and may further include a general external interface device, for example, a keyboard, a mouse, and the like, as input means.

11 is a block diagram showing a schematic configuration of the analysis device as described above.

The ECG sensor 110 is attached to the body of the subject 100, the sensor 110 is connected to the pre-processing unit 120, and the pre-processing unit 130 uses data from the subject such as high concentration or low concentration. It is connected to the analysis unit 130 that evaluates the concentration. The result generated by the analysis unit 130 is displayed by the display 140.

The following experiments were carried out for the objective evaluation of the concentration recognition method and the device to which it is applied according to the present invention.

<Experiment participants>

The subjects who participated in this experiment were 60 healthy physical subjects (30 male and female, 23.88 ± 2.09 years old, average age) without a history of autonomic nervous system. On the day before the experiment, sufficient sleep was requested to minimize fatigue, and caffeine and alcohol intake, which could affect the autonomic nervous system, were prohibited. To increase the participation in the experiment, the subject's cost was paid, and all the experiments were conducted under the deliberation of the Sangmyung University Ethics Committee.

<Experimental design and stimulus composition>

1A is a concentrated content screen used in the experiment of the present invention, and FIG. 1B is a photograph showing an experiment environment used in the experiment of the present invention.

Intensive experimental content as a stimulus is a game that targets with arrows, and is designed to enable low- and high-concentration levels using the Unity game engine (Unity Technologies, USA). Experimental content was produced with the difference in level by reflecting the research results that there was a difference in concentration induction according to the level of the task. The low concentration stage only needs to match the power gauge of the arrow, but the high concentration stage is made by giving the difference in the difficulty of matching the power gauge and the target position at the same time. The arrows were shot alternately once by two subjects, and the hit rate was displayed at the top left of the screen to induce concentration. In addition, a social relationship was formed between the two subjects by presenting the goals that the joint must achieve while performing the task. The detailed intensive experiment content screen and experiment environment are shown in FIG. 1.

2 is a flow chart showing an experimental procedure according to an exemplary embodiment. After wearing the electrocardiogram sensor (Sensor attachment), before performing the intensive task (Focus task), the experiment method was introduced (Introduction for task), and the manipulation practice (Pre-test for training) was performed to perform the task smoothly. In the focused task, the biosignal reference was measured in the first 3 minutes of the start, and the focused task (Focus Tack) was performed in the next 3 minutes. In order to eliminate the order effect, the order of the low task (low focus tack) and the high task (high focus task) was selected as random (Random order). The questionnaire was conducted after the task ended with the removal of the sensor.

Data collection and signal processing were performed as follows. Electrocardiogram (ECG) was measured with Lead I of the standard limb induction method. The raw signal thus obtained is pre-processed by a pre-processing part, for example, MP 100 power supply (Biopac System Inc., USA), ECG 100C amplifier, NI-DAQ-Pad9205 (National Instrument) Inc., USA). A circular signal from the sensor is amplified and a digitized ECG signal with a sampling frequency of 500 Hz can be obtained through the pre-processing unit including these elements.

For the ECG signal obtained through the pre-processing process, the R peak was calculated through the QRS detection algorithm to calculate the R-peak to R-peak interval (RRI).

Figure 3 illustrates an RRI (R-peak to R-peak Interval) extraction method through the QRS detection algorithm from ECG. 3, RRI was extracted from the ECG signal through a QRS detection algorithm.

Time domain data such as SDNN (standard deviation normal to normal) and pNN50 (Percentage of RR intervals that differ more than 50 ms), rMSSD (Root Mean Square of Successive Differences) were calculated using RRI, and RRI was 2 Hz. The heart rate variability (HRV) spectrum for extracting data in the frequency domain was extracted through fast fourier transform (FFT) analysis by sampling.

In the HRV spectrum, powers of VLF (very low frequency, 0.0033-0.04 Hz), LF (low frequency, 0.04-0.15 Hz) and HF (high frequency, 0.15-0.4 Hz) bands were extracted, and frequency domain indicators VLF / HF and LF / HF, lnVLF, lnLF, and lnHF were calculated. A total of 12 parameters were extracted through the electrocardiogram, and LabVIEW 2015 (National Instrument Inc., USA) was used for this process. This analysis process is performed by a computer-based analysis device on which the analysis software is executed.

After collecting the data as above, statistical significance test was performed. Statistical significance test according to concentration was analyzed by independent sample t-test using SPSS 21 (IBM, USA). 4 is a chart showing statistical analysis results for a time domain indicator.

As a result of analysis, the high concentration showed a high pattern as shown below in the values (M) of all time domain indicators of RRI, SDNN, rMSSD, and pNN50 than the low concentration, and statistically significant results were confirmed.

RRI :

Low focus: M = 648.54 ± 47.20

High focus: M = 783.88 ± 73.39

t (58) = -8.495, p = .000

SDNN :

Low concentration: M = 30.66 ± 11.47

High concentration: M = 44.41 ± 12.80

t (58) = -4.380, p = .000

rMSSD :

Low concentration: M = 25.45 ± 0.92

High concentration: M = 27.97 ± 1.32

t (58) = -8.587, p = .000

pNN50 :

Low concentration: M = 3.89 ± 5.67

High concentration: M = 23.34 ± 14.86

t (58) = -6.697, p = .000

5 is a chart showing statistical analysis results for frequency domain indicator variables.

As shown in FIG. 5, concentrations higher than low concentrations showed a large pattern in LF and HF values, and statistically significant results were confirmed. And, VLF also showed a greater pattern of higher concentration, such as LF and HF, but could not confirm statistically significant results.

LF :

Low concentration: M = 222.29 ± 121.77

High concentration: M = 467.76 ± 289.83

t (58) = -4.284, p = .000

HF :

Low concentration: M = 161.80 ± 166.76

High concentration: M = 309.57 ± 211.25

 t (58) = -3.007, p = .004

VLF :

Low concentration: M = 396.99 ± 339.53

High concentration: M = 637.88 ± 599.00

t (58) = -1.916, p = .060

6 is a chart showing statistical analysis results for VLF / HF and LF / HF in the frequency domain indicator variable.

The value of VLF / HF showed a pattern with a higher concentration than a lower concentration, but a statistically significant result could not be confirmed. And the LF / HF value showed a pattern with a higher concentration than a lower concentration, and no statistically significant results could be confirmed.

VLF / HF :

Low concentration: M = 0.605 ± 0.603

High concentration: M = 0.816 ± 0.729

t (58) = -1.222, p = .227

LF / HF :

Low concentration: M = 3.071 ± 6.385

High concentration: M = 2.101 ± 1.710

t (58) = 0.804, p = .425

7 is a chart showing the statistical analysis results of natural logarithm values (lnLF, lnHF, and lnVHF) taken from the frequency domain indicator variable.

Referring to FIG. 7, the high concentration showed a larger pattern of both lnLF and lnHF than the low concentration, and statistically significant results were confirmed. lnVLF showed the same pattern, but no statistically significant results could be confirmed.

lnLF :

Low concentration: M = 4.67 ± 0.99

High concentration: M = 5.49 ± 0.79

t (58) = -3.538, p = .001

lnHF :

Low concentration: M = 5.26 ± 0.57

High concentration: M = 6.09 ± 0.51

t (58) = -5.937, p = .000)

lnVLF :

Low concentration: M = 5.610 ± 0.906

High concentration: M = 6.060 ± 0.925

t (58) = -1.905, p = .062)

Based on the above statistical analysis results, an integrated rule-base was derived and verified through time and frequency domain indicators.

As mentioned above, as a result of statistical analysis of the time and frequency domain indicators, statistically significant results were confirmed, and RRI, rMSSD, and lnHF with a large difference in pattern between low concentration and high concentration were selected. The selected indicators had a preliminary work to set the unit to a similar level because there is a unit difference in value. The RRI was divided by 100, then the natural logarithm was taken to calculate ln (RRI / 100), and the rMSSD took the natural logarithm to calculate lnrMSSD to integrate indicators. The three indicators showed a pattern with higher concentrations than lower concentrations, and all were combined to create a new integrated indicator (Focus Score). The detailed formula is as shown in Equation 1 below.

As a result of statistical analysis of the integrated indicator (Focus Score), high concentration showed a greater pattern than low concentration (low concentration: M = 16.41 ± 0.65, high concentration: M = 18.73 ± 0.85), and statistically significant results were confirmed ( t (58) = -11.903, p = .000). 8 is a chart showing the statistical analysis results for the integrated indicator.

9 shows a rule base and data patterns derived through the above process.

As shown in Fig. 9, the data pattern according to the low concentration and high concentration was confirmed by plotting the value of the integrated indicator, and the rule base threshold value "17.183" that distinguished the two concentrations with a linear linear support vector machine (SVM) was determined. Derived.

That is, if the integrated indicator value is greater than the rule base threshold value of 17.183, it can be judged to indicate a high concentration, and a smaller, lower concentration.

In order to verify the integrated rule base as above, 30 people who were not used to derive the rule base were used.

As a result, in the case of high concentration, all 30 samples were accurately classified, and the recognition accuracy was confirmed as (30/30) × 100 = 100%. In the case of low concentration, 3 out of 30 samples showed errors that were classified as high concentration, so the recognition accuracy was confirmed as (27/30) × 100 = 90%. As a result of the integrated rule base verification of concentration, 57 samples out of a total of 60 samples were accurately classified, and the overall recognition accuracy was confirmed to be (57/60) × 100 = 95%. Fig. 10 shows the integrated rule base verification of concentration, and shows the distribution pattern of subject data. As shown in FIG. 10, data distributed thereon based on the rule base threshold 17.183 indicates high concentration, and data distributed below it indicates low concentration.

The purpose of this experiment is to develop a method of objectively and quantitatively recognizing concentration through a pattern of cardiac responses according to differences in concentration caused by an interaction situation.

The content of the concentration experiment consisted of a game that targets with arrows, and a difference in difficulty was given to give a difference in concentration. In addition, by presenting the goals that the community must achieve, a social relationship was formed between the two subjects.

As described in detail above, according to the experiment of the present invention, statistical differences were significant according to concentration differences in RRI, SDNN, rMMSD, and pNN50, which are time domain indicators extracted from ECG. In the frequency domain indicator, LF, HF, lnLF, and lnHF were statistically significant according to the difference in concentration. Among them, RRI, rMSSD, and lnHF with large differences in patterns were selected according to the difference in concentration to generate an integrated indicator. As a result of verifying the rule base derived through the integrated indicator through 60 data samples of low concentration and high concentration, (57/60) × 100 = 95% showed high recognition accuracy.

The RRI, selected as an indicator of the integrated rule base, contains information on the heart rate cycle at intervals between the R peak and the R peak. At a higher concentration than a lower concentration, the RRI value showed a large pattern, which can be interpreted as an increase in the heart rate due to a cognitive load because the target location and the power gauge of the arrow had to be considered at the same time. rMSSD is the square root value of the squared mean of the difference in heart rate intervals, and represents change in heart rate. At a higher concentration than a lower concentration, the rMSSD value showed a large pattern, which can be interpreted as a significant change in heart interval due to difficult difficulty. lnHF is an indicator of activation of the parasympathetic nerve, and it can be interpreted that the stimulus content of this experiment showed a large pattern at high concentration because it was an experiment requiring static concentration such as cognitive load.

In a competitive society that competes to occupy limited resources with each other, the concentration that increases the efficiency of tasks is important in all industries and services. Therefore, in the study of the present invention, an objective and quantitative method of recognizing concentration through a pattern of heart reaction according to a difference in concentration caused by an interaction situation was developed, and a method of recognizing concentration showing high recognition accuracy was proposed. . It is considered that if the concentration recognition method proposed in the present invention is applied to various fields as well as service fields including content, it is considered that the efficiency and interest will be improved by developing factors that can increase the concentration. Further studies may be possible with other physiological reactions in the future.

Although the present invention has been described with reference to the embodiments shown in the drawings, these are merely exemplary, and those skilled in the art can understand that various modifications and other equivalent embodiments are possible therefrom. Therefore, the true protection scope of the present invention should be defined only by the appended claims.

Claims (10)

Detecting ECG data from the subject using an ECG sensor;
Extracting a plurality of time domain indicators and a plurality of frequency domain indicators using the ECG data;
Generating an integrated indicator using the time domain indicator and the frequency domain indicator;
Comprising the step of evaluating the high concentration and low concentration of the subject by comparing the value of the integrated indicator to a rule base having an arbitrary threshold; concentration evaluation method using an electrocardiogram. According to claim 1,
Indicators of the plurality of time domains include SDNN, pNN50, rMSSD,
The indicator of the plurality of frequency domains includes power values of multiple bands including a very low frequency (VLF), a low frequency (LF), and a high frequency (HF) extracted from the HRV spectrum. According to claim 2,
The HRV spectrum: is extracted from the RRI extracted from the ECG data, concentration evaluation method using an electrocardiogram. According to claim 3,
The RRI is extracted from the ECG data by a QRS detection algorithm, and
The HRV spectrum is obtained by sampling the data from the RRI and extracted by FFT analysis of the sampling data, concentration evaluation method using an electrocardiogram. According to claim 2,
The frequency range of the HF is 0.15 ~ 04Hz,
The frequency range of the VLF is 0.0033 ~ 0.04Hz, and
The frequency range of the LF is 0.04 ~ 0.15Hz, concentration evaluation method using an electrocardiogram. According to claim 2 to claim 5,
The integrated indicator (Focus score) is defined by the following equation, a method for evaluating concentration using an electrocardiogram.

The method of claim 6,
The threshold is 17.183, the method of evaluating concentration using an electrocardiogram. An apparatus for performing the method of any one of claims 1 to 5,
An ECG sensor that detects an ECG signal from the subject;
A pre-processor for pre-processing the ECG signal; And
Concentration evaluation apparatus using an electrocardiogram, including; analysis unit for evaluating the high and low concentration of the subject using the signal from the pre-processing unit. The method of claim 8,
The analysis unit using an integrated indicator (Focus score) defined by the following equation, concentration evaluation device using an electrocardiogram.

The method of claim 9,
The threshold is 17.183, the concentration evaluation apparatus using an electrocardiogram.

Images