KR102214460B1 - The Recognition Method and Apparatus for Focus level using ECG - Google Patents

Abstract

A method and apparatus for evaluating concentration using an electrocardiogram are presented. The concentration evaluation method comprises: extracting a plurality of time domain indicators and a plurality of frequency domain indicators using ECG data detected from a subject; Generating an integrated indicator using the time domain indicator and the frequency domain indicator; And evaluating the high concentration and low concentration of the subject by comparing the value of the integrated indicator to a rule base having an arbitrary threshold.

Description

The Recognition Method and Apparatus for Focus level using ECG}

The present disclosure relates to a method and apparatus for recognizing concentration using an electrocardiogram, and more particularly, to a method and apparatus for evaluating concentration for a specific task using a cardiac response.

Modern society is a competitive society that competes for limited resources, and the efficiency of obtaining the target result with minimal effort is an important factor. A high level of concentration is required to increase efficiency in performing tasks. If all attention is focused on the task performed, the perception of the surroundings and the passage of time are forgotten, and concentration is actively generated due to the interest and enjoyment of performance. Methods of measuring and evaluating concentration include subjective qualitative measurement methods such as questionnaires and interviews, and objective quantitative measurement methods using physiological responses. Since the subjective evaluation method is performed after the task is completed, there is a time difference from the point in time when concentration actually occurs, and it cannot evaluate human emotions that change with the passage of time. In addition, there is a disadvantage that it is difficult for the subject to accurately remember the moment when concentration occurs. Therefore, there is a need for research on a method of objectively and quantitatively evaluating concentration through physiological responses. According to an existing study that evaluated the degree of concentration using the FPS game as a task, it was reported that the sympathetic nerves are activated when concentration increases the level of arousal and positivity. It was reported that the heart rate decreased when concentration was induced by presenting a negative image stimulus. Research has shown that the heart rate increases when concentration is induced by presenting a stimulus that requires cognitive load. It is also reported that concentration is induced depending on the level of the task. When the level of the task is low, a feeling of indifference and relaxation is felt, and when the level of the task is high, a sense of arousal and immersion is felt, causing concentration. Previous studies reported only the physiological pattern results of concentration in individual tasks that did not include interaction. However, humans are known to express their emotions in the community of society and form relationships and communicate while empathizing with the other's emotions. At this time, the sensibility arising from interactions with other people is called social sensibility, and social sensibility arises complexly by interacting not only with interactions with other people, but also with environmental factors such as context and context. Therefore, there is a need for a study on a method of objectively and quantitatively recognizing the degree of concentration through the analysis of the pattern of the heart response according to the difference in concentration caused in an interactive situation. Such research will be able to provide high-efficiency and interesting services by developing elements that can increase concentration in various industrial fields including contents.

[1] J. H. Connell, “Diversity and the coevolution of competitors, or the ghost of competition past”, Oikos, pp.131-138, 1980. [2] B. O'regan, and M. Grfitzeli, “A low-cost, high-efficiency solar cell based on dye-sensitized”, nature, Vol.353 No.6346, pp.737-740, 1991. [3] Kwon Sukman. Positive Psychology: Scientific Exploration of Happiness, Seoul: Hakjisa, 2008. [4] M. Csikszentmihalyi, Flow: The psychology of happiness, Random House, 2013. [5] Soon-cheol Jeong, Gye-rae Tak, Jeong-han Lee, Byeong-chan Min, “Integrated emotional evaluation system based on psychological and physiological evaluation”, Journal of the Korean Institute of Industrial Engineers, Vol.31, No.2, pp.127-134, 2005 . [6] SC Chung, BC Min, BW Min, YN Kim, MK Sim, and CJ Kim, “Real-time subjective sensibility assessment system using digitizer”, Journal of the Ergonomics Society of Korea, Vol.20, No.1, pp.1-13, 2001. [7] W. W. Wierwille, and F. T. Eggemeier, “Recommendations for mental workload measurement in a test and evaluation environment”, Human Factors, Vol.35, No.2, pp.263-281, 1993. [8] BC Min, SC Chung, BW Min, MK Sim, HK Chung, and CJ Kim, “A study on positive and negative visual stimuli using the real-time human sensibility assessment system”, Journal of the Ergonomics Society of Korea, Vol. 20, No. 1, pp. 31-43, 2001. [9] D. T. Lykken, and P. H. Venables, “Direct measurement of skin conductance: A proposal for standardization”, Psychophysiology, Vol.8, No.5, pp.656-672, 1971. [10] A. Lang, S. Zhou, N. Schwartz, PD Bolls, and RF Potter, “The effects of edits on arousal, attention, and memory for television messages: When an edit is an edit can an edit be too much ?”, Journal of Broadcasting and Electronic Media, Vol. 44, No. 1, pp. 94-109, 2000. [11] PD Bolls, A. Lang, and RF Potter, “The effects of message valence and listener arousal on attention, memory, and facial muscular responses to radio advertisements”, Communication Research, Vol. 28, No. 5, pp. 627-651, 2001. [12] J. L. Lacey, “Some autonomic-central nervous system interrelationships”, Physiological correlates of emotion, pp. 205-227, 1970. [13] D. Palomba, M. Sarlo, A. Angrilli, A. Mini, and L. Stegagno, “Cardiac responses associated with affective processing of unpleasant film stimuli”, International Journal of Psychophysiology, Vol. 36, No. 1, pp. 45-57, 2000. [14] A. Perry, L. Stein, and S. Bentin, “Motor and attentional mechanisms involved in social interaction-Evidence from mu and alpha EEG suppression”, Neuroimage, Vol.58, No.3, pp.895-904 , 2011. [15] GD Ellis, JE Voelkl, and C. Morris, “Measurement and analysis issues with explanation of variance in daily experience using the flow model”, Journal of leisure research, Vol.26, No.4, pp.337, 1994 . [16] P. Salovey, and J. D. Mayer, “Emotional intelligence”, Imagination, cognition and personality, Vol.9, No.3, pp.185-211, 1990. [17] R. J. Davidson, K. R. Sherer, and H. H. Goldsmith, Handbook of affective sciences, Oxford University Press, 2009. [18] S. Burnett, and S. J. Blakemore, “Functional connectivity during a social emotion task in adolescents and in adults”, european Journal of Neuroscience, Vol.29, No.6, pp.1294-1301, 2009. [19] KM Prkachin, RM Williams-Avery, C. Zwaal, and DE Mills, “Cardiovascular changes during induced emotion: An application of Lang's theory of emotional imagery”, Journal of Psychosomatic Research, Vol.47, No.3, pp. .255-267, 1999. [20] J. Pan, and W. J. Tompkins, “A real-time QRS detection algorithm”. IEEE transactions on biomedical engineering, No.3, pp.230-236, 1985. [21] Byung-moon Choi, Gyeong-gyu No, “Heart Rate Variability (HRV)”, intravenous anesthesia, Vol.8, No.2, 2004. [22] A. Malliani, M. Pagani, F. Lombardi, and S. Cerutti, “Cardiovascular neural regulation explored in the frequency domain”, Circulation, Vol.84, No.2, pp.482-492, 1991. [23] M. V. Kamath, and E. L. Fallen, “Power spectral analysis of heart rate variability: a noninvasive signature of cardiac autonomic function”, Critical reviews in biomedical engineering, Vol.21, No.3, pp.245-311, 1993.

According to an exemplary embodiment, a method and apparatus for objectively and quantitatively recognizing the degree of concentration through a pattern of a cardiac response according to a difference in concentration caused in an interactive situation is provided.

Concentration evaluation method using an electrocardiogram according to an exemplary embodiment:

Detecting ECG data from a subject using an ECG sensor;

Extracting a plurality of time domain indicators and a plurality of frequency domain indicators using the ECG data;

Generating an integrated indicator using the time domain indicator and the frequency domain indicator; And

And evaluating the high concentration and low concentration of the subject by comparing the value of the integrated indicator to a rule base having an arbitrary threshold.

According to an exemplary specific embodiment, the indicators of the plurality of time domains include SDNN, pNN50, and rMSSD, and the indicators of the plurality of frequency domains are very low frequency (VLF) and low frequency (LF) extracted from the HRV spectrum. ), may include a power value of a plurality of bands including a high frequency (HF).

According to an exemplary specific embodiment, the HRV spectrum: may be extracted from the RRI extracted from the ECG data.

According to an exemplary specific embodiment, the RRI may be extracted from the ECG data by a QRS detection algorithm, and the HRV spectrum may be sampled from the RRI and extracted by FFT analysis on the sampled data.

According to a specific exemplary embodiment, the frequency range of the HF is 0.15 ~ 04Hz,

The frequency range of the VLF may be 0.0033 to 0.04 Hz, and the frequency range of the LF may be 0.04 to 0.15 Hz.

According to an exemplary specific embodiment, the integrated indicator (Focus score) may be defined by the following equation.

According to an exemplary specific embodiment, the threshold value may be set to 17.183.

Concentration evaluation device using an electrocardiogram according to an exemplary embodiment: an ECG sensor for detecting an ECG signal from the subject; A preprocessor for preprocessing the ECG signal; And an analysis unit for evaluating high concentration and low concentration of the subject by using the signal from the preprocessor.

1A is a screen of concentrated content used in the experiment of the present invention.
1B is a photograph showing an experimental environment used in the experiment of the present invention.
2 is a flow chart showing an experimental procedure according to an exemplary embodiment.
3 illustrates an RRI (R-peak to R-peak Interval) extraction method according to an exemplary embodiment.
4 is a chart showing a statistical analysis result of a time domain indicator.
5 is a chart showing statistical analysis results for a frequency domain indicator variable.
6 is a chart showing statistical analysis results for VLF/HF and LF/HF in the frequency domain indicator variable.
7 is a chart showing the statistical analysis results of natural log values (lnLF, lnHF, lnVHF) taken from the frequency domain indicator variable.
8 is a chart showing a result of statistical analysis of an integrated indicator according to an exemplary embodiment.
9 shows a rule base and a data pattern derived through the above process.
Fig. 10 shows the integrated rule base verification of concentration and shows the distribution pattern of subject data.
11 is a block diagram showing a schematic configuration of an analysis device according to the present invention.

Hereinafter, a method and apparatus for recognizing concentration according to an exemplary embodiment of the present invention will be described in detail with reference to the accompanying drawings.

According to an exemplary embodiment, it is possible to implement a method and apparatus for objectively and quantitatively recognizing the degree of concentration through a pattern of a cardiac response according to a difference in concentration caused in an interactive situation. According to the exemplary embodiment of the present invention, elements that can increase concentration in various industrial fields including content are developed, thereby enabling the provision of services of high efficiency and interest.

The present invention extracts raw data for evaluating a subject's concentration using an ECG sensor, and the data is processed by a processing device. The processing device has a tool or software for analysis and a hardware system on which it is executed. Such a processing device may be a computer-based device, a general-purpose computer or a dedicated device including software containing algorithms and hardware capable of running the software.

The processing result from the processing device as described above may be displayed by a display device, and may further include a general external interface device such as a keyboard and a mouse as input means.

11 is a block diagram showing a schematic configuration of the analysis device as described above.

The ECG sensor 110 is attached to the body of the subject 100, and the sensor 110 is connected to the pre-processing unit 120, and the pre-processing unit 130 uses data from the subject to provide high concentration or low concentration. It is connected to the analysis unit 130 that evaluates the degree of concentration. The result generated by the analysis unit 130 is displayed on the display 140.

The following experiment was conducted for objective evaluation of the concentration recognition method according to the present invention and a device applying the same.

<Experiment Participants>

The subjects who participated in this experiment were 60 healthy subjects (30 males and females each, average age 23.88 ± 2.09 years old) without a history of the autonomic nervous system. Fatigue was minimized by requesting sufficient sleep the day before the experiment, and caffeine and alcohol consumption, which may affect the autonomic nervous system, were prohibited. In order to increase participation in the experiment, subject fees were paid, and all experiments were conducted under the deliberation of the Sangmyung University Ethics Committee.

<Experiment design and stimulation composition>

Fig. 1A is a screen of concentrated content used in the experiment of the present invention, and Fig. 1B is a photograph showing an experimental environment used in the experiment of the present invention.

Intensive experiment content as a stimulus is a game in which targets are matched with arrows, and it is designed to enable games with low concentration and high concentration levels using the Unity game engine (Unity Technologies, USA). Experimental content was produced by setting differences in levels by reflecting the research results that there is a difference in concentration induction according to the level of the task. The low concentration stage only needs to match the arrow's power gauge, but the high concentration stage is made by giving the difference in difficulty of matching the power gauge and target position at the same time. The arrows were shot by two subjects alternately, and the accuracy was displayed in the upper left corner of the screen to induce concentration. In addition, a social relationship was formed between the two subjects by presenting the goals that the joint should achieve during the task. The detailed intensive experiment content screen and experiment environment are shown in FIG. 1.

2 is a flow chart showing an experimental procedure according to an exemplary embodiment. After wearing the ECG sensor (Sensor attachment), before performing the focus task, an introduction for task was given and pre-test for training was performed. In the Focus task, a biosignal reference was measured for 3 minutes at the beginning of the start, and a Focus Tack was performed for 3 minutes. In order to remove the order effect, the order of the low focus tack and the high focus task was selected as a random order. The questionnaire was conducted after the task was finished with the remove sensor.

Data collection and signal processing were performed as follows. Electrocardiogram (ECG) was measured with Lead I of the standard limb guidance method. The raw signal obtained in this way is pre-processed by a pre-processing part. For example, MP 100 power supply (Biopac System Inc., USA), ECG 100C amplifier, NI-DAQ-Pad9205 (National Instrument) Inc., USA), etc. may be used. Through the preprocessing unit including these elements, a circular signal from the sensor is amplified and a digitized ECG signal can be obtained with a sampling frequency of 500 Hz.

For the ECG signal obtained through the pre-processing process, the R peak was calculated through the QRS detection algorithm, and RRI (R-peak to R-peak interval) was calculated.

3 illustrates a method for extracting RRI (R-peak to R-peak Interval) from ECG through a QRS detection algorithm. As shown in Figure 3, the RRI was extracted from the ECG signal through the QRS detection algorithm.

Time domain data such as SDNN (standard deviation normal to normal), pNN50 (Percentage of RR intervals that differ more than 50ms) and rMSSD (Root Mean Square of Successive Differences) were calculated using RRI, and RRI was set to 2Hz. By sampling, a heart rate variability (HRV) spectrum for extracting data in the frequency domain was extracted through fast fourier transform (FFT) analysis.

HRV spectrum was extracted from the power of the VLF (very low frequency, 0.0033-0.04 Hz), LF (low frequency, 0.04-0.15 Hz) and HF (high frequency, 0.15-0.4 Hz) bands, respectively, and from these, a frequency domain indicator VLF/HF, LF/HF, lnVLF, lnLF, and lnHF were calculated. There were a total of 12 parameters extracted through the electrocardiogram, and LabVIEW 2015 (National Instrument Inc., USA) was used for this process. This analysis process is performed by a computer-based analysis device running analysis software.

After collecting the data as above, statistical significance was verified. Statistical significance according to concentration was analyzed by independent sample t-test using SPSS 21 (IBM, USA). 4 is a chart showing a statistical analysis result of a time domain indicator.

As a result of the analysis, the high concentration showed higher patterns in the values (M) of all time domain indicators of RRI, SDNN, rMSSD, and pNN50 than low concentration, and statistically significant results were confirmed.

RRI :

Low focus: M = 648.54±47.20

High focus: M = 783.88±73.39

t(58) = -8.495, p = .000

SDNN :

Low concentration: M = 30.66±11.47

High concentration: M = 44.41±12.80

t(58) = -4.380, p = .000

rMSSD :

Low concentration: M = 25.45±0.92

High concentration: M = 27.97±1.32

t(58) = -8.587, p = .000

pNN50 :

Low concentration: M = 3.89±5.67

High concentration: M = 23.34±14.86

t(58) = -6.697, p = .000

5 is a chart showing statistical analysis results for a frequency domain indicator variable.

As shown in Fig. 5, the higher concentration than the lower concentration showed a larger pattern in the LF and HF values, and statistically significant results were confirmed. In addition, VLF also showed a higher concentration pattern, such as LF and HF, but statistically significant results could not be confirmed.

LF :

Low concentration: M = 222.29±121.77

High concentration: M = 467.76±289.83

t(58) = -4.284, p = .000

HF :

Low concentration: M = 161.80±166.76

High concentration: M = 309.57±211.25

t(58) = -3.007, p = .004

VLF :

Low concentration: M = 396.99±339.53

High concentration: M = 637.88±599.00

t(58) = -1.916, p = .060

6 is a chart showing statistical analysis results for VLF/HF and LF/HF in the frequency domain indicator variable.

As for the value of VLF/HF, high concentration showed a larger pattern than low concentration, but statistically significant results could not be confirmed. And, as for the value of LF/HF, high concentration showed a smaller pattern than low concentration, and statistically significant results could not be confirmed.

VLF/HF :

Low concentration: M = 0.605±0.603

High concentration: M = 0.816±0.729

t(58) = -1.222, p = .227

LF/HF :

Low concentration: M = 3.071±6.385

High concentration: M = 2.101±1.710

t(58) = 0.804, p = .425

7 is a chart showing the results of statistical analysis of natural log values (lnLF, lnHF, lnVHF) taken from a frequency domain indicator variable.

Referring to FIG. 7, both lnLF and lnHF values of high concentration showed a larger pattern than low concentration, and statistically significant results were confirmed. lnVLF also showed the same pattern, but statistically significant results could not be confirmed.

lnLF :

Low concentration: M = 4.67±0.99

High concentration: M = 5.49±0.79

t(58) = -3.538, p = .001

lnHF :

Low concentration: M = 5.26±0.57

High concentration: M = 6.09±0.51

t(58) = -5.937, p = .000)

lnVLF :

Low concentration: M = 5.610±0.906

High concentration: M = 6.060±0.925

t(58) = -1.905, p = .062)

Based on the statistical analysis results as described above, an integrated rule base was derived and verified through time and frequency domain indicators.

As mentioned above, statistical analysis results of time and frequency domain indicators confirmed statistically significant results, and RRI, rMSSD, and lnHF with large differences in patterns of low concentration and high concentration were selected. Since there is a difference in units of values for the selected indicators, preliminary work was performed to set the units to a similar level. RRI was divided by 100 and then the natural log was taken to calculate ln(RRI/100), and rMSSD was taken the natural log and lnrMSSD was calculated to integrate the indicator. The three indicators showed a pattern in which high concentration was larger than that of low concentration, and all were added together to create a new integrated indicator (Focus Score). The detailed formula is shown in Equation 1 below.

As a result of statistical analysis of the integrated indicator (Focus Score), high concentration showed a larger pattern than low concentration (low concentration: M = 16.41±0.65, high concentration: M = 18.73±0.85), and statistically significant results were confirmed ( t(58) = -11.903, p = .000). 8 is a chart showing a statistical analysis result of the integrated indicator.

9 shows a rule base and a data pattern derived through the above process.

As shown in Fig. 9, the data pattern according to the low concentration and the high concentration was confirmed by plotting the value of the integrated indicator, and a threshold value "17.183" of the rule base that divides the two concentrations using a linear support vector machine (SVM) is determined. Was derived.

That is, if the integrated indicator value is greater than the rule base threshold value of 17.183, it may be determined that it indicates high concentration, less than that, and indicates low concentration.

In order to verify the above integrated rule base, the data of 30 people who were not used to derive the rule base were used.

As a result, in the case of high concentration, all 30 samples were accurately classified, and the recognition accuracy was confirmed as (30/30)×100 = 100%. In the case of low concentration, 3 of the 30 samples showed an error that was classified as high concentration, so the recognition accuracy was confirmed as (27/30)×100 = 90%. As a result of the verification of the integrated rule base of concentration, 57 samples were accurately classified out of a total of 60 samples, and the overall recognition accuracy was confirmed as (57/60)×100 = 95%. Fig. 10 shows the integrated rule base verification of concentration and shows the distribution pattern of subject data. As shown in Fig. 10, based on the rule base threshold value of 17.183, data distributed above it indicates high concentration, and data distributed below it indicates low concentration.

This experiment is to develop a method of recognizing the degree of concentration objectively and quantitatively through the pattern of the cardiac response according to the difference in concentration caused in an interactive situation.

The contents of the intensive experiment consisted of a game in which targets were matched with arrows, and a difference in difficulty was given to give a difference in concentration. In addition, a social relationship was formed between the two subjects by presenting the goals that the joint should achieve.

As described in detail above, according to the experiment of the present invention, the difference was statistically significant according to the difference in concentration in the time domain indicators RRI, SDNN, rMMSD, and pNN50 extracted from the electrocardiogram. In the frequency domain indicator, LF, HF, lnLF, and lnHF were statistically significant according to the difference in concentration. Among them, RRI, rMSSD, and lnHF with a large difference in pattern were selected according to the difference in concentration to create an integrated indicator. As a result of verifying the rule base derived through the integrated indicator through 60 data samples of low concentration and high concentration, high recognition accuracy was shown as (57/60)×100 = 95%.

RRI, selected as an indicator of the integrated rule base, contains information on the period of the heart rate as the interval between the R peak and the R peak. At higher concentrations than lower concentrations, the RRI value showed a larger pattern. This can be interpreted as an increase in the heart rate due to the occurrence of cognitive load because the target position and the arrow power gauge must be considered at the same time. rMSSD is the square root value of the square mean of the difference between the heart rate intervals and represents the change information of the heart rate. At higher concentrations than at lower concentrations, a pattern with higher rMSSD values was shown, which can be interpreted as a significant change in cardiac interval due to difficult difficulty. lnHF is an activation index of the parasympathetic nerve, and it can be interpreted that the stimulus content of this experiment was an experiment that required static concentration such as cognitive load, so it showed a large pattern at high concentration.

In a competitive society that competes for each other for limited resources, the concentration that increases the efficiency of tasks plays an important role in all industries and service sectors. Therefore, in the research of the present invention, a method of objectively and quantitatively recognizing the degree of concentration through the pattern of the heart response according to the difference in concentration caused in an interactive situation was proposed, and a method of recognizing the degree of concentration with high recognition accuracy was proposed. . If the concentration recognition method proposed in the present invention is applied to various fields as well as service fields including contents, it is considered that efficiency and interest will be improved by developing elements that can increase concentration. Further studies are possible with other physiological reactions later.

The present invention has been described with reference to the embodiments shown in the drawings, but these are merely exemplary, and those of ordinary skill in the art will appreciate that various modifications and other equivalent embodiments are possible therefrom. Therefore, the true scope of protection of the present invention should be determined only by the appended claims.

Claims (10)

Detecting ECG data from the subject using an ECG sensor;
Digitizing the ECG data by a preprocessor;
An analysis step of generating an integrated indicator based on an indicator in a time domain and an indicator in a frequency domain from the ECG data with a computer-based analysis device to evaluate the concentration of the subject; including,
Analysis step by the analysis device:
Extracting RRI (R-peak to R-peak Interval) from the digitized ECG data;
Extracting a heart rate variability (HRV) spectrum from the RRI;
Extracting a time domain indicator including rMSSD (Root Mean Square of Successive Differences) using the ECG data and a frequency domain indicator including a power value of a high frequency (HF) band extracted from the HRV spectrum ;
Generating an integrated indicator (Focus score) defined by the following equation using the time domain indicator and the frequency domain indicator; And
Comparing the value of the integrated indicator to a rule base having an arbitrary threshold to evaluate the high concentration and low concentration of the subject.

The method of claim 1,
The time domain indicator further includes SDNN (standard deviation normal to normal) and pNN50 (Percentage of RR intervals that differ more than 50ms),
The indicator of the frequency domain further includes a power value of a plurality of bands including a very low frequency (VLF) and a low frequency (LF) extracted from the HRV spectrum. delete The method of claim 1,
The RRI is extracted from the ECG data by a QRS detection algorithm, and
The HRV spectrum is obtained by obtaining sampled data from the RRI and extracting the sampled data by FFT analysis. The method of claim 2,
The frequency range of the VLF is 0.0033 ~ 0.04Hz,
The frequency range of the LF is 0.04 to 0.15 Hz, and
The frequency range of the HF is 0.15 ~ 0.4Hz, concentration evaluation method using an electrocardiogram. delete The method of claim 1,
The threshold value is 17.183, the concentration evaluation method using an electrocardiogram. In the concentration evaluation apparatus for performing the method of any one of claims 1, 2, 4, 5 or 7,
An ECG sensor that detects an ECG signal from the subject;
A preprocessor for preprocessing the ECG signal; And
Containing, an analysis unit using the integrated indicator (Focus score) to evaluate the high concentration and low concentration of the subject by using the signal from the preprocessor; concentration evaluation device using an electrocardiogram. delete delete

Images