KR20170004549A

KR20170004549A - Method and system for extracting Heart Information of Frequency domain

Info

Publication number: KR20170004549A
Application number: KR1020150095044A
Authority: KR
Inventors: 황민철; 박상인; 이동원; 황성택; 원명주
Original assignee: 상명대학교서울산학협력단
Priority date: 2015-07-03
Filing date: 2015-07-03
Publication date: 2017-01-11

Abstract

The method of the present invention comprises: photographing a pupil of a subject; Extracting a change rate of a pupil size; And extracting information on the time domain of the heart using the rate of change.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for extracting information from a heart frequency region using a change rate of a pupil size,

The present invention relates to a method and apparatus for extracting cardiac information in a wavenumber region using a pupil size change rate.

Vital Sign Monitoring (VSM) refers to a technique for acquiring biometric information using a sensor attached to a user's body. Biometric information of the user acquired through the sensor includes pulse, blood pressure, electrocardiogram, body temperature and the like. The biometric information obtained from the user is applied to a variety of industrial fields such as U-healthcare industry, Emotion Marketing, Services, Therapy, security industry, education industry, etc. to create application and added value. In addition, it is possible to provide new products and services with functions and forms completely different from those of existing products through the innovation of value-added products and services in the industry (Jung Byung Joon & Choi Seon Hoon, 2008; 2014).

Conventional bio-signal monitoring technology is an unrealistic sensing method of attaching a sensor to a body in order to acquire a biological response to a user. Therefore, there is a need for non-restraint / non-self-sensing technology that measures biometrics while not being in contact with the human body and not giving any pain and not interfering with the user's activities (The Twins & Park Seung Hoon, 2013). Accordingly, techniques for inferring biometric information of a user using camera technology have been developed. The bioinformatic inference technique using a camera can be regarded as a completely non-constrained / noninvasive biological signal sensing technology. The MIT media lab developed a technique for inferring the heartbeat by analyzing the color information of the face that appears finely according to the blood flowing from the heart to the face (Poh et al ., 2011). The MIT CSAIL lab has also reported a technique for inferring heart beat by micro-movement of the head during blood flow from the heart to the head and PCA analysis of the micro-movement of the head to find the heart-related frequency domain Balakrishnan et al ., 2013).

The noncontact cardiac information reasoning technique reported above does not sufficiently consider cardiac variables with reasonably probable cardiac variables and high actual utilization. In addition, the number of samples used in the analysis was 12 based on the average value of the subjects in the data verification process. Therefore, the reliability of the verification is insufficient and the reliability of the verification is insufficient by verifying the inferred data based on the self-developed ECG sensor. Accordingly, the present invention proposes a methodology for securing an increase in the effective inference parameters of the cardiac information and high accuracy against the cardiac information reasoning technique of the prior art. The non-constrained / non-conscious biometric information reasoning technology developed in the present invention is applied to various industrial fields such as U-healthcare (wellness IT), emotion marketing, services, therapy etc., It is expected to create new value based on services.

Balakrishnan, G., Durand, F., and Guttag, J. (2013). Detecting pulse from head motions in video. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on (pp. 3430 . IEEE. Pan J., Tompkins W. J., 1985. A real Poh, M. Z., McDuff, D. J., and Picard, R. W. (2011). Advancements in noncontact, multiparameter physiological measurements using a webcam. Biomedical Engineering, IEEE Transactions on, 58 (1), 7 The Twins, Park Seung Hoon. (2013). IT convergence trend in the wellness field. However, (2008). Wireless Body Area Network Technology Trend. Electromagnetic wave technology, 19 (3), 35 Jong Hye - Sil. (2014). Trends and prospects of healthcare wearable devices. KHIDI Brief, 115.

The present invention provides a method and apparatus for extracting cardiac information in a wavenumber region using a pupil size change rate.

Method according to the invention:

Photographing a pupil of a subject;

Extracting a change rate of a pupil size;

And extracting time-domain information of the heart using the rate of change.

According to the present invention, the accuracy between the data obtained from the electrocardiogram sensor and the data deduced from the pupil data was analyzed, and high correlation between the two signals and a low error rate were confirmed. In addition, we have developed a system that can infer heart time information by acquiring pupil images in real time based on the above algorithm. The unconstrained / unacknowledged cardiac information reasoning technology developed in the present invention is applied to various industrial fields such as U-healthcare (Wellness IT), Emotion Marketing, Services, Therapy etc., It is expected to create new value based on services.

1 shows an electrocardiogram measuring point according to the present invention.
FIG. 2 shows a process of processing a pupil image signal according to the present invention.
FIG. 3 shows a process of ECG signal processing according to the present invention.
FIG. 4 shows the result of comparing the HRV spectrum (VLF power) signals based on the pupil and ECG (Participants 6).
FIG. 5 shows the result of comparing the HRV spectrum (LF power) signal based on the pupil and ECG (Participants 6).
FIG. 6 shows the result of comparing the HRV spectrum (HF power) signals based on the pupil and electrocardiogram (Participants 6).
FIG. 7 shows the result of comparing the HRV spectrum (VLF / HF ratio) signals based on the pupil and electrocardiogram (Participants 6).
FIG. 8 shows the result of comparing the HRV spectrum (LF / HF ratio) signals based on the pupil and electrocardiogram (Participants 6).
FIG. 9 shows the result of comparing the pupil and ECG-based normalized HRV ( ln VLF & ln HF) data pattern (Participants 6).
FIG. 10 shows the result of comparing the pupil and ECG-based normalized HRV ( ln LF & ln HF) data pattern (Participants 6).
Figure 11 shows the results of verifying the HRV spectrum (VLF power) signal accuracy based on the pupil and electrocardiogram.
Figure 12 shows the results of verifying the HRV spectrum (LF power) signal accuracy based on the pupil and electrocardiogram.
Fig. 13 shows the result of verifying the HRV spectrum (HF power) signal accuracy based on the pupil and electrocardiogram.
Figure 14 shows the results of verifying the HRV spectrum (VLF / HF ratio) signal accuracy based on the pupil and electrocardiogram.
Fig. 15 shows the result of verifying the HRV spectrum (LF / HF ratio) signal accuracy based on the pupil and electrocardiogram.
FIG. 16 shows the results of verifying the normalized HRV ( ln VLF) signal accuracy based on the pupil and electrocardiogram.
17 shows the results of verifying the accuracy of the normalized HRV ( ln LF) signal based on the pupil and electrocardiogram.
FIG. 18 shows the results of verifying the normalized HRV ( ln HLF) signal accuracy based on the pupil and electrocardiogram.
Figure 19 illustrates an interface screen of a system according to the present invention.

Hereinafter, a method for extracting information in a cardiac frequency region using a pupil size change rate and an embodiment of the apparatus will be described with reference to the accompanying drawings.

1. Subjects

In this study, 15 subjects (7 males and 7 females - mean age: 28.2 ± 3.24) participated in the experiment. The subjects were those who had no history or history of central and autonomic nervous system and visual function. Before participating in the experiment, we proposed drinking, smoking, and caffeine which could affect the nervous system. The subjects were informed of the outline of the experiment except for the purpose of the study, and then they received the consent of volunteers to participate voluntarily. I paid a certain amount of money to participate in the experiment and increased my intention to participate in the experiment.

2. Experimental Procedure

The subject was allowed to sit on a comfortable chair and the distance between the subject and the screen was 1 m. In the absence of motion, we stared at the gray bar (reference stimuli) presented on the screen for 3 minutes. The electrocardiogram (ECG, electrocardiogram) and the Pupil image were simultaneously measured while the subject stared at the screen. Illumination of the experimental environment and stimulation that could affect the pupil measurement was controlled by the same conditions for all subjects.

3. Data collection and signal processing

The electrocardiogram measured one channel data through the standard limb induction method (lead I). A detailed ECG measurement method is shown in Fig. Electrocardiograms were amplified using an ECG100C amplifier (Biopac system Inc., USA) and signals were collected at 500 Hz using a NI-DAQ-Pad9205 (National Instrument Inc., USA). The collected signals were processed using Labview 2010 software (National Instrument Inc., USA). The pupil images were acquired at 30 frames / second using a point gray camera (FL3-GE-50S5M-C, Canada) and the resolution of the images was 960x400.

In order to extract the pupil region from the acquired infrared image, we used OpenCV for binarization through gray scale and threshold. The pupil region was tracked using the contour detection method in the binarized image from which the region other than the pupil was removed. The pupil size was extracted by applying rect.width, which is the radius of the pupil, to the circular equation only when the condition for measuring the blinking of the pupil that affects the pupil size is satisfied.

The extracted pupil size data was re-sampled at 1 Hz and the pupil size data re-amplified at 1 Hz showed very low frequency (0.0033 - 0.04 Hz), LF band (Low Frequency, 0.04 - 0.15 Hz) (BPF) for each frequency band of HF band (High Frequency, 0.15 - 0.04 Hz) and power amount for each frequency band is calculated by FFT (fast fourier transform) analysis.

At this time, the power (%) of each frequency band was calculated by using the power obtained by FFT processing of the BPF-processed pupil size data at 0.0033-0.4 Hz as total power. The VLF, LF, HF, VLF / HF ratio, and LF / HF ratio of the extracted pupil were calculated from the VLF, LF, HF, VLF / . In addition, VFL, LF, and HF Power extract naturalized HRV ( ln VLF, ln LF, ln HF) data from natural log. Detailed signal processing is shown in FIG.

The electrocardiogram data obtained by the sensor were detected by R-peak of the ECG signal using the QRS Detection Algorithm (Pan & Tompkins, 1985). The detected R-peaks were calculated by RRI (R-peak to R-peak intervals) based on the difference from the adjacent R-peaks.

RRI data was re-sampled (2 Hz) and converted to clock-inverse data. HRV (heart rate variability) spectrum data was extracted by FFT analysis. Extracted HRV spectrum data were extracted from VFL (0.0033 - 0.04 Hz), LF (0.04 - 0.15 Hz) and HF (0.15 - 0.4 Hz) Respectively. In addition, VFL, LF, and HF Power extract naturalized HRV ( ln VLF, ln LF, ln HF) data from natural log. Detailed signal processing is shown in FIG.

The two data were compared and verified by correlation analysis and signal inference accuracy through mean error and error rate (%).

4. Research Results

We compared the HRV spectrum signal deduced based on the pupil size change rate and the HRV spectrum data extracted from the sensor based ECG data. Samples of the two signals of the participants (Participants 6) are shown in FIGS. It was confirmed that the patterns of the two signals are similar and the deviation between data is small.

In addition, we compared normalized HRV data extracted from sensor - based ECG data with normalized HRV signal based on pupil size change rate. Samples of the two data of the participants (Participants 6) are shown in FIGS. It was confirmed that the patterns of the two data are similar and the deviation between the data is small.

The correlation, mean error and error rate of 160 data samples were calculated by sliding window (Window size: 30s, Resolution 1s) for each subject. Correlation analysis was performed using Pearson's correlation analysis. The mean error was calculated by the mean of the difference between the two data, and the error rate was calculated using Equation 6.

The results of the accuracy comparison of the HRV spectrum (VLF power) data of 15 subjects are shown in Fig. The accuracy of the extracted HRV spectrum (VLF power) data based on the pupil and electrocardiogram was confirmed to be strong correlation (r = 0.765 ± 0.055, p <.001) and the mean error was 0.784 ± 0.277 , And the average error rate was 0.956 ± 0.332 (%).

The results of the accuracy comparison of HRV sepctrum (LF power) data of 15 subjects are shown in FIG. The accuracy of HRV sepctrum (LF power) data extracted from pupil and electrocardiogram was verified with strong positive correlation (r = 0.715 ± 0.058, p <.001) and mean error was 0.648 ± 0.216 , And the average error rate was 4.459 ± 1.298 (%).

The accuracy of HRV sepctrum (HF power) data of 15 subjects is shown in Fig. The accuracy of the HRV sepctrum (HF power) data extracted from the pupil and electrocardiogram was verified with a strong positive correlation (r = 0.715 ± 0.037, p <.001) and an average error of 0.697 ± 0.289 , And the average error rate was 5.610 ± 2.564 (%).

The results of the accuracy comparison of HRV sepctrum (VLF / HF ratio) data of 15 subjects are shown in Fig. The accuracy of the HRV sepctrum (VLF / HF ratio) data extracted from the pupil and electrocardiogram was verified with a strong positive correlation (r = 0.724 ± 0.040, p <.001) and the mean error was 4.219 ± 2.318 And the average error rate was 5.602 ± 2.848 (%).

The accuracy of HRV sepctrum (LF / HF ratio) data of 15 subjects is shown in Fig. The accuracy of HRV sepctrum (LF / HF ratio) data extracted from pupil and electrocardiogram was found to be strong correlations (r = 0.724 ± 0.066, p <.001) and mean error was 0.841 ± 0.404 And the average error rate was 2.269 ± 1.362 (%).

The accuracy comparison result of normalized HRV ( ln VLF) data of 15 subjects is shown in FIG. The accuracy of the extracted normalized HRV ( ln VLF) data based on the pupil and electrocardiogram was verified with a strong positive correlation (r = 0.765 ± 0.055, p <.001) and an average error of 0.021 ± 0.013 , And the average error rate was 0.496 ± 0.305 (%).

The results of comparing the accuracy of normalized HRV ( ln LF) data of 15 subjects are shown in FIG. The accuracy of the extracted normalized HRV ( ln LF) data based on the pupil and electrocardiogram was found to be strongly correlated (r = 0.715 ± 0.058, p <.001) and the mean error was 0.024 ± 0.014 , And the average error rate was 1.103 ± 0.650 (%).

The accuracy comparison result of Normalized HRV ( ln HF) data of 15 subjects is shown in Fig. The accuracy of the extracted normalized HRV ( ln HF) data based on pupil and electrocardiogram was confirmed to be strong correlations (r = 0.735 ± 0.037, p <.001) and the mean error was 0.008 ± 0.007 , And the average error rate was 0.639 ± 0.529 (%).

We developed a system that can infer cardiac time information based on real-time acquired pupil images based on the cardiac information inference algorithm. The developed system screen is shown in FIG. In FIG. 19, reference numeral 1 denotes an input image, reference numeral 2 denotes a binarized image, reference numeral 3 denotes a pupil tracking image, reference numeral 4 denotes a heartbeat reasoning signal, reference numeral 5 denotes a HR frequency spectrum inference result, (6) shows the results of normalized HRV inference.

Claims

Photographing a pupil of a subject;
Extracting a change rate of a pupil size;
And extracting information on the time domain of the heart using the rate of change.