CN114209299B

CN114209299B - IPPG technology-based human physiological parameter detection channel selection method

Info

Publication number: CN114209299B
Application number: CN202111505574.0A
Authority: CN
Inventors: 董立泉; 徐歌; 孔令琴; 原静; 赵跃进; 刘明; 惠梅
Original assignee: Beijing Institute of Technology BIT
Current assignee: Beijing Institute of Technology BIT
Priority date: 2021-12-10
Filing date: 2021-12-10
Publication date: 2023-10-20
Anticipated expiration: 2041-12-10
Also published as: CN114209299A

Abstract

The invention relates to a method for selecting a human physiological parameter detection channel based on an IPPG technology, and belongs to the field of physiological signal detection. According to the invention, the visible light source is used for illuminating skin tissues at different parts of a human body, and the CCD camera is used for collecting videos containing pulse information in corresponding illumination areas. IPPG signals in the three RGB channels are extracted from the video by image data processing. And building a classification model of the disease diagnosis of the IPPG signals of the three RGB channels through a classification algorithm, and respectively inputting the IPPG signals of the three RGB channels of the test set into the respective pre-established training set classification model. And respectively predicting the health condition of each channel of subjects in the test set according to the RGB channel IPPG signal disease classification model. And comparing the accuracy of classifying the diseases of each channel of the test set RGB, and selecting the channel with the highest accuracy as the diagnosis channel of the cardiovascular disease. The invention selects the optimal IPPG signal channel as the channel of disease classification aiming at different types of diseases, thereby improving the accuracy of disease diagnosis.

Description

IPPG technology-based human physiological parameter detection channel selection method

Technical Field

The invention relates to a method for selecting a human physiological parameter detection channel based on an IPPG technology, in particular to a method for selecting a disease diagnosis channel based on an imaging type photoplethysmography technology (image photoplenthysmography, IPPG), which belongs to the field of physiological signal detection.

Background

The pulse wave information of the human body contains various physiological parameters related to cardiovascular states, such as heart rate, heart rate variability, blood pressure, blood oxygen, respiratory rate, etc. Since pulse waves contain a great deal of physiological and pathological features of the human body, diagnosis of the physiology and pathology according to the change of the waveform and amplitude of the pulse waves has become one of the most basic and challenging scientific hot problems nowadays.

In recent years, with the development of imaging sensor technology, imaging photoplethysmography (image photoplenthysmography, IPPG) has been widely used for detecting physiological parameters of human body due to its characteristics of non-contact, non-invasive measurement, low cost, simple operation, etc. The IPPG technique generally irradiates a human body with visible light as a light source, acquires a video of a minute change in skin color caused by the intensity of reflected light absorbed by blood and tissues of the human body using an imaging sensor, and extracts an IPPG signal from the video by an image processing technique. And (3) rapidly separating out various physiological parameter characteristics related to the cardiovascular state in the IPPG signal, and further judging the cardiovascular health state according to the physiological parameter characteristics. When video image processing is carried out, the RGB channels of the video image processing device can extract IPPG signals, and the green channel is often used as a diagnosis channel for simple cardiovascular state evaluation such as heart rate and heart rate variability due to the fact that the signal-to-noise ratio of the IPPG signals in the green channel is highest. However, the pathological mechanisms of different types of diseases such as diabetes and heart disease have different effects on the cardiovascular system, and the pathological characteristics of the diseases are obviously different in the three RGB channels of IPPG signals. The depth to which light of different wavelengths reaches the tissue is also different, the red channel reaching the deepest layer of the tissue, followed by green and blue light. Thus, when IPPG signals generated by light with different wavelengths are returned to the CCD imaging sensor, the deep information of the carried tissues is also different. Therefore, simply because the signal-to-noise ratio of the IPPG signal of the green channel is high, the physiological and pathological features on the RGB channel are ignored, so that the diagnosis accuracy of the health state of patients with different cardiovascular diseases is not high. For the state evaluation of complex cardiovascular related diseases, the optimal IPPG signal channel is comprehensively selected as a diagnosis channel of the cardiovascular related diseases by combining the optical physiological characteristics and the actual pathological characteristics of the IPPG signal in the RGB channel.

Disclosure of Invention

The invention aims to solve the problem that the diagnosis accuracy of the health state is not high when the diagnosis and classification of the different blood vessel diseases are carried out based on the green channel IPPG signal at present, and provides a method for selecting a human physiological parameter detection channel based on the IPPG technology. When the method is used for classifying diseases based on IPPG signals, the IPPG signals of the optimal color channels are selected for different types of cardiovascular diseases to serve as channels for disease diagnosis, namely, the IPPG signals of the optimal color channels are selected to correctly distinguish healthy people from disease subjects, so that the accuracy of disease classification diagnosis is improved. The invention is applicable to the selection of IPPG channels for different cardiovascular diseases, and is not limited to a specific disease. The invention avoids the error of the disease classification effect of the low-precision IPPG signal channel on the disease condition of the patient and assists doctors to better diagnose the disease.

The aim of the invention is achieved by the following technical scheme.

A method for selecting human physiological parameter detection channels based on IPPG technology comprises the following steps:

step 1, acquiring a video of a pulse beating part of a subject;

step 2, performing image processing on the video acquired in the step 1 to obtain an IPPG signal;

1) Selecting a region containing a face image for a first frame image of an acquired face video of a subject, and selecting [ a ] in a rectangular matrix ₁ ,b ₁ ]Is used as an interested region extracted by the IPPG signal; wherein a is ₁ Height, b ₁ Is of width, a ₁ And b ₁ The size of the image is smaller than that of the image acquired by the CCD camera;

2) Calculating the pixel mean value of the region of interest to obtain an original IPPG signal;

3) Performing RGB channel separation on the original IPPG signals to extract the IPPG signals of three RGB channels;

4) Preprocessing the IPPG signals of the three channels to achieve the purpose of removing the influence of the non-physiological parameter characteristics, and obtaining preprocessed IPPG signals of the RGB three channels;

step 3, repeating the steps 1 and 2, and collecting different healthy people and disease patients for a plurality of times to obtain a plurality of preprocessed RGB three-channel IPPG signals; the patient with the disease is a patient with cardiovascular disease;

step 4, marking the IPPG signals obtained in the step 3 to distinguish the IPPG signals of healthy people and disease patients; i.e. healthy people with IPPG signal tag of 0 and disease patients with IPPG signal tag of 1; then as a dataset;

and 5, dividing the data set of each RGB channel into a training set and a testing set, wherein the training sets of the RGB channels are used for training the classification model, inputting the testing set into the classification model trained by each channel for classification, comparing the classification result of each subject in the testing set of each channel with a real label value, and determining the IPPG signal channel with the highest classification accuracy as the disease diagnosis channel.

Advantageous effects

1. The invention relates to a method for selecting human physiological parameter detection channels based on an IPPG technology, which is used for realizing the selection of disease diagnosis channels based on the difference of the physiological characteristics of IPPG signals under different color channels of healthy people and disease patients.

2. The invention is suitable for the acquisition of IPPG signals of different parts of a human body and the selection of the color channels of the IPPG signals aiming at different types of cardiovascular diseases.

3. The invention uses the IPPG non-contact optical detection mode to realize the selection of color channels for diagnosing different diseases in a non-invasive, simple and convenient way, improves the accuracy of disease diagnosis, and can be used as the basis for assisting doctors in professional judgment to avoid inaccurate classification effect precision and wrong disease condition caused by the selection of the non-efficient IPPG signal color channels.

Drawings

Fig. 1 is a schematic diagram of facial video acquisition of a method for selecting a human physiological parameter detection channel based on IPPG technology according to an embodiment;

FIG. 2 is a general flow chart of a method for human physiological parameter detection channel selection based on IPPG technology provided by an embodiment;

fig. 3 is an algorithm flow chart of a method for selecting a human physiological parameter detection channel based on IPPG technology according to an embodiment.

Detailed Description

To make the objects, advantages and features of the present invention more apparent, a method and apparatus for disease diagnosis channel selection based on IPPG technology according to the present invention will be described in further detail with reference to the accompanying drawings and specific examples. It should be noted that: the accompanying drawings, which are all in a very simplified form and are incorporated in and constitute a part of the actual structure, are provided solely for the purpose of facilitating and clearly aiding in the description of embodiments of the invention; the acquisition part of the IPPG signal is not limited to the face of the human body, and other parts of the human body which can extract pulse waves are also applicable. The invention is also not limited to the classification of certain cardiovascular diseases and healthy people, and is applicable to the classification of related diseases caused by the change of the physiological parameter characteristics of the cardiovascular diseases. The classification algorithm employed in the present invention is not limited to a certain classification algorithm.

Example 1 of the present invention was carried out on healthy persons and diabetics as subjects.

A method for selecting human physiological parameter detection channels based on IPPG technology is provided, wherein a face video acquisition schematic diagram is shown in fig. 1, and a general flow chart is shown in fig. 2.

Step 1, using a visible light source to illuminate skin tissues of different parts of a human body, and simultaneously using a CCD camera to acquire videos containing pulse information in corresponding illumination areas;

step 1-1, starting a light source and a camera:

the subject sits still on the chair, uses the visible light source to carry out the illumination to the skin tissue of different positions of human body, opens the CCD camera simultaneously. A polarizer is arranged in front of the visible light source, and an analyzer is arranged in front of the CCD camera. The polarizer and analyzer remove the effects of specularly reflected light. The skin tissue of different parts of the human body refers to the parts of the human body such as the face, the arms, the fingers and the like, which can extract pulse waves. The embodiment describes a human face video as a part for extracting IPPG signals.

Step 1-2, video acquisition:

and collecting diffuse reflection light imaging videos containing pulse information in corresponding face illumination areas by using a CCD camera. The length of time for acquiring the face video by the CCD camera is set to be 30 seconds (or more than 30 seconds), the frame rate of the camera is 30fps, and the size of the acquired image is 1920 multiplied by 1000. The subject remains relatively stationary during the photographing.

Step 2, performing image processing on the video acquired in the step 1 to obtain an IPPG signal:

step 2-1, selecting a region containing a face image for a first frame image of a captured subject face video, selecting [ a ] in a rectangular matrix ₁ ,b ₁ ]Is used as an interested region extracted by the IPPG signal; wherein a is ₁ Height, b ₁ Is of width, a ₁ Size greater than 300 pixels, b ₁ A is greater than 300 pixels in size, a ₁ And b ₁ The size is smaller than the size of the acquired image;

and 2-2, calculating a pixel mean value aiming at the region of interest selected in the step 2-1 to obtain an original IPPG signal.

Matrix [ a ] in IPPG imaging device ₁ ,b ₁ ]Each pixel value within the region can be calculated by equation (1):

C(x,y)＝I×(ρ _s (t)+ρ _d (t))+V _n (1)

wherein C (x, y) represents a light intensity value corresponding to a pixel having coordinates (x, y); i represents the light intensity of the light source; ρ _s (t) and ρ _d (t) respectively representing specular reflection coefficient and diffuse reflection coefficient; v (V) _n Representing quantization noise of the image sensor.

Removing quantization noise V of the image sensor by equation (2) _n I.e. all pixel averaging processing is performed on each frame of image.

Wherein, the liquid crystal display device comprises a liquid crystal display device,representing the average light intensity of all pixels on a frame of image. The polarizer and the analyzer remove the non-physiological parameter light intensity information related to the specular reflection, and all +_ under the time sequence t is obtained according to the formula (3)>The set constitutes the IPPG signal.

And 2-3, performing RGB channel separation on the original IPPG signals in the step 2-2, and extracting the IPPG signals of three RGB channels. A flow chart of the algorithm is shown in fig. 3.

And 2-4, preprocessing the IPPG signals of the three channels in the step 2-3 to obtain preprocessed IPPG signals of the RGB three channels. The IPPG signal pretreatment of the RGB three channels adopts a band-pass filter, a trending technology and the like to remove the influence of high-frequency noise, baseline drift and other non-physiological parameter characteristics.

Step 3, repeating the steps 1 and 2, and collecting different healthy people and diabetics for a plurality of times to obtain a plurality of preprocessed RGB three-channel IPPG signals;

the data volume of IPPG signals of the RGB three channels of the healthy person and the diabetic patient is more than 100 groups, wherein the data volume of the IPPG signals of the diabetic patient accounts for about 1/3 of the data volume of the IPPG data of the healthy subject.

Step 4, marking the IPPG signals of the RGB three channels obtained in the step 3 to distinguish the IPPG signals of healthy people and diabetics; i.e. healthy people with IPPG signal tag of 0 and diabetics with IPPG signal tag of 1; then as a dataset;

step 4-1, marking the IPPG signals of the RGB three channels obtained in the step 3 to distinguish the IPPG signals of healthy people and diabetics; i.e. healthy people with IPPG signal tag of 0 and diabetics with IPPG signal tag of 1;

and 4-2, respectively combining the IPPG signals of the RGB three channels obtained in the step 4-1 and the labels to form data sets, wherein the data sets are respectively an R channel data set, a G channel data set and a B channel data set.

And 5, dividing the data set of each channel of RGB into a training set and a testing set, wherein the training sets of the respective channels of RGB are used for training a classification model, then respectively inputting the testing set into the classification model trained by each channel for classification, comparing the classification result of each subject in the testing set of each channel with a real label value, and determining the IPPG signal channel with the highest classification accuracy as the disease diagnosis channel.

Step 5-1, randomly dividing the IPPG signal data set of the three RGB channels obtained in the step 4-2 into a training set and a testing set according to the same proportion, and respectively adopting the same classification algorithm to establish a classification model for the three training sets; the dividing ratio of the training set to the testing set is 4:1; the classification algorithm may be, but is not limited to, a DNN/deep FM/SVM/RF and the like.

And 5-2, inputting the test set of each RGB channel into the classification model of the three RGB channels established in the step 5-1, and predicting the health state of the subjects in each test set by using the classification model of the three RGB channels.

And 5-3, comparing the predicted health state and the real label value of each subject in the RGB three-channel test set to obtain the accuracy of judging the health state of the RGB three groups of subjects. The index of the accuracy adopts the following formula

Accuracy Accuracy _d Indicating the proportion of healthy and diabetic patients that are correctly predicted. Wherein TP is _d Correctly predicting the health of a subjectThe state is healthy; TN (TN) _d Indicating that the healthy state of the subject is correctly predicted as diabetic: FP (Fabry-Perot) _d Indicating that the health status of the subject was incorrectly predicted as a healthy person: FN (Fn) _d Indicating that the subject's health status was incorrectly predicted to be diabetic.

And 5-4, comparing the accuracy of the health status judgment of the subjects in the RGB three-group test set, and determining the IPPG signal channel with the highest classification accuracy as a diabetes diagnosis channel.

Through the steps, the IPPG signal channel most suitable for diabetes is selected as a disease diagnosis channel, so that the disease diagnosis accuracy is improved.

In a specific embodiment, the method for selecting a human physiological parameter detection channel based on IPPG technology is that an optimal IPPG signal color channel is a red channel for a patient with diabetes. The pulse wave is finally revealed by IPPG signals, which are affected by, for example, blood viscosity, absorption and scattering by tissues, and elasticity of arterial walls when the ventricle transmits to arterial vessels. The blood of diabetics is influenced by hyperglycemia for a long time, the degree of influence of blood sugar on different layers of skin tissues is inconsistent, and the depths of reaching tissues of signal channels with different colors are inconsistent, so physiological parameter information carried by IPPG signals in three channels of RGB (red, green and blue) is greatly different due to the fact that the physiological parameter information comprises diabetes pathological information and tissue optical information. The red light is the light with the deepest penetration depth of the tissues in the visible light, the carried physiological information is most abundant, and the classification of the diabetes patients and the healthy people by adopting the IPPG signals of the red channel can obtain more accurate disease classification results, so that the method has important significance for the analysis of the further pathological information of the patients.

In example 2 of the present invention, a healthy person and a heart disease patient are taken as subjects, and facial videos thereof are taken as a site for extracting IPPG signals.

Step 1 and step 2 are the same as in example 1;

step 3, repeating the steps 1 and 2, and collecting different healthy people and heart disease patients for a plurality of times to obtain a plurality of preprocessed RGB three-channel IPPG signals;

the data volume of IPPG signals of the RGB three channels of the healthy person and the heart disease patient is more than 100 groups, wherein the data volume of the IPPG signals of the heart disease patient accounts for about 1/3 of the data volume of the IPPG data of the healthy subject.

Step 4, marking the IPPG signals of the RGB three channels obtained in the step 3 to distinguish the IPPG signals of healthy people and heart disease patients; i.e. healthy people with IPPG signal tag of 0 and cardiac patient with IPPG signal tag of 1; then as a dataset;

step 4-1, marking the IPPG signals of the RGB three channels obtained in the step 3 to distinguish the IPPG signals of healthy people and heart disease patients; i.e. healthy people with IPPG signal tag of 0 and cardiac patient with IPPG signal tag of 1;

Accuracy Accuracy _h Indicating the proportion of healthy and heart disease patients that are correctly predicted. Wherein TP is _h Correctly predicting the health status of the subject as a healthy person; TN (TN) _h Indicating that the healthy state of the subject is correctly predicted to be a heart disease patient: FP (Fabry-Perot) _h Indicating that the health status of the subject was incorrectly predicted as a healthy person: FN (Fn) _h Indicating that the subject's health status was incorrectly predicted to be a heart disease patient.

And 5-4, comparing the accuracy of the health status judgment of the subjects in the RGB three-group test set, and determining the IPPG signal channel with the highest classification accuracy as a heart disease diagnosis channel.

In a specific embodiment, the method for selecting the human physiological parameter detection channel based on the IPPG technology can obtain the optimal color channel for distinguishing heart disease subjects from healthy subjects as a green channel. Heart beats of a heart patient have a certain obstacle, and pulse wave signals transmitted from ventricles are affected by the heart beats to a uniform extent in RGB channels. Although the depth of the signal channels with different colors reaching the tissue is not uniform, the difference of pathological information carried by the IPPG signals is uniform in the three channels of RGB, and only the intensity difference exists. The green channel has highest response in CCD imaging equipment due to high signal-to-noise ratio, and is less affected by motion artifact than the red channel, so that the classification of heart disease patients and healthy people by adopting the IPPG signal of the green channel can obtain more accurate disease diagnosis results.

The above embodiments are only for illustrating the technical solution of the present invention, and it should be understood by those skilled in the art that although the present invention has been described in detail with reference to the above embodiments: modifications and equivalents may be made thereto without departing from the spirit and scope of the invention, which is intended to be encompassed by the claims.

Claims

1. The method for selecting the human physiological parameter detection channel based on the IPPG technology is characterized by comprising the following steps of: the method comprises the following steps:

step 1, acquiring a video of a pulse beating part of a subject;