CN113591769A - Non-contact heart rate detection method based on photoplethysmography - Google Patents
Non-contact heart rate detection method based on photoplethysmography Download PDFInfo
- Publication number
- CN113591769A CN113591769A CN202110912250.2A CN202110912250A CN113591769A CN 113591769 A CN113591769 A CN 113591769A CN 202110912250 A CN202110912250 A CN 202110912250A CN 113591769 A CN113591769 A CN 113591769A
- Authority
- CN
- China
- Prior art keywords
- signal
- signals
- value
- rgb
- channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013186 photoplethysmography Methods 0.000 title claims abstract description 30
- 238000001514 detection method Methods 0.000 title claims abstract description 20
- 238000000034 method Methods 0.000 claims abstract description 50
- 238000001914 filtration Methods 0.000 claims abstract description 23
- 238000012880 independent component analysis Methods 0.000 claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 9
- 238000005070 sampling Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 3
- 238000012899 de-mixing Methods 0.000 claims 2
- 230000009466 transformation Effects 0.000 claims 1
- 230000008859 change Effects 0.000 abstract description 7
- 238000004458 analytical method Methods 0.000 abstract description 3
- 230000001121 heart beat frequency Effects 0.000 abstract 1
- 230000000875 corresponding effect Effects 0.000 description 7
- 239000008280 blood Substances 0.000 description 5
- 210000004369 blood Anatomy 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 5
- 208000024172 Cardiovascular disease Diseases 0.000 description 4
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000001228 spectrum Methods 0.000 description 4
- 230000010349 pulsation Effects 0.000 description 3
- 230000017531 blood circulation Effects 0.000 description 2
- 230000001276 controlling effect Effects 0.000 description 2
- 238000010219 correlation analysis Methods 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000031700 light absorption Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 238000010183 spectrum analysis Methods 0.000 description 2
- 210000001519 tissue Anatomy 0.000 description 2
- 238000010521 absorption reaction Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000004217 heart function Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 229910052760 oxygen Inorganic materials 0.000 description 1
- 239000001301 oxygen Substances 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 210000003491 skin Anatomy 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/024—Detecting, measuring or recording pulse rate or heart rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2134—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on separation criteria, e.g. independent component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/10—Image enhancement or restoration using non-spatial domain filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30101—Blood vessel; Artery; Vein; Vascular
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Cardiology (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Radiology & Medical Imaging (AREA)
- Heart & Thoracic Surgery (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Physiology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
Abstract
The invention discloses a non-contact heart rate detection method based on a photoplethysmography, and belongs to the field of image processing. Firstly, acquiring a face video by adopting a camera; secondly, identifying a face region as an interested region by using Haar features; extracting red, green and blue RGB three-color channel signals in the region of interest, separating the signals by adopting an independent component analysis method, and filtering by adopting an amplitude selection filter; and finally, performing frequency domain analysis by adopting Fourier transform to find the period of the waveform peak, and obtaining the heart beat frequency according to the period change.
Description
Technical Field
The invention relates to the field of image processing, in particular to a method for detecting heart rate according to a human face video sequence based on a photoplethysmography.
Background
At present, the number of cardiovascular diseases in China is 3.30 hundred million, and the morbidity and mortality of cardiovascular diseases are at the head of all diseases. The morbidity and mortality of cardiovascular diseases are still in the rising stage, and in the future, cardiovascular diseases are still the first diseases which are harmful to the life health of human bodies. The heart rate is one of four vital signs of a human body, and whether the heart rate is stable or not directly reflects the heart function, which is an important basis for clinical diagnosis. Therefore, how to conveniently, accurately and cost-effectively measure the heart rate is crucial.
In recent years, a heart rate detection method based on photoplethysmography has been proposed, which is a non-invasive detection method for detecting a change in blood volume in a living tissue by using a photoelectric means. When light of a certain wavelength is irradiated to the skin surface, the light beam is transmitted to a photoelectric receiver by transmission, reflection and the like. Wherein the absorption rate of light by skin, muscle and tissue is unchanged in the whole blood circulation; the blood flows under the action of the heart, the volume of the blood changes periodically, when the heart contracts, the blood volume is increased, the light absorption amount is increased, and the detected light intensity is minimum; when the heart relaxes, the blood volume decreases, the light absorption amount decreases, and the detected light intensity is maximum. The light intensity change signal is converted into an electric signal by using a photoelectric converter, and the change of the pulse blood oxygen volume, including a plurality of human body physiological information including heart rate, can be obtained. According to research, heart rate can be detected by utilizing light intensity change of images based on a photoplethysmography method, and compared with the original method, the method does not need contact or a special sensor, and can meet the requirement of daily heart rate detection. Therefore, a complete heart rate detection scheme is provided based on photoplethysmography, and comprises the steps of identifying and extracting original RGB (Red Green Blue ) signals from an ROI (Region of Interest) in a face Region; performing data processing on the RGB signals by adopting a filter; separating the signals by adopting independent component analysis, and finding out a PPG (photoplethysmography) signal according to correlation analysis; and performing frequency domain analysis by utilizing Fourier transform, finding out the frequency corresponding to the peak, and calculating to obtain a heart rate result. In data processing, an Amplitude selection filtering method proposed by WenJin Wang et al in the literature of Amplitude-selective filtering for remote-PPG is adopted for filtering, and parameters, formulas and the like in the Amplitude selection filtering method are optimized; in the signal separation, a fastICA method proposed by Bingham et al in the document A Fast Fixed-point Algorithm For Independent Component Analysis of Complex Valued Signals is adopted, correlation Analysis is carried out by a method For calculating a correlation coefficient, a selection Algorithm of the heart pulsation signal is improved, and the heart pulsation signal is used as a PPG signal For subsequent calculation so as to calculate the heart rate of the human body more accurately.
Disclosure of Invention
The technical problem solved by the invention is as follows: in order to carry out accurate and stable remote heart rate detection, a complete heart rate detection scheme based on photoplethysmography is provided, and a filtering method and a signal separation method are optimized and improved. Compared with the existing method, the method has the advantages that the measurement accuracy is higher, the measurement result is more stable, the method has certain consistency with the real heart rate measured by the finger-clipped oximeter, and the requirement of heart rate detection in daily life can be basically met. The method specifically comprises the following five steps:
step 1: controlling a camera to aim at a face area to obtain a face video sequence;
step 2: identifying a face region as a region of interest ROI by using Haar features from the face video sequence acquired in the step 1;
and step 3: separating a red channel signal, a green channel signal and a blue channel signal from the obtained ROI as red, green and blue RGB three-color channel signals, and performing filtering processing by using an amplitude selection filter to obtain an observation signal;
and 4, step 4: based on the obtained observation signals, separating by using an independent component analysis method, and selecting a signal with the highest correlation with a green channel signal as a photoplethysmography signal PPG;
and 5: and performing Fourier transform on the obtained PPG to obtain a spectrogram, finding out corresponding frequency according to a peak in the spectrogram, and calculating a heart rate result.
The invention provides a non-contact heart rate measuring method based on a photoplethysmography and provides a set of complete non-contact heart rate detection scheme. According to the color change of the red, green and blue RGB channel signals in the obtained human face video sequence, the change of the intensity of the absorbed light caused by heart pulsation can be obtained. The face area can be quickly and accurately identified by using OpenCV, and the face detection requirement in daily life is met. OpenCV is a cross-platform computer vision and machine learning software library. The method is easily influenced by factors such as illumination, shaking and the like, and generates larger noise. Therefore, the amplitude selection filtering method is adopted to filter most of noise signals which are not in the heart rate range, and the accuracy of the measuring result is ensured. The independent component analysis method is adopted to separate mutually independent original signals from the obtained observation signals, so that the motion artifacts can be removed, and the accuracy of the measurement result is improved. And meanwhile, a correlation coefficient is calculated to obtain a signal with the highest correlation with a green channel, so that the problem of disorder caused by an independent component analysis method is solved. And finally, performing time-frequency conversion through Fourier transform, and finding out the frequency corresponding to the peak in the frequency spectrum to obtain the frequency of the heart rate.
The invention has the beneficial and positive effects that:
(1) the invention aims at the accuracy problem of the heart rate detection result, and optimizes and improves some methods in the process. An amplitude selection filtering method is adopted for data preprocessing, formulas, parameters and the like in the data preprocessing are optimized, and the filtering effect is enhanced; and the independent component analysis is adopted to separate the observed signals, remove motion artifacts, calculate the correlation coefficient to solve the problem of disorder and improve the accuracy of the measurement result.
(2) At present, a complete non-contact heart rate detection scheme does not exist, and the invention provides a complete non-contact heart rate detection scheme which has certain transportability and expansibility. Experiments prove that the method has certain consistency with the measurement result of the finger-clipped oximeter, and the method can be used for daily heart rate detection.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.
FIG. 1 is a flow chart of a non-contact heart rate detection method based on photoplethysmography according to the present invention;
FIG. 2 is a signal time domain diagram after red, green and blue channels are separated after a face region is identified;
FIG. 3 is a time domain diagram of a signal after amplitude selective filtering;
FIG. 4 is a time domain plot of three independent signals after being separated by independent component analysis;
fig. 5 is a spectrum diagram after fourier transform.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings. The embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict.
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
As shown in fig. 1, the non-contact heart rate detection method based on photoplethysmography includes the following five steps:
step 1, controlling a camera to aim at a face area of a human face to obtain a human face video sequence. According to the quinestet sampling theorem, the sampling frequency is greater than twice the highest frequency of the original signal, so that the sampled signal contains all information in the original signal. In practical applications, the sampling frequency is generally required to be 2.56-4 times the highest frequency of the original signal. The normal heart rate of a human body ranges from 40 to 240 beats/minute, corresponding to a frequency range of 0.7 to 4 Hz. According to the sampling theorem, the sampling frequency is required to be greater than 15Hz, i.e. the minimum frame rate of the camera is 15 Hz. The invention adopts a camera with a frame rate of 30 frames to collect the face video.
And 2, recognizing a face region as a region of interest ROI by using Haar features in the face video sequence acquired in the step 1. And (3) realizing face region recognition on the face video sequence in the step (1) by using an OpenCV (open circuit vehicle vision library) algorithm library based on Haar features. The face region can be quickly identified by using a haarcascade _ frontage _ alt2.xml classifier in OpenCV, and 60% of the length and width of the face region is taken as an ROI. OpenCV is a cross-platform computer vision and machine learning software library. Haar features are a common feature descriptor in the field of computer vision. haarcascade _ frontage _ alt2.xml is a trained face recognizer. In the face video acquired in the step 1, face region recognition is realized by using OpenCV based on Haar features. Firstly, a Cascade classifier method is used for selecting a Haar feature cascade classifier, and through actual tests, the haarcascade _ frontage _ alt2.xml classifier is selected. Wherein, CascadeClassifier is a class of the cascade classifier used for target detection in OpenCV. And then, reading a face video sequence by using a VideoCapture method, and graying a face video image by using a cvtColor method after reading the face video, thereby improving the identification accuracy. The video capture is a class for completing the reading operation of the face video in the OpenCV. cvtColor is a color space conversion function, and can realize the space conversion of RGB color (Red Green Blue, color system) into a gray image. And finally, performing face recognition by using a detectMultiScale method. Wherein, the parameter scaleFactor is 1.1, and the parameter minNeighbors is 1.3. The detectMultiScale can detect all faces in the image, and save the coordinates and sizes of the faces (the coordinates and sizes are represented by rectangles) by using a vector (vector). The method obtains a rectangular area of a human face, selects 60% of the length and width of the rectangular area as a final ROI, calculates the mean value of all pixels in each channel ROI as a sample value of a current frame, and obtains RGB three-channel time domain signals in the ROI. The flow and results of step 1 and step 2 are shown in FIG. 2.
And 3, separating a red channel signal, a green channel signal and a blue channel signal from the obtained ROI as red, green and blue RGB three-color channel signals, and performing filtering processing by using an amplitude selection filter to obtain an observation signal. After the RGB three-channel time domain signal is obtained in the step 2, an Amplitude selection filtering method proposed in the literature Amplitude-selective filtering for remote-PPG by WenJin Wang et al is adopted for data processing. Wherein the observation signal is a mixed red, green, blue, RGB three-color channel signal.
In some optional implementations of some embodiments, the separating the red channel signal, the green channel signal, and the blue channel signal from the obtained ROI as three color channel signals of red, green, and blue RGB, and performing a filtering process using an amplitude selective filter to obtain the observation signal may include the following steps:
firstly, an amplitude selection filtering method is adopted to carry out data preprocessing, and the separated RGB three-color channel signals are normalized by adopting the following formula:
wherein p is 1, 2 or 3. Value of pA 1 indicates the red channel. A value of 2 for p indicates the green channel. A value of 3 for p indicates a blue channel. t represents time. x'p(t) denotes p in the normalized RGB three color channel signal. x is the number ofp(t) denotes p in the RGB three-color channel signal. Mu.spRepresenting the mean of p in the RGB three-color channel signal. μ denotes the mean value. SigmapRepresenting the standard deviation of p in the RGB three-color channel signal. σ denotes the standard deviation.
Secondly, after normalization, converting the time domain signal into a frequency domain signal by adopting Fourier transform, setting a weight matrix according to the amplitude of the red channel signal, and when the amplitude of the red channel signal is at a set threshold value aminAnd amaxIn between, the weight is set to 1; otherwise, the weight is set to Δ:
where n represents the number of sampling points obtained by converting the time domain signal to the frequency domain signal using fourier transform. WnRepresenting a weight matrix of n. (F)1，n) Representing the red channel signal after conversion to the frequency domain. abs (F)1，n) Which means that the red channel signal after conversion into the frequency domain is evaluated in absolute value. a ismaxIndicating the maximum value among the set threshold values. a ismaxThe value of (a) is 0.002. a isminIndicating the minimum value of the set threshold values. a isminIs 0.0001. Δ represents a numerical value. The value of Δ is 0.001.
Wherein, the selection of specific parameter values is tested by adopting an experiment, 28 segments of face videos of 10 subjects are selected for parameter adjustment and experimental verification, and finally a is determinedmaxIs 0.002, aminWas 0.0001 and Δ was 0.001.
After the weight matrix acts on the frequency domain signal, the frequency domain signal is converted into a time domain signal by using inverse Fourier transform, and a filtered signal is obtained. Fig. 3 is a time domain diagram of the amplitude selective filtered signal, as shown in fig. 3. Fig. 3 shows RGB three-color channel signals (same as fig. 2) and RGB three-color channel signals after amplitude selective filtering, and it can be seen that the amplitude value is reduced as a whole, the data at the extreme end with too large amplitude is reduced, and the filtering effect is effective.
And 4, separating by using an independent component analysis method based on the obtained observation signals, and selecting a signal with the highest correlation with a green channel signal as a photoplethysmography signal PPG. The signal filtered in the step 3 is subjected to Fast Independent Component Analysis (fastICA) method proposed by Bingham in the document A Fast Fixed-point Algorithm For Independent Component Analysis of Complex Valued Signals and the like, so as to realize Independent Component Analysis. Using the FastICA method in the sklern algorithm library, where n _ components is 3, the observed signal is decomposed into three independent signals that are not correlated to obtain a signal after independent component analysis. The sklern (fully called Scikit-lern) is a machine learning tool based on Python language, and is a common third-party module in machine learning. Python is a computer programming language.
In some optional implementations of some embodiments, the above-mentioned separating based on the obtained observed signal using an independent component analysis method, and selecting a signal with the highest correlation with the green channel signal as the photoplethysmography signal PPG, may include the following steps:
firstly, a fast independent component analysis method fastICA based on an immobile point is adopted to realize independent component analysis, three unmixed signals are separated from an observed RGB mixed signal, a signal with the highest correlation with a green channel is determined by calculating a correlation coefficient according to prior knowledge and is used as PPG, wherein the RGB mixed signal is a signal which is not separated after being filtered, the three unmixed signals are three signals obtained after being separated by the independent component analysis method, the prior knowledge is pulse information of a green channel reflecting the heart, and the calculation of the correlation coefficient adopts the following formula:
where ρ isijRepresenting the ith signal and the jth signalCorrelation coefficient between the signals. i represents a serial number. j represents a serial number. ρ represents a correlation coefficient between signals. E2]Representing a mathematical expectation. t represents time. si(t) represents the ith signal at time t. sj(t) represents the jth signal at time t. E2]2Representing the square of the mathematical expectation.
And step two, after the PPG is obtained, the PPG is used in the subsequent calculation of the step 5.
Fig. 4 is a time domain diagram of three independent signals after being separated by independent component analysis. Fig. 4 shows that the filtered signal is analyzed by the independent component and then decomposed into three independent signals which are not related. And calculating correlation coefficients of the three independent signals and a green channel signal respectively to obtain a second signal with the highest correlation with the green channel, and selecting the second independent signal as a PPG signal for subsequent spectrum analysis.
And 5, obtaining a spectrogram by Fourier transform of the obtained PPG, finding out corresponding frequency according to a peak in the spectrogram, and calculating a heart rate result. And 4, adding a Hanning window to the PPG signal obtained in the step 4 to prevent spectrum leakage, wherein the time domain expression of the Hanning window is as follows:
w (t) represents a hanning window function value at time t. W () represents the hanning window function value. t represents time. And pi represents the circumferential ratio. T denotes the number of entire sampling times.
And then, adopting a five-point moving average filtering method, namely, regarding the sampled data as a queue with the length of 5, calculating the average value of 5 data as the sampled data, then removing the data at the head of the queue, regarding the newly sampled data as the tail of the queue, and repeating the process to obtain the next data.
And finally, Fourier transform is adopted to convert the time domain signal into a frequency domain signal, the heart rate is a periodic signal and corresponds to a peak in a frequency spectrum, the frequency corresponding to the peak with the maximum amplitude in the heart rate range (0.7-4Hz) is found, namely the frequency of the heart rate is obtained, and the final heart rate can be calculated by the following formula:
heart rate＝f×60
where heart rate represents the heart rate result. f represents a frequency value.
As shown in fig. 5, fig. 5 is a spectrogram after fourier transform, and the frequency corresponding to the peak highest in the heart rate range can be calculated to be 1.01Hz, so the heart rate measured this time is approximately 1.01 × 60 ≈ 60.6 ≈ 61 times/min.
The technical content not described in detail in the present invention, such as using the fourier transform to perform the spectrum analysis, including the windowing function, belongs to the common general knowledge of those skilled in the art who work on the signal extraction and analysis.
Although illustrative embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the embodiments, and various changes may be made apparent to those skilled in the art as long as they are within the spirit and scope of the present invention as defined and defined by the appended claims, and all matters of the invention which utilize the inventive concepts are protected.
Claims (3)
1. A method of non-contact heart rate detection based on photoplethysmography, comprising:
step 1: controlling a camera to aim at a face area to obtain a face video sequence;
step 2: identifying a face region as a region of interest ROI by using Haar features from the face video sequence acquired in the step 1;
and step 3: separating a red channel signal, a green channel signal and a blue channel signal from the obtained ROI as red, green and blue RGB three-color channel signals, and performing filtering processing by using an amplitude selection filter to obtain an observation signal;
and 4, step 4: based on the obtained observation signals, separating by using an independent component analysis method, and selecting a signal with the highest correlation with a green channel signal as a photoplethysmography signal PPG;
and 5: and performing Fourier transform on the obtained PPG to obtain a spectrogram, finding out corresponding frequency according to a peak in the spectrogram, and calculating a heart rate result.
2. The method of claim 1, wherein said separating the red, green and blue channel signals from the obtained ROI as RGB three-color channel signals, the filtering process using an amplitude selective filter, and obtaining the observation signal comprises:
adopting an amplitude selection filtering method to carry out data preprocessing, and carrying out normalization on the separated RGB three-color channel signals by adopting the following formula:
wherein the value of p is 1, 2 or 3, the value of p is 1 to represent a red channel, the value of p is 2 to represent a green channel, the value of p is 3 to represent a blue channel, t represents time, and x'p(t) denotes p, x in the normalized RGB three-color channel signalp(t) represents p, mu in RGB three-color channel signalpRepresenting the mean value of p, mu the mean value, sigma, in RGB three-color channel signalspRepresents the standard deviation of p in the RGB three-color channel signal, and sigma represents the standard deviation;
after normalization, Fourier transformation is adopted to convert the time domain signal into a frequency domain signal, a weight matrix is set according to the amplitude of the red channel signal, and when the amplitude of the red channel signal is at a set threshold value aminAnd amaxIn between, the weight is set to 1; otherwise, the weight is set to Δ:
where n denotes the number of sampling points obtained by converting a time domain signal into a frequency domain signal using Fourier transform, and WnA weight matrix representing n, (F)1，n) Representation conversion toRed channel signal after frequency domain, abs (F)1，n) Representing the absolute value, a, of the red channel signal after conversion into the frequency domainmaxIndicates the maximum value of the set threshold value, amaxHas a value of 0.002, aminRepresents the minimum value of the set threshold values, aminIs 0.0001, Δ represents a value, and Δ is 0.001.
3. The method according to claim 2, wherein said separating, based on the obtained observed signals, using an independent component analysis method, selecting as the photoplethysmography signal PPG the signal having the highest correlation with the green channel signal, comprises:
the fast independent component analysis method fastICA based on the fixed point is adopted to realize independent component analysis, three signals after de-mixing are separated from the observed RGB mixed signals, according to priori knowledge, a signal with the highest correlation with a green channel is determined by calculating a correlation coefficient and serves as PPG, wherein the RGB mixed signals are not separated after being subjected to filtering processing, the three signals after de-mixing are three signals obtained after being separated by the independent component analysis method, the priori knowledge is pulse information of a green channel reflecting the heart, and the calculation of the correlation coefficient adopts the following formula:
where ρ isijDenotes a correlation coefficient between the ith signal and the jth signal, i denotes a serial number, j denotes a serial number, ρ denotes a correlation coefficient between the signals, E [ deg. ]]Representing a mathematical expectation, t represents time, si(t) denotes the i-th signal at time t, sj(t) represents the jth signal at time t, E]2Represents the square of the mathematical expectation;
once the PPG was obtained, the PPG was used in the subsequent calculation of step 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110912250.2A CN113591769B (en) | 2021-08-10 | 2021-08-10 | Non-contact heart rate detection method based on photoplethysmography |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110912250.2A CN113591769B (en) | 2021-08-10 | 2021-08-10 | Non-contact heart rate detection method based on photoplethysmography |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113591769A true CN113591769A (en) | 2021-11-02 |
| CN113591769B CN113591769B (en) | 2023-09-15 |
Family
ID=78256610
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110912250.2A Active CN113591769B (en) | 2021-08-10 | 2021-08-10 | Non-contact heart rate detection method based on photoplethysmography |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113591769B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114271800A (en) * | 2021-12-01 | 2022-04-05 | 西北工业大学 | Non-invasive continuous blood pressure monitoring method in office environment and application |
| WO2024056087A1 (en) * | 2022-09-16 | 2024-03-21 | Jing Wei Chin | Method and device for camera-based heart rate variability monitoring |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016092290A1 (en) * | 2014-12-08 | 2016-06-16 | Oxehealth Limited | Method and apparatus for physiological monitoring |
| CN108471989A (en) * | 2016-01-15 | 2018-08-31 | 皇家飞利浦有限公司 | The equipment, system and method for the photo-plethysmographic image of vital sign information for generating carrying object |
| CN109480808A (en) * | 2018-09-27 | 2019-03-19 | 深圳市君利信达科技有限公司 | A kind of heart rate detection method based on PPG, system, equipment and storage medium |
| CN110384491A (en) * | 2019-08-21 | 2019-10-29 | 河南科技大学 | A kind of heart rate detection method based on common camera |
-
2021
- 2021-08-10 CN CN202110912250.2A patent/CN113591769B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016092290A1 (en) * | 2014-12-08 | 2016-06-16 | Oxehealth Limited | Method and apparatus for physiological monitoring |
| CN108471989A (en) * | 2016-01-15 | 2018-08-31 | 皇家飞利浦有限公司 | The equipment, system and method for the photo-plethysmographic image of vital sign information for generating carrying object |
| CN109480808A (en) * | 2018-09-27 | 2019-03-19 | 深圳市君利信达科技有限公司 | A kind of heart rate detection method based on PPG, system, equipment and storage medium |
| CN110384491A (en) * | 2019-08-21 | 2019-10-29 | 河南科技大学 | A kind of heart rate detection method based on common camera |
Non-Patent Citations (1)
| Title |
|---|
| 冯军;汤文明;曹剑剑;余瑛;: "非接触式心率测量研究初步", 中国生物医学工程学报, no. 05 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114271800A (en) * | 2021-12-01 | 2022-04-05 | 西北工业大学 | Non-invasive continuous blood pressure monitoring method in office environment and application |
| WO2024056087A1 (en) * | 2022-09-16 | 2024-03-21 | Jing Wei Chin | Method and device for camera-based heart rate variability monitoring |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113591769B (en) | 2023-09-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6521845B2 (en) | Device and method for measuring periodic fluctuation linked to heart beat | |
| CN105147274B (en) | A kind of method that heart rate is extracted in the face video signal from visible spectrum | |
| CN112949349B (en) | Method and system for displaying pulse condition waveform in real time based on face video | |
| CN109350030B (en) | System and method for processing human face video heart rate signal based on phase amplification | |
| CN113591769B (en) | Non-contact heart rate detection method based on photoplethysmography | |
| CN106491117A (en) | A kind of signal processing method and device based on PPG heart rate measurement technology | |
| CN111259895B (en) | Emotion classification method and system based on facial blood flow distribution | |
| CN111387959A (en) | Non-contact physiological parameter detection method based on IPPG | |
| Wedekind et al. | Automated identification of cardiac signals after blind source separation for camera-based photoplethysmography | |
| CN107334469A (en) | Non-contact more people's method for measuring heart rate and device based on SVMs | |
| CN114781465A (en) | rPPG-based non-contact fatigue detection system and method | |
| CN115153473B (en) | Non-contact heart rate detection method based on multivariate singular spectrum analysis | |
| CN112294282A (en) | Self-calibration method of emotion detection device based on RPPG | |
| CN114387479A (en) | Non-contact heart rate measurement method and system based on face video | |
| CN112043257A (en) | Non-contact video heart rate detection method for motion robustness | |
| CN113397519A (en) | Cardiovascular health state detection device | |
| CN111839492A (en) | Heart rate non-contact type measuring method based on face video sequence | |
| CN113080966A (en) | Automatic depression detection method based on sleep stages | |
| CN117136027A (en) | Method and system for extracting heart rate from RGB image | |
| Karmuse et al. | A robust rPPG approach for continuous heart rate measurement based on face | |
| CN111671421A (en) | Electroencephalogram-based children demand sensing method | |
| CN114271800B (en) | Non-invasive continuous blood pressure monitoring method and application in office environment | |
| CN114246570B (en) | Near-infrared heart rate detection method by fusing peak signal-to-noise ratio and Peerson correlation coefficient | |
| CN115245318A (en) | Automatic identification method of effective IPPG signal based on deep learning | |
| CN113397505A (en) | Physiological signal detection method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |