CN115376179A - Non-contact heart rate measuring method and system - Google Patents

Abstract

The invention provides a non-contact heart rate measurement method and system, belonging to the technical field of physiological parameter detection equipment, which acquires multiple frames of human face images of a person to be tested; determines the face area of interest in each frame of human face images; The skin pixels in the area of interest of the face are accumulated and averaged to obtain the IPPG signal; the chrominance signal is obtained based on the IPPG signal; the variational mode decomposition algorithm is performed on the chrominance signal to obtain multiple modal components, and the one with the largest peak in the spectrum The modal component is the heart rate signal. The invention can separate the intensity of the pixel value from the light intensity and the color through the CHROM algorithm, and eliminate the noise; the signal is subjected to the VMD algorithm, and according to the heartbeat frequency characteristics, the main signal is decomposed into different modes by using the VMD algorithm, which ensures the accuracy of each mode. The frequency ranges of the signals do not overlap each other, and a relatively complete heartbeat signal without harmonic residue is separated.

Description

Non-contact heart rate measurement method and system

technical field

The invention relates to the technical field of physiological parameter detection equipment, in particular to a non-contact heart rate measurement method and system.

Background technique

Conventional heart rate measurement requires checking with electrocardiographic techniques through the chest leads. This method requires multiple electrodes to be accurately placed on designated parts of the body. In order to reduce the contact impedance, the electrodes are often filled with conductive paste to enhance conductivity, which makes the detection process time-consuming and laborious, and it cannot be detected while the subject is in motion.

Photoplethysmography (PPG) detects heart rate. The principle is that the pulsating blood traveling in the cardiovascular system changes the blood volume in the skin tissue, and the oxygenated blood circulation causes fluctuations in the number of hemoglobin molecules and proteins, resulting in The fluctuation of optical absorption across the spectrum, and the curve of this optical absorption fluctuation with time is the pulse wave, which can be processed to extract the heart rate signal. However, due to the single location of the PPG monitoring area and the need to contact the skin of the test subject, its application range is limited.

Imaging Photoplethysmography (IPPG) is an imaging-based pulse wave measurement method. The principle of this technology is to use an RGB camera to capture small color changes reflected in the skin to identify the stage of blood circulation and extract The pulse wave signal output has the advantages of no contact with the tested part, simple and easy operation, etc., which can solve the problem that patients with skin burns or limb defects and infants are difficult to use contact instruments to measure physiological parameters. However, imaging photoplethysmography technology is sensitive to light and motion, and can only measure heart rate more accurately when the light source is sufficient and in a resting state.

Contents of the invention

The object of the present invention is to provide a non-contact heart rate measurement method and system based on chromaticity remote photoplethysmography (Chrominance-Based RPPG, CHROM) and variational mode decomposition (Variational Mode Decomposition, VMD), to solve the above-mentioned At least one technical problem exists in the background technology. In order to achieve the above object, the present invention has taken the following technical solutions:

In one aspect, the present invention provides a non-contact heart rate measurement method, comprising:

Obtain multiple frames of face images of the person to be tested;

Determine the face region of interest in each frame of face image;

Accumulate and average the skin pixels in the region of interest of all faces to obtain the IPPG signal;

Obtain a chrominance signal based on the IPPG signal;

A variational mode decomposition algorithm is performed on the chrominance signal to obtain multiple modal components, and the modal component with the largest peak value in the spectrum is the heart rate signal.

Preferably, obtaining the ROI of the human face includes: using a landmark face recognition model to obtain facial landmarks, setting the ROI of the human face, and using a skin color detector to remove the environmental information of the edge of the human face.

Preferably, the skin color detector uses a multi-color space skin color detection algorithm.

Preferably, obtaining the chrominance signal includes:

RGB three-channel extraction is performed on the IPPG signal, and then the RGB channel signal is normalized;

Project the normalized RGB values to two orthogonal chrominance vectors;

Filter the two orthogonal vectors with a Butterworth bandpass filter to obtain the filtered signal;

A chrominance signal is calculated based on the filtered signal.

Preferably, the alternate direction multiplier algorithm is used to update and iteratively solve the saddle point in the orthogonal chromaticity vector calculation, and the modal component and the augmented Lagrange function multiplier are iteratively updated in the frequency domain by using the augmented Lagrange function and the constrained variational model until satisfying Iterate the termination condition to obtain the modal components.

Preferably, the constrained variational model involved in the variational mode decomposition algorithm is:

表示偏导运算，δ(t)表示单位脉冲函数，j表示虚数单位，*表示卷积运算，f表示目标信号；Among them, {u _k } represents the kth modal component, {w _k } represents the center frequency of the kth modal component, K represents the number of modal components,

Represents the partial derivative operation, δ(t) represents the unit pulse function, j represents the imaginary number unit, * represents the convolution operation, and f represents the target signal;

The penalty factor α and the Lagrange multiplier λ are introduced to solve the variational constraint problem, and the expression of the augmented Lagrange function is as follows:

Preferably, the center frequency of the modal component is updated using the following formula in the frequency domain:

Among them, ω represents the frequency, i represents the i-th modal component, and d represents the derivative.

Combine the following formula to update λ:

Among them, τ represents the fidelity coefficient, ∧ represents the Fourier transform, and n represents the number of iterations.

Preferably, the iteration termination condition is:

Among them, ε represents the discrimination accuracy, and ε>0.

In a second aspect, the present invention provides a non-contact heart rate measurement system, comprising:

An acquisition module, configured to acquire multiple frames of human face images of the person to be tested;

A recognition module, configured to determine the face region of interest in each frame of face images;

The first calculation module is used to accumulate and average the skin pixels in the region of interest of all faces to obtain the IPPG signal;

The second calculation module is used to obtain the chrominance signal based on the IPPG signal;

The third calculation module is used to perform a variational mode decomposition algorithm on the chrominance signal to obtain multiple mode components, and the mode component with the largest peak value in the frequency spectrum is the heart rate signal.

In a third aspect, the present invention provides a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the above-mentioned non-transitory Contact heart rate measurement method.

In a fourth aspect, the present invention provides an electronic device, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the The computer program stored in the memory, so that the electronic device executes the instructions for realizing the above-mentioned non-contact heart rate measurement method.

Beneficial effects of the present invention:

Through the CHROM algorithm, the intensity, light intensity and color of the pixel value can be separated to eliminate noise; the signal is subjected to the VMD algorithm, and according to the heartbeat frequency characteristics, the main signal is decomposed into different modes by using the VMD algorithm to ensure the signal between the modes The frequency ranges do not overlap with each other, and a relatively complete heartbeat signal without harmonic residue is separated.

For complex facial conditions, especially when there is non-skin pixel interference information such as hair, glasses or beard, the multi-color space skin color detection algorithm is used to remove non-skin color pixels to obtain ROI with high stability and high signal-to-noise ratio.

When extracting the pulse wave signal, there will inevitably be noise introduced in the pulse wave due to the movement of the face and the change of the light direction. Eliminate pulse wave information with high signal-to-noise ratio to achieve high-accuracy non-contact heart rate detection.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description, or may be learned by practice of the invention.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without making creative efforts.

Fig. 1 is the flowchart of the non-contact heart rate measurement method described in the embodiment of the present invention;

FIG. 2 is a schematic diagram of a face region of interest according to an embodiment of the present invention;

Fig. 3 is a schematic diagram of the processing process of the face image described in the embodiment of the present invention;

FIG. 4 is a structural diagram of an acquisition module according to an embodiment of the present invention;

Fig. 5 is a flowchart of the automatic optimization of the number of modal components and penalty factors according to the embodiment of the present invention.

Detailed ways

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with the drawings are exemplary, and are only used to explain the present invention, but not to be construed as limiting the present invention.

Those skilled in the art can understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

It should also be understood that terms such as those defined in commonly used dictionaries should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted in an idealized or overly formal sense unless defined as herein explain.

Those skilled in the art will understand that unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the word "comprising" used in the description of the present invention refers to the presence of said features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, Integers, steps, operations, elements and/or groups thereof.

In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other.

In order to facilitate the understanding of the present invention, the present invention will be further explained below with specific embodiments in conjunction with the accompanying drawings, and the specific embodiments are not intended to limit the embodiments of the present invention.

Those skilled in the art should understand that the drawings are only schematic diagrams of the embodiments, and the components in the drawings are not necessarily necessary for implementing the present invention.

Example 1

This embodiment 1 provides a non-contact heart rate measurement system, using this system to achieve non-contact heart rate measurement, without applying conductive paste to the skin, without using electrodes to contact designated parts of the body, it detects the face through precise positioning, Combining the elimination of light intensity and color information and noise removal, the heart rate signal is extracted.

In the present embodiment 1, the non-contact heart rate measurement system mainly includes the following functional modules:

An acquisition module, configured to acquire multiple frames of human face images of the person to be tested;

A recognition module, configured to determine the face region of interest in each frame of face images;

The first calculation module is used to accumulate and average the skin pixels in the region of interest of all faces to obtain the IPPG signal;

The second calculation module is used to obtain the chrominance signal based on the IPPG signal;

The third calculation module is used to perform a variational mode decomposition algorithm on the chrominance signal to obtain multiple mode components, and the mode component with the largest peak value in the frequency spectrum is the heart rate signal.

In the present embodiment 1, the above-mentioned non-contact heart rate measurement system is used to realize a non-contact heart rate measurement method, which includes:

Use the acquisition module to acquire multiple frames of face images of the person to be tested;

Utilize the recognition module to determine the region of interest of the face in each frame of face image;

Use the first calculation module to accumulate and average the skin pixels in the region of interest of all faces to obtain an IPPG signal;

Use the second calculation module to process the IPPG signal to obtain a chrominance signal;

The third module is used to perform a variational modal decomposition algorithm on the chrominance signal to obtain multiple modal components, and the modal component with the largest peak value in the spectrum is the heart rate signal.

In Embodiment 1, obtaining the face ROI includes: using the landmark face recognition model to obtain face landmarks, setting the face ROI, and using the skin color detector to remove the environmental information of the edge of the face.

In Embodiment 1, obtaining the chrominance signal includes:

RGB three-channel extraction is performed on the IPPG signal, and then the RGB channel signal is normalized;

Project the normalized RGB values to two orthogonal chrominance vectors;

Filter the two orthogonal vectors with a Butterworth bandpass filter to obtain the filtered signal;

A chrominance signal is calculated based on the filtered signal.

In this embodiment 1, the alternate direction multiplier algorithm is used to update and iteratively solve the saddle point in the calculation of the orthogonal chromaticity vector, and the augmented Lagrange function and the constrained variational model are used to iteratively update the modal component and the augmented Lagrange function multiplier in the frequency domain , until the iteration termination condition is met, and the modal components are obtained.

In this embodiment 1, the constrained variational model involved in the variational mode decomposition algorithm is:

表示偏导运算，δ(t)表示单位脉冲函数，j表示虚数单位，*表示卷积运算，f表示目标信号；Among them, {u _k } represents the kth modal component, {w _k } represents the center frequency of the kth modal component, K represents the number of modal components,

Represents the partial derivative operation, δ(t) represents the unit pulse function, j represents the imaginary number unit, * represents the convolution operation, and f represents the target signal;

The penalty factor α and the Lagrange multiplier λ are introduced to solve the variational constraint problem, and the expression of the augmented Lagrange function is as follows:

In Embodiment 1, the center frequency of the modal component is updated in the frequency domain using the following formula:

Among them, ω represents the frequency, i represents the i-th modal component, and d represents the derivative.

Combine the following formula to update λ:

Among them, τ represents the fidelity coefficient, ∧ represents the Fourier transform, and n represents the number of iterations.

In this embodiment 1, the iteration termination condition is:

Among them, ε represents the discrimination accuracy, and ε>0.

In this embodiment 1, the proposed non-contact heart rate measurement system and the heart rate measurement method realized by using the system are based on Chrominance-Based RPPG (CHROM) and variational mode decomposition (Variational Mode Decomposition) , VMD), which can be used for non-contact heart rate measurement under complex lighting and large-scale motion, and has important application value for real-time monitoring of vital signs in complex environments.

Example 2

In Embodiment 2, a non-contact heart rate measurement system is provided, which acquires a face image, performs signal processing on the face image, and finally obtains a heart rate signal.

In Embodiment 2, the non-contact heart rate measurement system mainly includes the following functional modules:

An acquisition module, configured to acquire multiple frames of human face images of the person to be tested;

A recognition module, configured to determine the face region of interest in each frame of face images;

The first calculation module is used to accumulate and average the skin pixels in the region of interest of all faces to obtain the IPPG signal;

The second calculation module is used to obtain the chrominance signal based on the IPPG signal;

The third calculation module is used to perform a variational mode decomposition algorithm on the chrominance signal to obtain multiple mode components, and the mode component with the largest peak value in the frequency spectrum is the heart rate signal.

In Embodiment 2, the above-mentioned non-contact heart rate measurement system is used to realize a non-contact heart rate measurement method, which includes:

Use the acquisition module to acquire multiple frames of face images of the person to be tested;

Utilize the recognition module to determine the region of interest of the face in each frame of face image;

Use the first calculation module to accumulate and average the skin pixels in the region of interest of all faces to obtain an IPPG signal;

Use the second calculation module to process the IPPG signal to obtain a chrominance signal;

The third module is used to perform a variational modal decomposition algorithm on the chrominance signal to obtain multiple modal components, and the modal component with the largest peak value in the spectrum is the heart rate signal.

In Embodiment 2, the acquisition module includes an industrial camera, which is used to supplement light on the human face in combination with a halogen lamp (its light wave range includes visible light and invisible light range) to obtain better image quality. The human face video sequence captured is input in the PC end (computer system) by USB3. processing, and finally obtain the heart rate signal.

First, frame processing is performed on the captured video to obtain each frame of face image.

Obtaining the area of interest of the face includes: using the landmark face recognition model to obtain face marker points, setting the area of interest of the face, and using the skin color detector to remove the environmental information of the edge of the face.

The face recognition program in the recognition module uses the landmark model generated by the regression tree method (Ensemble of Regression Trees, ERT) based on gradient improvement learning. Obtain 68 face marker points through the landmark model, and set the face interest area as shown in Figure 2. Among them, the picture in the upper left corner is the mask map of face marker points, the picture in the upper right corner is the actual face mask map, and the picture in the lower left corner The picture in the corner is the actual ROI map after using the skin color detector, and the picture in the lower right corner is the binary map of the actual ROI map. In the detection process, it is inevitable that the edge of the face contains a small amount of environmental information. In the subsequent use of the skin color detector, the area of interest is eliminated by the skin color detector to remove the environmental information of the edge of the face.

The original IPPG signal is obtained by accumulating and averaging all the processed skin pixels.

In this embodiment 2, using the second calculation module to obtain the chrominance signal includes: performing RGB three-channel extraction on the IPPG signal, and then performing normalization processing on the RGB channel signal; projecting the normalized RGB value to two positive Intersecting chrominance vectors; filtering two orthogonal vectors with a Butterworth bandpass filter to obtain filtered signals; calculating chrominance signals based on the filtered signals. specific:

Using the CHROM algorithm to process the original IPPG signal, the steps are as follows:

Extract the RGB three-channel from the original IPPG signal, and then normalize the RGB channel signal; project the normalized RGB value to two orthogonal chrominance vectors X _chrom and Y _chrom :

X _chrom (t)＝3x _r (t)-2x _g (t) (2)

Y _chrom (t)＝1.5x _r (t)+x _g (t)-1.5x _b (t) (3)

The final output S(t) is:

S(t)= _Xf - _αYf ; (4)

Among them, x _r represents the red channel component of the original IPPG signal, x _g represents the green channel component of the original IPPG signal, x _b represents the blue channel component of the original IPPG signal, t represents time, and the signal waveform changes with time t, X _f , Y _f are the signals of X _chrom and Y _chrom filtered by the fifth-order 0.7Hz-4Hz Butterworth bandpass filter; α is the ratio of the standard deviations of X _f and Y _f , and S(t) is the The IPPG signal.

After the CHROM algorithm, most of the noise in the original IPPG signal is eliminated. Then perform the VMD algorithm on S(t) to obtain a more accurate heart rate signal.

In this embodiment 2, the alternate direction multiplier algorithm is used to update and iteratively solve the saddle point in the calculation of the orthogonal chromaticity vector, and the augmented Lagrange function and the constrained variational model are used to iteratively update the modal component and the augmented Lagrange function multiplier in the frequency domain , until the iteration termination condition is met, and the modal components are obtained.

In Example 2, the constrained variational model involved in the variational mode decomposition algorithm is:

表示偏导运算，δ(t)表示单位脉冲函数，j表示虚数单位，*表示卷积运算，f表示目标信号，e表示超越数。Among them, {u _k }={u ₁ ,u ₂ ,...,u _k } means the kth modal component, {w _k }={w ₁ ,w ₂ ,...,w _k } means the The center frequency of k modal components, K represents the number of modal components,

Represents the partial derivative operation, δ(t) represents the unit impulse function, j represents the imaginary number unit, * represents the convolution operation, f represents the target signal, and e represents the transcendental number.

The penalty factor α and the Lagrange multiplier λ are introduced to solve the variational constraint problem, and the expression of the augmented Lagrange function is as follows:

In Embodiment 2, an alternate multiplier algorithm is used to update and iteratively solve the saddle point in formula (7), and u _k , w _k , and λ are iteratively updated in the frequency domain.

In this embodiment 2, the VMD algorithm decomposes the signal into 3 modal components (the number of modal components is determined by experiment), the penalty factor α is 2000, and the decomposition steps are as follows:

n为零；Step 1: Initialize

n is zero;

Step 2: u _k and w _k are iteratively updated by formula (8) and formula (9) respectively:

Among them, ω represents the frequency, i represents the i-th modal component, and d represents the derivative.

Step 3: Update λ through formula (10):

Among them, τ represents the fidelity coefficient, ∧ represents the Fourier transform, and n represents the number of iterations.

Step 4: Repeat steps 2 and 3 until the iteration termination condition is met, which is given by (11).

Among them, ε represents the discrimination accuracy, and ε>0, the superscript n is the number of iteration steps, and the subscript k represents the current mode number.

Step 5: Output 3 modal components.

Among the three modal components output by the VMD algorithm, the modal component with the largest peak value in the frequency spectrum is the heart rate signal.

In Example 2, the proposed non-contact heart rate measurement system and the heart rate measurement method realized by using the system are based on Chrominance-Based RPPG (CHROM) and variational mode decomposition (Variational Mode Decomposition) , VMD), which can be used for non-contact heart rate measurement under complex lighting and large-scale motion, and has important application value for real-time monitoring of vital signs in complex environments.

Example 3

In Embodiment 3, a non-contact heart rate measurement system is provided, which acquires a face image, performs signal processing on the face image, and finally obtains a heart rate signal.

In the third embodiment, the non-contact heart rate measurement system mainly includes the following functional modules:

An acquisition module, configured to acquire multiple frames of human face images of the person to be tested;

A recognition module, configured to determine the face region of interest in each frame of face images;

The first calculation module is used to accumulate and average the skin pixels in the region of interest of all faces to obtain the IPPG signal;

The second calculation module is used to obtain the chrominance signal based on the IPPG signal;

The third calculation module is used to perform a variational mode decomposition algorithm on the chrominance signal to obtain multiple mode components, and the mode component with the largest peak value in the frequency spectrum is the heart rate signal.

In this embodiment 3, the above-mentioned non-contact heart rate measurement system is used to realize a non-contact heart rate measurement method, which includes:

Use the acquisition module to acquire multiple frames of face images of the person to be tested;

Utilize the recognition module to determine the region of interest of the face in each frame of face image;

Use the first calculation module to accumulate and average the skin pixels in the region of interest of all faces to obtain an IPPG signal;

Use the second calculation module to process the IPPG signal to obtain a chrominance signal;

The third module is used to perform a variational modal decomposition algorithm on the chrominance signal to obtain multiple modal components, and the modal component with the largest peak value in the spectrum is the heart rate signal.

In this embodiment 3, as shown in Figure 4, the acquisition module includes an industrial camera A1, which is used to complement the face of subject A4 through the industrial camera, combined with the halogen lamp A2 (its light wave range includes visible light and invisible light range). light for better image quality. The human face video sequence that captures is input in the PC end A3 (computer system) by USB3. The image is processed to finally obtain the heart rate signal.

First, frame processing is performed on the captured video to obtain each frame of face image.

Obtaining the face area of interest includes: using the landmark face recognition model to obtain face marker points, setting the face area of interest, and using the skin color detector to remove the environmental information of the edge of the face.

The face recognition program in the recognition module uses the landmark model generated by the regression tree method (Ensemble of Regression Trees, ERT) based on gradient improvement learning. Obtain 68 face marker points through the landmark model, set the area of interest of the face as shown in the upper left corner of Figure 2, use marker points 0, 8, and 16 to generate new marker points HL and marker points HR, where HL and HR The coordinate points of are calculated as follows:

Among them, y _p8 is the ordinate of marking point 8, y _p0 is the ordinate of marking point 0, x _p0 is the abscissa of marking point 0, y _p0 is the ordinate of marking point 0, x _p16 is the abscissa of marking point 16 Coordinates, y _p16 is the vertical coordinate of the marker point 16; x _HR , y _HR and x _HL , y _HL are decomposed into the horizontal and vertical coordinates of the new marker point HR and the horizontal and vertical coordinates of the new marker point HL; Points 16, HR, and HL are sequentially connected to obtain the face area, and the mark point 48 to mark point 59 are sequentially connected to remove the mouth area, and finally the face mask map is obtained, that is, the region of interest.

In the detection process, it is inevitable that the edge of the face contains a small amount of environmental information. In the subsequent use of the skin color detector, the area of interest is removed by the skin color detector to remove the environmental information of the face edge. Specifically, the skin color detector uses more The color space skin color detection algorithm removes non-skin interference as much as possible while retaining skin area information. Its core method is: use RGB, YcrCb and CMYK three different color space skin color thresholds for accurate skin color pixel judgment.

The multi-color space skin color detection algorithm is specifically:

(1) Skin color threshold conditions in the RGB color space are as follows:

R＞95∩G＞40∩B＞20∩R>G∩R＞B∩|R-G|>15

Among them, R is the value of the red channel, G is the value of the green channel, and B is the value of the blue channel;

(2) The skin color threshold condition in the YCrCb color space is as follows:

Cr>135∩Cb>85∩Yl>80∩Cr<=(1.5874*Cb)+20∩

Cr＞＝(0.3447*Cb)+76.2068∩Cr＞＝(-4.5653*Cb)+234.5652∩

Cr<=(-1.15*Cb)+301.78∩Cr<=(-2.2868*Cb)+433.85

Wherein, Yl is the color brightness value, Cr is the concentration offset value of red, and Cb is the blue component offset value;

(3) Skin color threshold conditions in CMYK color space are as follows:

K<0.8∩0<=C<0.05∩0.1<=Y/M<4.8

∩0.088<Y<1∩0<C/Y<1

Among them, C is the cyan channel value, M is the magenta channel value, Y is the yellow channel value, and K is the black channel value;

For the face area of interest, use the threshold value of (1)-(3) to calculate, and the skin color threshold condition in the RGB color space, the skin color threshold condition in the YCrCb color space and the skin color threshold in the CMYK color space in the face interest area Conditional pixels are reserved, and the RGB values of pixels that do not meet the threshold conditions are set to (0,0,0), so as to eliminate the environmental information of the edge of the face and obtain an accurate region of interest on the face.

The original IPPG signal is obtained by accumulating and averaging all the processed skin pixels.

In this embodiment 3, using the second calculation module to obtain the chrominance signal includes: performing RGB three-channel extraction on the IPPG signal, and then performing normalization processing on the RGB channel signal; projecting the normalized RGB value to two normal channels Intersecting chrominance vectors; filtering two orthogonal vectors with a Butterworth bandpass filter to obtain filtered signals; calculating chrominance signals based on the filtered signals. specific:

Using the CHROM algorithm, the original IPPG signal is processed.

After the CHROM algorithm, most of the noise in the original IPPG signal is eliminated. Then perform the VMD algorithm on S(t) to obtain a more accurate heart rate signal.

In Example 3, the alternate direction multiplier algorithm is used to update and iteratively solve the saddle point in the calculation of the orthogonal chromaticity vector, and the augmented Lagrange function and the constrained variational model are used to iteratively update the modal component and the augmented Lagrange function multiplier in the frequency domain , until the iteration termination condition is met, and the modal components are obtained.

In Embodiment 3, the VMD algorithm decomposes the signal into K modal components.

The difference from Embodiment 2 is that the modal component is no longer fixed at 3, and the penalty factor α is no longer fixed at 2000. Before the modal decomposition, it is necessary to set the parameters, the number of modal components K and the penalty factor α. Improper parameters may cause modal aliasing of the signal, loss of signal frequency components, etc. In order to solve the above situation, the present invention proposes an automatic optimization algorithm for the number K of modal components and the penalty factor α, and performs automatic parameter confirmation.

As shown in Figure 5, the automatic optimization process of the number K of modal components and the penalty factor α includes:

①Because the main frequency domain range of heart rate is small (about 0.7-4Hz), so, set the number of modal components K∈[1, 4], and the step size is 1, and get 4 K values; set the penalty factor The range of α is α∈[0, 2000], the step size of penalty factor α is 400, and 6 values of penalty factor α are obtained. If α is properly selected, the correlation between the modal components is small, and if α is not selected properly, the correlation between the modal components will become larger.

②Combine 4 K values and 6 penalty factor α values to obtain 24 parameter pairs, judge whether the modal components obtained by each parameter pair meet the optimization correlation constraints and frequency loss constraints, and keep Parameter pairs for the optimal correlation constraint and the frequency loss constraint:

The optimal correlation constraints are:

表示第c次寻优和第c-1次寻优相关性的比值；当α设置过大后，或导致模态分量之间的相关性突然上升，所以，将判断阈值设定为了0.5，当相关性比值小于0.5时，保留c-1次寻优值α为该模态数K值下的α；最后得到4对参数对，即4对K和α。例如，[K＝1,α＝400]，[K＝2,α＝400]，[K＝3,α＝400]，[K＝4,α＝400]。其中α是不确定的，当无法满足条件时，α迭代到2000停止，并使用2000为α值。Among them, C( ) represents the correlation function, k is the k-th modal component, and K is the number of modal components; MC is the average correlation value between two adjacent modes under a fixed modal number K; C represents the optimization times of parameter α, the formula

Indicates the ratio of the correlation between the cth optimization and the c-1th optimization; when α is set too large, it may cause the correlation between the modal components to rise suddenly, so the judgment threshold is set to 0.5, when When the correlation ratio is less than 0.5, retain the c-1 optimization value α as the α under the value of the modal number K; finally get 4 pairs of parameters, that is, 4 pairs of K and α. For example, [K=1, α=400], [K=2, α=400], [K=3, α=400], [K=4, α=400]. Where α is uncertain, when the condition cannot be met, α iterations stop at 2000, and 2000 is used as the value of α.

In the process of modal decomposition, frequency loss may occur, and the limiting conditions of frequency loss are:

Among them, S(t) is the IPPG signal processed by the chroma algorithm, and ||·|| ₂ is the two-norm; keep the parameter pairs smaller than the threshold condition.

③Use the method of maximum envelope kurtosis to select the optimal parameter pair among the parameter pairs retained in step ②:

时模态数K下第k模态分量的希尔伯特变换的模；需要注意的是，因为步骤②保留下来的参数对中，每个模态数K下，只有一对参数对，所以在数量上，参数对的数量和模态数K的数量是等价的。where k is the kth modal component,

The modulus of the Hilbert transform of the k-th modal component under the modal number K; it should be noted that, because of the parameter pairs retained in step ②, there is only one pair of parameter pairs under each modal number K, so Quantitatively, the number of parameter pairs is equivalent to the number of modal numbers K.

为

的四阶中心矩，σ(·)为平方差，ek_k为第k模态分量希尔伯特变换取模后的峰度值；Among them, the molecule

for

The fourth-order central moment of , σ( ) is the square difference, ek _k is the kurtosis value after the modulo Hilbert transform of the k-th modal component;

为K模态数下，所有模态分量希尔伯特变换取模后的峰度值组成的向量，

为

中最大的峰度值，

为所有参数对中最大峰度值；in,

is a vector composed of the kurtosis values of all modal components Hilbert transform modulo under the K mode number,

for

The largest kurtosis value in ,

Maximum kurtosis value for all parameter pairs;

中模态数K值和α，就完成了对参数的自动寻优。④ Return at last

The automatic optimization of the parameters is completed when the K value and α of the mode number are determined.

Among the K modal components output by the automatic optimization VMD algorithm, the modal component with the largest peak value in the frequency spectrum is the heart rate signal; the final heart rate value can be obtained by solving the maximum peak frequency through Fourier transform.

The specific steps involved in Embodiment 3 are consistent with Embodiment 2, and will not be repeated here.

Example 4

In Embodiment 4, a non-contact heart rate measurement method is provided, which method includes the following steps:

Obtain multiple frames of face images of the person to be tested;

Determine the face region of interest in each frame of face image;

Accumulate and average the skin pixels in the region of interest of all faces to obtain the IPPG signal;

Obtain a chrominance signal based on the IPPG signal;

A variational mode decomposition algorithm is performed on the chrominance signal to obtain multiple modal components, and the modal component with the largest peak value in the spectrum is the heart rate signal.

As shown in Fig. 1, in the non-contact heart rate measuring method described in the present embodiment 4, at first, face detection is carried out to the obtained face image through the landmark face recognition model, and the region of interest (ROI) is obtained, and then Skin color detection removes the edge environment pixels, and then obtains the original pulse wave signal (IPPG), extracts and generates RGB three-channel signals, then uses the Chrom algorithm to obtain the chroma signal, and uses the variational mode decomposition (VDM algorithm) to obtain the chroma signal The denoised IPPG signal is finally subjected to Fourier transform to obtain the heart rate.

In this embodiment 4, the human face video image of the subject is acquired by an industrial camera, and the halogen lamp 2 (whose light wave range includes visible light and invisible light range) is used to supplement the light on the human face to obtain better image quality. Input the captured face video sequence into the PC terminal 3 through the USB3.0 data transmission line and wait for the PC terminal to process the image. First, frame processing is performed on the captured video to obtain each frame of face image.

The face recognition program uses the landmark face recognition model generated by the regression tree method (Ensemble of Regression Trees, ERT) based on gradient enhancement learning. Then, 68 face marker points are obtained through the landmark face recognition model, and the face area of interest is shown in Figure 2.

In the detection process, it is inevitable that the edge of the face contains a small amount of environmental information. In the subsequent use of the skin color detector, the area of interest is eliminated by the skin color detector to remove the environmental information of the edge of the face.

The IPPG signal is obtained by accumulating and averaging all the processed skin pixels.

Using the CHROM algorithm to process the original IPPG signal, the steps are as follows:

The RGB three-channel extraction is performed on the original IPPG, and then the RGB channel signal is normalized.

Project the normalized RGB values to two orthogonal chrominance vectors X _chrom and Y _chrom with the following formula:

X _chrom (t)＝3x _r (t)-2x _g (t) (2)

Y _chrom (t)＝1.5x _r (t)+x _g (t)-1.5x _b (t) (3)

The final output S(t) is:

S(t)= _Xf - _αYf ; (4)

Among them, x _r represents the red channel component of the original IPPG signal, x _g represents the green channel component of the original IPPG signal, x _b represents the blue channel component of the original IPPG signal, t represents time, and the signal waveform changes with time t, X _f , Y _f are the signals of X _chrom and Y _chrom filtered by the fifth-order 0.7Hz-4Hz Butterworth bandpass filter; α is the ratio of the standard deviations of X _f and Y _f , and S(t) is the The IPPG signal.

After the CHROM algorithm, most of the noise in the original IPPG signal is eliminated. Then perform the VMD algorithm on S(t) to obtain a more accurate heart rate signal.

The constrained variational models involved in the VMD algorithm are as follows:

表示偏导运算，δ(t)表示单位脉冲函数，j表示虚数单位，*表示卷积运算，f表示目标信号，e表示超越数。Among them, {u _k }={u ₁ ,u ₂ ,...,u _k } means the kth modal component, {w _k }={w ₁ ,w ₂ ,...,w _k } means the The center frequency of k modal components, K represents the number of modal components,

Represents the partial derivative operation, δ(t) represents the unit impulse function, j represents the imaginary number unit, * represents the convolution operation, f represents the target signal, and e represents the transcendental number.

A penalty factor α and a Lagrange multiplier λ are introduced to solve the variational constrained problem. The resulting augmented Lagrange expression is as follows:

The saddle point in equation (7) is iteratively solved by using the alternate multiplier algorithm to update iteratively, and u _k , w _k , λ are iteratively updated in the frequency domain.

The VMD algorithm decomposes the signal into three modal components (the number of modal components is determined by experiments), and the penalty factor α is 2000. The decomposition steps are as follows:

n为零；(1) Initialization

n is zero;

(2) u _k and w _k are iteratively updated by equations (8) and (9) respectively:

Among them, ω represents the frequency, i represents the i-th modal component, and d represents the derivative.

(3) Update λ through formula (10):

Among them, τ represents the fidelity coefficient, ∧ represents the Fourier transform, and n represents the number of iterations.

(4) Repeat steps 2 and 3 until the iteration termination condition is met, which is given by (11).

Among them, ε represents the discrimination accuracy, and ε>0, the superscript n is the number of iteration steps, and the subscript k represents the current mode number.

(5) Output 3 modal components.

The schematic diagram of the processing process of the human face image described in Embodiment 4 is shown in Figure 3. After the landmark human face recognition returns the marker point, the region of interest is drawn according to the figure in the upper left corner in Figure 2, and then the edge is removed by skin color detection. Then the pulse wave is processed by RGB three-channel, the RGB three-channel signal is processed by CHROM algorithm to obtain the pulse wave signal with light intensity and color removed, and then the mode decomposition and denoising are carried out by VDM, and the heart rate signal waveform and the heart rate signal are reconstructed. Fourier spectrogram.

In Example 4, through the precise positioning of the face by the landmark model, the CHROM algorithm and the VMD algorithm are used to denoise the original IPPG signal, thereby obtaining an accurate heart rate and realizing the acquisition of the heart rate in a non-contact detection manner.

It solves the following main technical problems:

Face tracking detection has a better tracking effect on large-scale movement of the face, but when the face is relatively still, each frame of the video is relatively independent, resulting in the jitter of the face recognition marker points, resulting in additional Introduce noise; the movement of the face will cause changes in the incident angle of light and skin color, thereby introducing noise. The intensity, light intensity and color of the pixel value can be separated through the CHROM algorithm to eliminate noise; VMD is performed on the pulse wave signal, according to the heartbeat frequency Features, using the VMD algorithm to decompose the main signal into different modes, ensuring that the signal frequency ranges between the modes do not overlap each other, and separating a relatively complete heartbeat signal without harmonic residue.

Example 5

In Embodiment 5, a non-contact heart rate measurement method is provided, which method includes the following steps:

Obtain multiple frames of face images of the person to be tested;

Determine the face region of interest in each frame of face image;

Accumulate and average the skin pixels in the region of interest of all faces to obtain the IPPG signal;

Obtain a chrominance signal based on the IPPG signal;

A variational mode decomposition algorithm is performed on the chrominance signal to obtain multiple modal components, and the modal component with the largest peak value in the spectrum is the heart rate signal.

As shown in Figure 1, in the non-contact heart rate measuring method described in the present embodiment 5, at first, face detection is carried out to the acquired face image by the landmark face recognition model, and the region of interest (ROI) is obtained, and then Skin color detection removes the edge environment pixels, and then obtains the original pulse wave signal (IPPG), extracts and generates RGB three-channel signals, then uses the Chrom algorithm to obtain the chroma signal, and uses the variational mode decomposition (VDM algorithm) to obtain the chroma signal The denoised IPPG signal is finally subjected to Fourier transform to obtain the heart rate.

In this embodiment 5, the human face video image of the subject is obtained by an industrial camera, and a halogen lamp (whose light wave range includes visible light and invisible light range) is used to supplement the light on the human face to obtain better image quality. Input the captured face video sequence to the PC through the USB3.0 data transmission line and wait for the PC to process the image. First, frame processing is performed on the captured video to obtain each frame of face image.

The face recognition program uses the landmark face recognition model generated by the regression tree method (Ensemble of Regression Trees, ERT) based on gradient enhancement learning. Then obtain 68 face marker points through the landmark model, use marker points 0, 8, and 16 to generate marker points HL and marker points HR, and the coordinate points of HL and HR are calculated as follows:

Among them, y _p8 is the ordinate of marking point 8, y _p0 is the ordinate of marking point 0, x _p0 is the abscissa of marking point 0, y _p0 is the ordinate of marking point 0, x _p16 is the abscissa of marking point 16 Coordinates, y _p16 is the vertical coordinate of the marker point 16; x _HR , y _HR and x _HL , y _HL are decomposed into the horizontal and vertical coordinates of the new marker point HR and the horizontal and vertical coordinates of the new marker point HL; Points 16, HR, and HL are sequentially connected to obtain the face area, and the mark point 48 to mark point 59 are sequentially connected to remove the mouth area, and finally the face mask map is obtained, that is, the face area of interest, as shown in the upper left corner of Figure 2 As shown in the figure.

In the detection process, it is inevitable that the edge of the face contains a small amount of environmental information. In the subsequent use of the skin color detector, the area of interest is eliminated by the skin color detector to remove the environmental information of the edge of the face. Specifically, the skin color detector uses a multi-color space skin color detection algorithm to remove non-skin interference as much as possible while retaining skin area information. The core method is: use RGB, YcrCb and CMYK three different color space skin color thresholds for accurate Skin color pixel judgment.

The multi-color space skin color detection algorithm is specifically:

(1) Skin color threshold conditions in the RGB color space are as follows:

R＞95∩G＞40∩B＞20∩R>G∩R＞B∩|R-G|>15

Among them, R is the value of the red channel, G is the value of the green channel, and B is the value of the blue channel;

(2) The skin color threshold condition in the YCrCb color space is as follows:

Cr>135∩Cb>85∩Yl>80∩Cr<=(1.5874*Cb)+20∩

Cr＞＝(0.3447*Cb)+76.2068∩Cr＞＝(-4.5653*Cb)+234.5652∩

Cr<=(-1.15*Cb)+301.78∩Cr<=(-2.2868*Cb)+433.85

Wherein, Yl is the color brightness value, Cr is the concentration offset value of red, and Cb is the blue component offset value;

(3) Skin color threshold conditions in CMYK color space are as follows:

K<0.8∩0<=C<0.05∩0.1<=Y/M<4.8

∩0.088<Y<1∩0<C/Y<1

Among them, C is the cyan channel value, M is the magenta channel value, Y is the yellow channel value, and K is the black channel value;

For the face area of interest, use the threshold value of (1)-(3) to calculate, and the skin color threshold condition in the RGB color space, the skin color threshold condition in the YCrCb color space and the skin color threshold in the CMYK color space in the face interest area Conditional pixels are reserved, and the RGB values of pixels that do not meet the threshold conditions are set to (0,0,0), so as to eliminate the environmental information of the edge of the face and obtain an accurate region of interest on the face.

The IPPG signal is obtained by accumulating and averaging all the processed skin pixels.

Using the CHROM algorithm to process the original IPPG signal, the steps are as follows:

The RGB three-channel extraction is performed on the original IPPG, and then the RGB channel signal is normalized.

Project the normalized RGB values to two orthogonal chrominance vectors X _chrom and Y _chrom .

After the CHROM algorithm, most of the noise in the original IPPG signal is eliminated. Then perform the VMD algorithm on S(t) to obtain a more accurate heart rate signal.

The VMD algorithm decomposes the signal into K modal components.

The difference from Embodiment 4 is that the modal component is no longer fixed at 3, and the penalty factor α is no longer fixed at 2000. Before the modal decomposition, it is necessary to set the parameters, the number of modal components K and the penalty factor α. Improper parameters may cause modal aliasing of the signal, loss of signal frequency components, etc. In order to solve the above situation, the present invention proposes an automatic optimization algorithm for the number K of modal components and the penalty factor α, and performs automatic parameter confirmation.

The automatic optimization process of the number K of modal components and the penalty factor α includes:

①Because the main frequency domain range of heart rate is small (about 0.7-4Hz), so, set the number of modal components K∈[1, 4], and the step size is 1, and get 4 K values; set the penalty factor The range of α is α∈[0, 2000], the step size of penalty factor α is 400, and 6 values of penalty factor α are obtained. If α is properly selected, the correlation between the modal components is small, and if α is not selected properly, the correlation between the modal components will become larger.

②Combine 4 K values and 6 penalty factor α values to obtain 24 parameter pairs, judge whether the modal components obtained by each parameter pair meet the optimization correlation constraints and frequency loss constraints, and keep Parameter pairs for the optimal correlation constraint and the frequency loss constraint:

The optimal correlation constraints are:

表示第c次寻优和第c-1次寻优相关性的比值；当α设置过大后，或导致模态分量之间的相关性突然上升，所以，将判断阈值设定为了0.5，当相关性比值小于0.5时，保留c-1次寻优值α为该模态数K值下的α；最后得到4对参数对，即4对K和α。例如，[K＝1,α＝400]，[K＝2,α＝400]，[K＝3,α＝400]，[K＝4,α＝400]。其中α是不确定的，当无法满足条件时，α迭代到2000停止，并使用2000为α值。Among them, C( ) represents the correlation function, k is the k-th modal component, and K is the number of modal components; MC is the average correlation value between two adjacent modes under a fixed modal number K; C represents the optimization times of parameter α, the formula

Indicates the ratio of the correlation between the cth optimization and the c-1th optimization; when α is set too large, it may cause the correlation between the modal components to rise suddenly, so the judgment threshold is set to 0.5, when When the correlation ratio is less than 0.5, retain the c-1 optimization value α as the α under the value of the modal number K; finally get 4 pairs of parameters, that is, 4 pairs of K and α. For example, [K=1, α=400], [K=2, α=400], [K=3, α=400], [K=4, α=400]. Where α is uncertain, when the condition cannot be met, α iterations stop at 2000, and 2000 is used as the value of α.

In the process of modal decomposition, frequency loss may occur, and the limiting conditions of frequency loss are:

Among them, S(t) is the IPPG signal processed by the chroma algorithm, and ||·|| ₂ is the two-norm; keep the parameter pairs smaller than the threshold condition.

③Use the method of maximum envelope kurtosis to select the optimal parameter pair among the parameter pairs retained in step ②:

时模态数K下第k模态分量的希尔伯特变换的模；需要注意的是，因为步骤②保留下来的参数对中，每个模态数K下，只有一对参数对，所以在数量上，参数对的数量和模态数K的数量是等价的。where k is the kth modal component,

The modulus of the Hilbert transform of the k-th modal component under the modal number K; it should be noted that, because of the parameter pairs retained in step ②, there is only one pair of parameter pairs under each modal number K, so Quantitatively, the number of parameter pairs is equivalent to the number of modal numbers K.

为

的四阶中心矩，σ(·)为平方差，ek_k为第k模态分量希尔伯特变换取模后的峰度值；Among them, the molecule

for

The fourth-order central moment of , σ( ) is the square difference, ek _k is the kurtosis value after the modulo Hilbert transform of the k-th modal component;

为K模态数下，所有模态分量希尔伯特变换取模后的峰度值组成的向量，

为

中最大的峰度值，

为所有参数对中最大峰度值；in,

is a vector composed of the kurtosis values of all modal components after Hilbert transform modulo under the K mode number,

for

The largest kurtosis value in ,

Maximum kurtosis value for all parameter pairs;

中模态数K值和α，就完成了对参数的自动寻优。④ Return at last

The automatic optimization of the parameters is completed when the K value and α of the mode number are determined.

Among the K modal components output by the automatic optimization VMD algorithm, the modal component with the largest peak value in the frequency spectrum is the heart rate signal; the final heart rate value can be obtained by solving the maximum peak frequency through Fourier transform.

The specific steps involved in Embodiment 5 are consistent with Embodiment 4, and will not be repeated here.

Example 6

Embodiment 6 of the present invention provides a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, a non-contact heart rate measurement method is implemented instructions, the method includes:

Obtain multiple frames of face images of the person to be tested;

Determine the face region of interest in each frame of face image;

Accumulate and average the skin pixels in the region of interest of all faces to obtain the IPPG signal;

Obtain a chrominance signal based on the IPPG signal;

A variational mode decomposition algorithm is performed on the chrominance signal to obtain multiple modal components, and the modal component with the largest peak value in the spectrum is the heart rate signal.

Example 7

Embodiment 7 of the present invention provides a computer program (product), including a computer program. When the computer program is run on one or more processors, it is used to implement the non-contact heart rate measurement method as described above. The method include:

Obtain multiple frames of face images of the person to be tested;

Determine the face region of interest in each frame of face image;

Accumulate and average the skin pixels in the region of interest of all faces to obtain the IPPG signal;

Obtain a chrominance signal based on the IPPG signal;

A variational mode decomposition algorithm is performed on the chrominance signal to obtain multiple modal components, and the modal component with the largest peak value in the spectrum is the heart rate signal.

Example 8

Embodiment 8 of the present invention provides an electronic device, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the A computer program stored in the memory to cause the electronic device to perform the non-contact heart rate measurement method as described above, the method comprising:

Obtain multiple frames of face images of the person to be tested;

Determine the face region of interest in each frame of face image;

Accumulate and average the skin pixels in the region of interest of all faces to obtain the IPPG signal;

Obtain a chrominance signal based on the IPPG signal;

A variational mode decomposition algorithm is performed on the chrominance signal to obtain multiple modal components, and the modal component with the largest peak value in the spectrum is the heart rate signal.

In summary, the non-contact heart rate detection method and system described in the embodiments of the present invention can separate the intensity of the pixel value from the light intensity and color through the CHROM algorithm, and eliminate noise; the VMD algorithm is performed on the signal, and according to the heartbeat frequency characteristics, The VMD algorithm is used to decompose the main signal into different modes, which ensures that the signal frequency ranges between the modes do not overlap each other, and separates a relatively complete heartbeat signal without harmonic residue.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device Steps are provided for implementing the functions specified in the flow chart or flow charts and/or block diagram block or blocks.

Although the specific implementation of the present invention has been described above in conjunction with the accompanying drawings, it does not limit the protection scope of the present invention. Those skilled in the art should understand that on the basis of the technical solutions disclosed in the present invention, those skilled in the art do not need to pay Various modifications or deformations that can be made through creative labor shall be covered within the scope of protection of the present invention.

Claims (10)

1. A non-contact heart rate measurement method, comprising: Obtain multiple frames of face images of the person to be tested; Determine the face region of interest in each frame of face image; Accumulate and average the skin pixels in the region of interest of all faces to obtain the IPPG signal; Obtain a chrominance signal based on the IPPG signal; A variational mode decomposition algorithm is performed on the chrominance signal to obtain multiple modal components, and the modal component with the largest peak value in the spectrum is the heart rate signal. 2. The non-contact heart rate measuring method according to claim 1, wherein obtaining the face region of interest comprises: utilizing the landmark face recognition model to obtain the face marker points, setting the face region of interest, using skin color The detector removes the environmental information at the edge of the face. 3. The non-contact heart rate measurement method according to claim 2, wherein the skin color detector uses a multi-color space skin color detection algorithm. 4. The non-contact heart rate measurement method according to claim 1, wherein obtaining the chromaticity signal comprises: RGB three-channel extraction is performed on the IPPG signal, and then the RGB channel signal is normalized; Project the normalized RGB values to two orthogonal chrominance vectors; Filter the two orthogonal vectors with a Butterworth bandpass filter to obtain the filtered signal; A chrominance signal is calculated based on the filtered signal. 5. The non-contact heart rate measurement method according to claim 4, characterized in that, adopt the alternate direction multiplier algorithm to update and iteratively solve the saddle point in the orthogonal chromaticity vector calculation, and use the augmented Lagrange function and the constraint variable in the frequency domain The modal components and the augmented Lagrange function multipliers are iteratively updated for each model until the iteration termination condition is met, and the modal components are obtained. 6. The non-contact heart rate measuring method according to claim 5, characterized in that: The constrained variational models involved in the variational mode decomposition algorithm are:

Among them, {u _k } represents the kth modal component, {w _k } represents the center frequency of the kth modal component, K represents the number of modal components,

Represents the partial derivative operation, δ(t) represents the unit pulse function, j represents the imaginary number unit, * represents the convolution operation, and f represents the target signal; The penalty factor α and the Lagrange multiplier λ are introduced to solve the variational constraint problem, and the expression of the augmented Lagrange function is as follows:

7. The non-contact heart rate measurement method according to claim 6, characterized in that: In the frequency domain, the center frequencies of the modal components are updated using the following formula:

Among them, ω represents the frequency, i represents the i-th modal component, and d represents the derivative. Combine the following formula to update λ:

Among them, τ represents the fidelity coefficient, ∧ represents the Fourier transform, and n represents the number of iterations. 8. The non-contact heart rate measurement method according to claim 7, wherein the iteration termination condition is:

Among them, ε represents the discrimination accuracy, and ε>0. 9. A non-contact heart rate measurement system, comprising: An acquisition module, configured to acquire multiple frames of human face images of the person to be tested; A recognition module, configured to determine the face region of interest in each frame of face images; The first calculation module is used to accumulate and average the skin pixels in the region of interest of all faces to obtain the IPPG signal; The second calculation module is used to obtain the chrominance signal based on the IPPG signal; The third calculation module is used to perform a variational mode decomposition algorithm on the chrominance signal to obtain multiple mode components, and the mode component with the largest peak value in the frequency spectrum is the heart rate signal. 10. An electronic device, characterized in that it comprises: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, and the computer program is stored in the memory, and when the electronic device runs, the processor executes the The computer program stored in the memory enables the electronic device to execute the instructions of the non-contact heart rate measurement method according to any one of claims 1-8.

Images