CN114569101A - Non-contact heart rate detection method and device and electronic equipment - Google Patents
Non-contact heart rate detection method and device and electronic equipment Download PDFInfo
- Publication number
- CN114569101A CN114569101A CN202210222186.XA CN202210222186A CN114569101A CN 114569101 A CN114569101 A CN 114569101A CN 202210222186 A CN202210222186 A CN 202210222186A CN 114569101 A CN114569101 A CN 114569101A
- Authority
- CN
- China
- Prior art keywords
- color space
- heart rate
- image
- signal
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 86
- 239000008280 blood Substances 0.000 claims abstract description 69
- 210000004369 blood Anatomy 0.000 claims abstract description 69
- 238000000034 method Methods 0.000 claims abstract description 62
- 238000004458 analytical method Methods 0.000 claims abstract description 31
- 238000006243 chemical reaction Methods 0.000 claims abstract description 24
- 238000000605 extraction Methods 0.000 claims abstract description 18
- 238000012545 processing Methods 0.000 claims abstract description 18
- 238000004422 calculation algorithm Methods 0.000 claims description 26
- 238000001914 filtration Methods 0.000 claims description 16
- 230000003287 optical effect Effects 0.000 claims description 10
- 238000012795 verification Methods 0.000 claims description 4
- 230000007613 environmental effect Effects 0.000 abstract description 5
- 230000000875 corresponding effect Effects 0.000 description 27
- 238000005259 measurement Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 9
- 238000002474 experimental method Methods 0.000 description 9
- 238000009532 heart rate measurement Methods 0.000 description 9
- 230000008859 change Effects 0.000 description 7
- 239000000284 extract Substances 0.000 description 7
- 210000004204 blood vessel Anatomy 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000013507 mapping Methods 0.000 description 5
- 238000013186 photoplethysmography Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 238000001228 spectrum Methods 0.000 description 4
- 238000007920 subcutaneous administration Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000010183 spectrum analysis Methods 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 208000024172 Cardiovascular disease Diseases 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000010521 absorption reaction Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 230000002596 correlated effect Effects 0.000 description 2
- 210000004207 dermis Anatomy 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 230000002526 effect on cardiovascular system Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000031700 light absorption Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 102000001554 Hemoglobins Human genes 0.000 description 1
- 108010054147 Hemoglobins Proteins 0.000 description 1
- 238000002835 absorbance Methods 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000010009 beating Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 208000026106 cerebrovascular disease Diseases 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000002565 electrocardiography Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011158 quantitative evaluation Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000284 resting effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 210000003491 skin Anatomy 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000001931 thermography Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/024—Detecting, measuring or recording pulse rate or heart rate
- A61B5/02416—Detecting, measuring or recording pulse rate or heart rate using photoplethysmograph signals, e.g. generated by infrared radiation
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0062—Arrangements for scanning
- A61B5/0064—Body surface scanning
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0075—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/024—Detecting, measuring or recording pulse rate or heart rate
- A61B5/02416—Detecting, measuring or recording pulse rate or heart rate using photoplethysmograph signals, e.g. generated by infrared radiation
- A61B5/02427—Details of sensor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/725—Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Surgery (AREA)
- Public Health (AREA)
- Pathology (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Cardiology (AREA)
- Physiology (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
Abstract
The invention provides a non-contact heart rate detection method, a non-contact heart rate detection device and electronic equipment, wherein the method comprises the following steps: acquiring a target image containing a target object through a preset camera; determining a region of interest based on the target image; carrying out color space conversion on an image area corresponding to the region of interest to obtain an image signal under an appointed color space; performing signal extraction on the image signal in the designated color space to obtain a blood volume pulse signal, wherein the blood volume pulse signal comprises time-frequency information of the signal; a heart rate of the target subject is determined based on the blood volume pulse signal. According to the method, the camera is used for collecting the image of the target object in real time, time-frequency analysis processing is carried out on the image, and a heart rate value is obtained, so that non-contact heart rate detection is realized, the application in an actual scene is met, the heart rate can be detected in a natural light state, and the method has the characteristics of convenience in operation, low equipment requirement, small environmental restriction, strong instantaneity, high accuracy and the like.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a non-contact heart rate detection method and device and electronic equipment.
Background
With the development and the acceleration of the modernization process of the society, the living habits of some people are changed and influenced by other factors, so that the morbidity and the mortality of cardiovascular and cerebrovascular diseases are increased year by year. According to modern medical research, when a person's resting heart rate exceeds 90/min, the incidence of cardiovascular disease is positively correlated with heart rate speed. Among them, most cardiovascular-related diseases can be reasonably prevented and avoided. Therefore, the daily heart rate detection scheme has stronger requirements on convenience in use, wide scenes and high accuracy.
In the related art, the heart rate detection can be divided into contact type heart rate detection and non-contact type heart rate detection. Among them, research and application of non-contact heart rate detection based on IPPG (Image Photoplethysmography) mainly extract BVP (Blood Volume Pulse) signals in video signals, and further estimate heart rate, and most documents and systems can estimate heart rate accurately, but do not have good real-time performance, which needs to be improved and promoted in both clinical medicine and home heart rate detection scenarios.
Disclosure of Invention
The invention aims to provide a non-contact heart rate detection method, a non-contact heart rate detection device and electronic equipment so as to improve the accuracy and the real-time performance of heart rate detection.
In a first aspect, the present invention provides a method for non-contact heart rate detection, the method comprising: acquiring a target image containing a target object through a preset camera; determining a region of interest based on the target image; carrying out color space conversion on an image area corresponding to the region of interest to obtain an image signal under an appointed color space; carrying out signal extraction on the image signal in the specified color space to obtain a blood volume pulse signal; wherein, the blood volume pulse signal comprises time-frequency information of the signal; based on the blood volume pulse signal, a heart rate of the target subject is determined.
In an alternative embodiment, the step of determining the region of interest based on the target image includes: carrying out face recognition on the target image to obtain a face image; and determining the region of interest from the face image.
In an optional embodiment, after the step of determining the region of interest based on the target image, the method further includes: determining a target offset within the region of interest; and correcting the region of interest according to the target offset to obtain a corrected region of interest, and taking the corrected region of interest as a final region of interest.
In an alternative embodiment, the step of determining the target offset within the region of interest includes: performing target tracking on the target image through a sparse optical flow tracking algorithm; and calculating the target offset in the region of interest according to the target tracking result.
In an alternative embodiment, the specified color space is a Lab color space; the step of performing color space conversion on the image area corresponding to the region of interest to obtain the image signal in the designated color space includes: extracting RGB channel characteristic values of an image area corresponding to the interested area to obtain an image signal in an RGB color space; and carrying out color space conversion on the image signals in the RGB color space to obtain the image signals in the Lab color space.
In an optional embodiment, the step of performing color space conversion on the image signal in the RGB color space to obtain the image signal in the Lab color space includes: converting the image signal in the RGB color space into an XYZ color space to obtain an image signal in the XYZ color space; and converting the image signal in the XYZ color space into the Lab color space to obtain the image signal in the Lab color space.
In an alternative embodiment, the step of extracting the image signal in the designated color space to obtain the blood volume pulse signal includes: filtering the image signal in the designated color space to obtain a filtered signal; and performing CMOR4-2 wavelet transform on the filtered signal to obtain a blood volume pulse signal.
In an alternative embodiment, the specified color space is a Lab color space; the step of performing filtering processing on the image signal in the designated color space to obtain a filtered signal includes: and performing band-pass filtering on the image signal corresponding to the channel a in the Lab color space to obtain a filtering signal.
In an alternative embodiment, the step of determining the heart rate of the target subject based on the blood volume pulse signal includes: performing time-frequency analysis on the blood volume pulse signal to obtain an initial heart rate value; and carrying out verification processing on the initial heart rate value to obtain the heart rate of the target object.
In a second aspect, the invention provides a non-contact heart rate detection device, comprising: the image acquisition module is used for acquiring a target image containing a target object through a preset camera; the interesting region determining module is used for determining an interesting region based on the target image; the color space conversion module is used for performing color space conversion on the image area corresponding to the region of interest to obtain an image signal in a specified color space; the signal extraction module is used for extracting signals of the image signals in the specified color space to obtain blood volume pulse signals; wherein, the blood volume pulse signal contains time-frequency information of the signal; a heart rate determination module for determining a heart rate of the target subject based on the blood volume pulse signal.
In a third aspect, the present invention provides an electronic device comprising a processor and a memory, the memory storing machine executable instructions capable of being executed by the processor, the processor executing the machine executable instructions to implement the contactless heart rate detection method according to any one of the preceding embodiments.
The embodiment of the invention has the following beneficial effects:
the invention provides a non-contact heart rate detection method, a non-contact heart rate detection device and electronic equipment.A target image containing a target object is collected through a preset camera; then, determining an interested region based on the target image so as to perform color space conversion on an image region corresponding to the interested region to obtain an image signal under an appointed color space; then, carrying out signal extraction on the image signal in the appointed color space to obtain a blood volume pulse signal, wherein the blood volume pulse signal comprises time-frequency information of the signal; the heart rate of the target subject is then determined based on the blood volume pulse signal. According to the mode, the image of the target object is collected in real time through the camera, time-frequency analysis processing is carried out on the image, and a heart rate value is obtained, so that non-contact heart rate detection is achieved, the application in an actual scene is met, the heart rate can be detected in a natural light state, and the method has the characteristics of convenience and rapidness in operation, low equipment requirement, small environmental restriction, strong real-time performance, high accuracy and the like.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the invention as set forth above.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a non-contact heart rate detection method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a measurement device and environment provided by an embodiment of the present invention;
FIG. 3 is a flow chart of another non-contact heart rate detection method according to an embodiment of the invention;
FIG. 4 is a flow chart of another non-contact heart rate detection method according to an embodiment of the invention;
FIG. 5 is a diagram illustrating a 15 second BVP signal according to an embodiment of the present invention;
fig. 6 is a schematic diagram of time-frequency analysis of a 15-second BVP signal according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating a 15 second BVP signal according to an embodiment of the present invention;
FIG. 8 is a spectrum diagram of a spectrum analysis using FFT according to an embodiment of the present invention;
FIG. 9 is a graph of a spectrum of a time-frequency analysis performed using CMOR4-2 according to an embodiment of the present invention;
FIG. 10 is a heart rate estimate correlation scatter plot provided by an embodiment of the present invention;
fig. 11 is a schematic structural diagram of a non-contact heart rate detecting device according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The related art heart rate detection can be classified into contact type heart rate detection and non-contact type heart rate detection, wherein the contact type heart rate detection includes conventional pulse cutting method, Electrocardiography (ECG), Photoplethysmography (PPG), and the non-contact type heart rate detection may include thermal imaging method, ultrasonic doppler method, imaging type photo-plethysmography (IPPG). The main problem of contact heart rate detection is that the measurement scenario is limited and is greatly limited by the equipment, while the problem of non-contact heart rate detection is that the accuracy of heart rate detection is greatly affected by other external factors.
However, with the upgrade of computer platforms and the popularization and application of camera sensors, as well as the maturity and development of various algorithms including face recognition and image processing, the implementation of non-contact heart rate detection becomes possible and has higher reliability. In the non-contact heart rate detection related art, the optical principle and the physiological principle of imaging type photoplethysmography enable the non-contact heart rate detection. The beer-Lambert law, as an optical principle of non-contact heart rate detection, determines the difference degree of video signal acquisition, and the light absorption amount is in direct proportion to the blood volume of blood vessels. The blood volume of the capillary vessels of the human body changes periodically along with the pulse and becomes the basis for extracting a blood volume pulse signal (BVP for short), and a clear BVP signal can be obtained by further processing an original signal, so that the heart rate is accurately estimated.
The other key field in the non-contact heart rate detection related technology is face recognition and ROI (region of interest) region extraction, the face recognition is to perform dynamic recognition on a face part in a signal acquisition video, and the accuracy of the dynamic recognition and the efficiency of an algorithm influence the anti-interference performance, the accuracy and the real-time performance of the non-contact heart rate detection to a certain extent. And the ROI extraction aims to select a part with a better BVP signal in the face video signal part, on the other hand, the processing amount of the video signal is reduced, and after the ROI signal is extracted, the next signal processing and the BVP signal extraction are carried out.
The research and application of non-contact heart rate detection based on IPPG mainly extracts BVP signals in video signals and further estimates heart rates, and most of documents and systems can accurately estimate heart rates but do not have good real-time performance, so that improvement and promotion are needed in clinical medicine and home heart rate detection scenes.
Based on the above problems, embodiments of the present invention provide a non-contact heart rate detection method, a non-contact heart rate detection device, and an electronic device, and the technology can be applied to a household or medical human heart rate detection scenario. In order to facilitate understanding of the embodiment of the present invention, a non-contact heart rate detection method disclosed in the embodiment of the present invention is first described in detail, and as shown in fig. 1, the method includes the following specific steps:
and step S102, acquiring a target image containing a target object through a preset camera.
The preset camera can be a common household camera, and can also be a camera on a mobile phone or a computer. The target object may be a person or a human face, where the person is also the subject.
In a specific implementation, an ordinary home host (CPU: AMD Ryzen 52600 GPU: AMD radiaon RX 570) and an entry-level camera (1280 × 72020 fps) may be used to perform image acquisition, the resolution of the camera is set to 1080 × 720 during the acquisition, the frame rate is 20 frames/s, the target object is kept for 15 seconds under a relatively static condition facing the camera, the measurement device and the environment are as shown in fig. 2, and the distance between the tested person and the camera in fig. 2 is 0.5 m.
And step S104, determining a region of interest based on the target image.
In practical application, the present invention adopts IPPG technology, which has the optical principles of beer-Lambert's law and scattering theory of light. The beer-lambert law is expressed as: when the light-absorbing medium is a transparent medium, the light-absorbing degree of the medium is independent of the light intensity; the larger the number of molecules that absorb light, the higher the amount of light absorbed. In the IPPG technique, the amount of light absorbed is proportional to the blood volume of blood vessels in the dermis of the human body, and in the IPPG-based heart rate detection technique, the beer-lambert law is reflected in that the larger the blood volume of capillary vessels in the dermis of the human body, the larger the amount of light absorbed.
The IPPG technology is based on the principle that a camera or other imaging devices are used for acquiring image information of the face or other parts of a subject, the change information of the blood volume of subcutaneous capillaries caused by pulse is recorded, and the heart rate of the subject is estimated through processing and analyzing and calculating the image information, so that the tissue structure of human skin is the physiological basis of IPPG.
In specific implementation, each part of a human body can carry out non-contact heart rate detection by the principle of IPPG, but because subcutaneous capillaries of the human face are most abundant, the blood volume of the blood vessels is greatly changed along with the periodicity of the pulse, and the absorption amount of the subcutaneous capillaries of the human face part to light is greatly changed relative to other parts and is changed along with the periodicity of the pulse according to the Bell-Lambert law, the human face is usually selected as the part for non-contact heart rate detection, and the part is preprocessed after physiological signals are obtained. Thus, the region of interest selected from the target image may be a face region in the target image.
And step S106, performing color space conversion on the image area corresponding to the region of interest to obtain an image signal under the specified color space.
The preset camera generally obtains images with three color channels of R (Red ), G (Green ) and B (Blue). In a specific implementation, the light absorption of hemoglobin in the face region changes with the change of the spectrum, and thus the specified color space may be an RGB color space or an Lab color space.
Under ambient light conditions, the pulse wave signals have different relative intensities in the three color channels of the RGB camera, while the red and blue channels will also contain pulse signals, but the green channel has a larger pulse magnitude and less noise. The G channel signal in the RGB color channel is sensitive to the BVP signal recording the heart activity, so that the subsequent heart rate detection analysis can be performed in the RGB color space.
The L component in the Lab color space represents the brightness value of the image pixel, namely the range of the image brightness from pure black to pure white; a represents the range of the image from red to green; b represents the range of the image from yellow to blue. The Lab color space is characterized in that the three channels are independent, namely, only brightness is influenced and color is not changed when an L channel is adjusted, only red and green are influenced and brightness and yellow blue are not changed when an a channel is adjusted, and only yellow blue is influenced and brightness and red and green are not changed when a b channel is adjusted. In the RGB mode, a certain channel can be independently adjusted through tools such as color levels, curves and the like, but since the final image is formed by overlapping three channels, other colors and brightness of the image can be influenced even if one channel is adjusted, and the brightness of the Lab color space is separated from the colors, so that the image adjustment is easy and intuitive. Thus, the RGB color space can be converted to the Lab color space for subsequent heart rate detection analysis in the Lab color space.
Step S108, extracting signals of the image signals in the specified color space to obtain blood volume pulse signals; wherein the blood volume pulse signal comprises time-frequency information of the signal.
In non-contact heart rate detection, digital filtering or blind source separation may be used for extraction of the BVP signal (corresponding to the blood volume pulse signal described above). In some embodiments, in consideration of the operation efficiency, the real-time performance, and the like, the BVP signal may be extracted from the image signal in the specified color space by a digital filtering method.
Step S110, determining the heart rate of the target object based on the blood volume pulse signal.
In specific implementation, the blood volume pulse signal can be subjected to video analysis to obtain the heart rate of the target object. In some embodiments, the blood volume pulse signal analysis may also be analyzed by FFT (Fast Fourier Transform), STFT (Short-Time Fourier Transform), or IBls (Inter-Beat Intervals) methods provided in the prior art to obtain the heart rate value, but these methods may also have certain limitations in application. For example, the IBIs analysis method extracts the heart rate parameter by calculating the peak time interval of the blood volume pulse signal, but because the blood volume pulse signal extracted by the camera in a non-contact manner has large noise interference and unobvious periodic peaks, the situations of peak deviation, missing detection, multiple detection and the like easily occur, and the accuracy of the heart rate parameter measurement result is seriously influenced.
The FFT analysis method is simple and intuitive, has small operand, and is one of the most common methods in non-contact heart rate measurement. The heart rate parameter is calculated by converting the blood volume pulse signal extracted by the video image processing algorithm into a frequency domain space and further extracting the corresponding frequency value of the maximum amplitude value in the specified pass band range of the frequency domain. However, the FFT analysis method can only extract a single main heart rate parameter within a period of time, and cannot obtain heart rate variation information within a measurement period of time. Even with STFT analysis, it is a difficult problem to trade-off the size of the time window between the time-domain and frequency-domain features. In addition, the measurement error is large under noise interference by the FFT analysis method and the STFT analysis method.
In some embodiments, in order to solve the above problems of the analysis method in the prior art, the present invention may further use the CMOR4-2 wavelet to plot an energy spectrum of the blood volume pulse signal, and simultaneously analyze the time domain component and the frequency domain component of the blood volume pulse signal, and extract a heart rate parameter with high accuracy and correct response variation information from the time domain component and the frequency domain component, so as to improve the accuracy of heart rate detection.
The embodiment of the invention provides a non-contact heart rate detection method, which comprises the steps of firstly, collecting a target image containing a target object through a preset camera; then, determining an interested region based on the target image so as to perform color space conversion on an image region corresponding to the interested region to obtain an image signal under an appointed color space; then, carrying out signal extraction on the image signal in the appointed color space to obtain a blood volume pulse signal, wherein the blood volume pulse signal comprises time-frequency information of the signal; the heart rate of the target subject is then determined based on the blood volume pulse signal. According to the method, the camera is used for collecting the image of the target object in real time, time-frequency analysis processing is carried out on the image, and a heart rate value is obtained, so that non-contact heart rate detection is realized, the application in an actual scene is met, the heart rate can be detected in a natural light state, and the method has the characteristics of convenience in operation, low equipment requirement, small environmental restriction, strong instantaneity, high accuracy and the like.
Corresponding to the above method embodiment, another non-contact heart rate detection method is further provided in an embodiment of the present invention, and is implemented on the basis of the above method embodiment, where the method mainly describes a specific process of determining a region of interest based on a target image (implemented by the following steps S304-S306), and as shown in fig. 3, the method includes the following steps:
step S302, a target image containing a target object is collected through a preset camera.
And step S304, carrying out face recognition on the target image to obtain a face image.
In specific implementation, a preset classifier can be adopted to perform face recognition on the target image to obtain a recognized face image. The preset classifier may be a nearest distance classifier, a linear classifier, a Haar cascade classifier, or the like.
Step S306, determining a region of interest from the face image.
With the IPPG technique, theoretically, it is possible to deal with changes in the absorbance of light due to changes in the blood volume of blood vessels at any part, and therefore, normally, non-contact heart rate detection by IPPG is possible also at other parts of the human body. However, since the human face has abundant subcutaneous capillaries, the blood volume of the blood vessels changes greatly along with the pulse, the absorption change of illumination is most obvious, and the blood vessels are most extensive in actual measurement scenes, so that the physiological signals of the face are usually collected to extract the blood volume pulse waves. Namely, the face region in the face image is determined as the region of interest.
In step S308, the target offset within the region of interest is determined.
In the related art, the movement of the target object may interfere with the non-contact heart rate detection, and therefore how to solve the influence of the movement interference on the non-contact heart rate detection is one of the important points that needs to be studied at the present stage. The mainstream solution is to introduce and improve various image tracking algorithms to correct the region of interest, and avoid the interference of redundant signals and noise as much as possible, so as to achieve the purpose of resisting the motion interference. In consideration of the real-time performance and the operation efficiency of the system, the invention introduces a sparse optical flow tracking algorithm (called KLT algorithm for short) and corrects the region of interest on the basis of the KLT algorithm. Of course, the present invention can also employ the current mainstream solution to modify the region of interest.
And S310, correcting the region of interest according to the target offset to obtain a corrected region of interest, and taking the corrected region of interest as a final region of interest.
In a specific implementation, the determination method of the target offset in the region of interest may be: performing target tracking on the target image through a sparse optical flow tracking algorithm; and calculating the target offset in the region of interest according to the target tracking result.
In the tracking of moving objects, sparse optical flow tracking is a classic object tracking algorithm, can draw the tracking track and the running direction of a moving object, is a simple, real-time and efficient tracking algorithm, is proposed by two authors of Bruce d. The KLT algorithm works with three assumptions: constant brightness, short-range movement, and spatial consistency. Based on the characteristics of the KLT algorithm, the KLT algorithm can be applied in a non-contact heart rate detection scenario: heart rate detection with slight movement with consistent space and constant brightness.
Specifically, the KLT algorithm principle is as follows:
and the brightness value of any pixel point p (x, y) in the image region corresponding to the interested region is kept unchanged after the value at t-1 is moved (u, v) at the time t. Luminance constancy equation, luminance I (x, y, t-1) at time t-1 equals luminance I (x + u (x, y), y + v (x, y), t) at time t:
I(x,y,t-1)＝I(x+u(x,y),y+v(x,y),t)
assuming that the same movement distance (u, v) is maintained for all the pixels around pixel point p (x, y), and assuming that the window size is 3x3, the following equation holds for the pixels within 9 windows based on the luminance constancy and the spatial consistency (the same movement (u, v)):
pi＝(xi，yi)
wherein, ItRepresenting the brightness value of a certain pixel point at the time t; p is a radical ofiThe ith pixel point, x, representing the pixel spaceiAbscissa value, y, representing the ith pixeliA longitudinal coordinate value representing the ith pixel point; p is a radical ofiAnd xiAnd yiPresenting a mapping relation; u v represent the vertical and horizontal components of the displacement of the pixel p (x, y) at a time, respectively;
representing the gradient in the x-and y-directions.
Through the constraint equation, the Hessian matrix of the window can be obtained:
order to
Wherein M is Hessian matrix of the window, A is auxiliary matrix defined in least square method derivation process, IxRepresenting time xBrightness value of individual pixel point, IyRepresenting the brightness value of a certain pixel point at the y moment.
Eigenvalue λ of the Hessian matrix described above1And λ2The spatial consistency can be guaranteed to meet the conditions only when the KLT is large, so that the KLT needs to start from the corner detection.
In the non-contact heart rate detection based on IPPG, the KLT algorithm is used for correcting the region of interest, so that the motion interference resistance of the system can be improved.
Step S312, performing color space conversion on the image area corresponding to the final region of interest to obtain an image signal in the designated color space.
Step S314, extracting signals of the image signals in the specified color space to obtain blood volume pulse signals; wherein the blood volume pulse signal comprises time-frequency information of the signal.
And step S316, carrying out video analysis on the blood volume pulse signal to obtain the heart rate of the target object.
According to the non-contact heart rate detection method, real-time video acquisition can be performed by using a common household camera and a host, and non-contact real-time heart rate detection based on IPPG is realized by using python. And in the method, a KLT algorithm is added to improve the anti-motion interference performance, target tracking is performed through the KLT algorithm, the offset of the region of interest is calculated, and the region of interest is adjusted. Meanwhile, the novel medical instrument has the characteristics of no wound, no need of wearing related sensors, convenience and quickness in use and the like, and has a great popularization prospect in daily application.
Corresponding to the above method embodiment, another non-contact heart rate detection method is further provided in the embodiment of the present invention, where the method is implemented on the basis of the above method embodiment, and the method mainly describes a specific process of performing color space conversion on an image region corresponding to a region of interest to obtain an image signal in a specified color space (implemented by the following steps S406 to S408), a specific process of performing signal extraction on the image signal in the specified color space to obtain a blood volume pulse signal (implemented by the following steps S410 to S412), and a specific process of performing video analysis on the blood volume pulse signal to obtain a heart rate of a target object (implemented by the following steps S414 to S416); as shown in fig. 4, the method includes the steps of:
step S402, collecting a target image containing a target object through a preset camera.
In specific implementation, in order to ensure that the resolution of the image acquired by the camera can be high enough to extract a clear BVP signal and avoid the too high resolution of the image from causing too long calculation time consumption, the resolution of the camera can be set to 1280 × 720 pixels.
In the video image acquisition stage, can set up camera sampling frame rate to 20fps, through reducing the sampling rate and improving the sampling time interval, make real-time heart rate parameter calculation have sufficient time, prevent the phenomenon of skipping frame, guarantee system steady operation.
Step S404, determining a region of interest based on the target image.
In the specific implementation, the image preprocessing is carried out on the frame data, a cascade classifier with Haar characteristics can be used for carrying out face recognition and positioning an interested area in a target image, the target offset in the interested area is calculated through a KLT algorithm, and the interested area is corrected in the next cycle detection, so that the motion interference resistance of the system is improved.
Step S406, extracting RGB channel feature values of the image region corresponding to the region of interest to obtain an image signal in an RGB color space.
The camera sensor generally has R, G, B three color channels, so that the RGB channel feature values of the image can be directly extracted from the image area corresponding to the region of interest to obtain the image signal in the RGB color space.
Step S408, performing color space conversion on the image signal in the RGB color space to obtain an image signal in the Lab color space.
In a specific implementation, since it is difficult to directly convert the RGB color space into the Lab color space, the step S408 can be implemented by the following steps 10-11:
and 11, converting the image signal in the XYZ color space into a Lab color space to obtain the image signal in the Lab color space.
In practical applications, the XYZ color space and the RGB color space have the following mapping relationship, that is, the image signal in the RGB color space can be converted into the XYZ color space through the following mapping relationship:
in practical applications, the Lab color space and the XYZ color space have the following mapping relationship, that is, the image signal in the XYZ color space can be converted into the Lab color space by the following mapping relationship:
l in the above formula*、a*、b*Respectively representing the values of three channels in the Lab color space; x, Y, Z are values converted to channels in XYZ space via RGB space, and Xn, Yn, Zn are usually the default values 95.047, 100.0，108.883。
And step S410, filtering the image signal in the Lab color space to obtain a filtered signal.
In a specific implementation, the image signal corresponding to the a channel in the Lab color space may be subjected to band-pass filtering to obtain a filtered signal, which may also be referred to as an initial BVP signal.
And step S412, performing CMOR4-2 wavelet transform on the filtering signal to obtain a blood volume pulse signal.
And step S414, carrying out time-frequency analysis on the blood volume pulse signal to obtain an initial heart rate value.
The CMOR4-2 wavelet is utilized to draw the energy spectrogram of the blood volume pulse signal, simultaneously analyze the time domain component and the frequency domain component of the blood volume pulse, and extract the heart rate parameter which has high precision and correctly reflects the change information.
And step S416, verifying the initial heart rate value to obtain the heart rate of the target object.
During specific implementation, the initial heart rate value needs to be verified, and after verification, abnormal data in the initial heart rate value is removed to obtain a final heart rate value, which is the heart rate of the target object.
In order to test the performance of the heart rate detection method provided by the invention, contents such as an experimental paradigm, a quantitative evaluation index, an experimental result and the like of the non-contact heart rate detection based on the IPPG under the Lab color space are introduced below.
The experiment used a common home host (CPU: AMD Ryzen 52600 GPU: AMD Raedeon RX 570) and an entry level camera (1280 × 72020 fps). Setting the resolution of the camera to 1080 × 720, setting the frame rate to 20 frames/s, keeping the tested person (equivalent to the target object) facing the camera for 15 seconds under the relative static condition, obtaining blood volume pulse signals through FFT and CMOR4-2 respectively by the measuring equipment and environment as shown in figure 2, performing contrast by using a yuwell YX303 pulse oximeter, and calculating the heart rate by energy spectrum analysis.
In order to perform qualitative and quantitative judgment and analysis on the experimental results, the experiment uses the following evaluation indexes for evaluation: the heart rate estimation Error HRer, the Mean Error of the heart rate measurement (Mer), the Standard Deviation of the heart rate estimation Error (SDer), the Root Mean square Error of the heart rate estimation (RMSE), the Mean percentage Error Mep and the pearson correlation coefficient p.
(1) HRer represents the difference between the system detection value, HRme, and the true heart rate value, HRre, expressed as:
HRer＝HRme-HRre
(2) average estimation error M of heart rate measurementserMeans the average value of the heart rate measurement errors, the smaller the average error is, the more accurate the heart rate measurement is, and Mer is used for expressing:
wherein N represents the number of measurements of a valid experiment,
representing the error between the ith heart rate measurement and the true heart rate.
(3) The mean absolute percentage error of heart rate is expressed using Mep:
(4) standard deviation SD of heart rate estimation errorerExpressed as follows:
wherein N represents the number of measurements of a valid experiment,
representing the error between the ith heart rate measurement and the true heart rate, MerEstimating heart rateAverage error of the meter. The standard deviation of heart rate estimation reflects the discrete degree of heart rate estimation error, and the more stable the non-contact heart rate detection system is, the SDerThe smaller the value of (c).
(5) The root mean square error of the heart rate estimate is the sum of the squares of the errors of the heart rate estimate from the true value divided by the amount of data samples and taken as the square root, expressed as RMSE:
wherein,
representing the error between the ith heart rate measurement and the true heart rate, the value of RMSE can reflect the accuracy of the heart rate estimate, being the degree of deviation between the heart rate estimate and the true heart rate value.
(6) The Pearson correlation coefficient p can represent a linear correlation between the heart rate estimate and the true heart rate value, and is expressed as follows:
wherein Cov represents covariance, Std represents standard deviation, Hrme and Hrre represent estimated value and true value of the ith test heart rate,
and
means for heart rate detection and actual heart rate for all valid experiments are indicated. The larger the absolute value of ρ is, the stronger the correlation between the heart rate estimate and the true heart rate is, and the higher the accuracy of the heart rate estimate is.
The experiment adopts the same tested subject in the same environment, the test is carried out under three scenes of rest, slight motion and violent motion respectively after the KLT algorithm is not used and the KLT algorithm is added, each scene is measured for ten times and is compared with a medical oximeter, and the error, the standard deviation, the root mean square error and the Pearson correlation coefficient under each scene are calculated. Table 1 shows the comparison data before and after the chase algorithm was added.
TABLE 1
It is evident by experiment that the average absolute percentage error of heart rate at light exercise after the KLT algorithm was added was reduced from 18.30% to 7.3% before addition and the pearson correlation coefficient was increased from 0.07 to 0.82.
Time-frequency analysis was performed using CMOR 4-2. Because the human body limiting heart rate is 220 times/minute and the frequency of the corresponding BVP signal is 3.67Hz, before time-frequency analysis of the BVP signal, the acquired signal is low-pass filtered, noise above 5Hz is filtered, then time-frequency analysis is carried out on the BVP signal, more accurate main frequency information is obtained and a corresponding heart rate value is calculated, wherein the 15-second BVP signal is shown in FIG. 5, and the time-frequency analysis of the 15-second BVP signal is shown in FIG. 6.
Fig. 7 is a diagram illustrating a 15-second BVP signal, fig. 8 is a spectrogram of spectral analysis using FFT, and fig. 9 is a spectrogram of time-frequency analysis using CMOR 4-2. When the heart rate changes along with time, although the traditional FFT method can be used for analyzing the main frequency signal in a small time window, when the BVP signal changes along with time or interference of other obvious frequencies occurs due to heart rate fluctuation, the signal calculation generates obvious fluctuation, and the CMOR4-2 is used for time-frequency analysis, so that the information of the change of the frequency of the BVP signal along with time corresponding to the heart rate can be embodied, and the heart rate can be calculated by excluding the interference signal according to the change trend.
During specific implementation, the experiment is divided into ten groups, 1 person in each group is subjected to an accuracy experiment, 10 measurement contrasts are carried out on each tested person by using a non-contact heart rate detection method and a medical oximeter, the error value, the standard deviation, the root mean square error and the Pearson correlation coefficient of each group are calculated, and the experimental data are shown in table 2:
TABLE 2
In ten groups of effective comparison measurement, the maximum average error is 2.96 times/minute, the minimum average error is 1.39 times/minute, the average percentage error is between 1.6% and 3.5%, and the accuracy is higher; the root mean square error is between 1.53 and 3.05, the smaller the root mean square error is, the smaller the discrete degree of the experimental result and the true value is, the higher the precision is, in the non-contact heart rate detection, the lower the root mean square error is, the higher the precision of the experimental result can be considered, and the higher the precision of the experimental result can be considered, so the higher the precision of the experimental result can be considered; the average correlation coefficient of the measurement result compared with the medical oximeter is 0.9876, and the heart rate estimation correlation scatter diagram is shown in fig. 10, it can be found from statistics that when the pearson correlation coefficient is between 0.8 and 1.0, the two parameters are considered to be extremely correlated, so that the correlation between the estimated heart rate and the real heart rate is extremely strong, and the heart rate detection is effective.
The heart rate detection method based on python realizes non-contact heart rate detection of heart rate, meets the application in practical scenes, can detect the heart rate in a natural light state, and has the characteristics of convenience and quickness in operation, low equipment requirement, small environmental restriction and the like. There are also some problems with contactless heart rate detection and directions that can be improved. The anti-motion interference is one of the difficulties in non-contact heart rate detection based on IPPG, and after a KLT algorithm is introduced to correct an ROI (region of interest), the heart rate measurement accuracy and the Pearson correlation coefficient under slight motion are obviously improved.
According to the non-contact heart rate detection method, in order to accurately detect the real-time heart rate parameters of the target object in a comfortable and non-contact environment, a real-time heart rate parameter extraction system based on a common camera is designed. The system stores the graph of the region of interest in a target image obtained through camera shooting as an RGB channel characteristic value, converts an RGB color space to an Lab color space through color space conversion, filters a channel information under the Lab color space to obtain an initial BVP signal, performs time-frequency analysis on the BVP signal after CMOR4-2 wavelet transformation, and calculates and obtains a heart rate parameter. Different from methods such as FFT, STFT and IBls, the method extracts the BVP signal by using CMOR4-2, and the BVP signal based on CMOR4-2 transformation can reflect time-frequency information and can prevent abnormal beating of heart rate parameters under the condition of obvious heart rate change.
In addition, experimental results show that the method has the accuracy of more than 96.5% in a relatively static environment, the system is applicable to common scenes, and the heart rate can be detected in a common household environment. Meanwhile, the novel medical instrument has the characteristics of no wound, no need of wearing related sensors, convenience and quickness in use and the like, and has a great popularization prospect in daily application.
For the above method embodiment, an embodiment of the present invention further provides a non-contact heart rate detecting apparatus, as shown in fig. 11, the apparatus includes:
the image collecting module 80 is configured to collect a target image including a target object through a preset camera.
A region of interest determining module 81 for determining a region of interest based on the target image.
And the color space conversion module 82 is configured to perform color space conversion on the image area corresponding to the region of interest to obtain an image signal in a specified color space.
The signal extraction module 83 is configured to perform signal extraction on the image signal in the designated color space to obtain a blood volume pulse signal; wherein, the blood volume pulse signal contains the time frequency information of the signal.
A heart rate determination module 84 for determining a heart rate of the target subject based on the blood volume pulse signal.
The non-contact heart rate detection device firstly collects a target image containing a target object through a preset camera; then, determining an interested region based on the target image so as to perform color space conversion on an image region corresponding to the interested region to obtain an image signal under an appointed color space; then, carrying out signal extraction on the image signal in the appointed color space to obtain a blood volume pulse signal, wherein the blood volume pulse signal comprises time-frequency information of the signal; the heart rate of the target subject is then determined based on the blood volume pulse signal. According to the method, the camera is used for collecting the image of the target object in real time, time-frequency analysis processing is carried out on the image, and a heart rate value is obtained, so that non-contact heart rate detection is realized, the application in an actual scene is met, the heart rate can be detected in a natural light state, and the method has the characteristics of convenience in operation, low equipment requirement, small environmental restriction, strong instantaneity, high accuracy and the like.
Further, the region of interest determining module 81 is further configured to: carrying out face recognition on the target image to obtain a face image; and determining the region of interest from the face image.
In a specific implementation, the apparatus further includes a correction module, configured to determine a target offset within the region of interest after determining the region of interest based on the target image; and correcting the region of interest according to the target offset to obtain a corrected region of interest, and taking the corrected region of interest as a final region of interest.
Further, the modification module is further configured to: performing target tracking on the target image through a sparse optical flow tracking algorithm; and calculating the target offset in the region of interest according to the target tracking result.
In some embodiments, the specified color space is a Lab color space; the color space converting module 82 is further configured to: extracting RGB channel characteristic values of an image area corresponding to the region of interest to obtain image signals in RGB color space; and carrying out color space conversion on the image signals in the RGB color space to obtain the image signals in the Lab color space.
Further, the color space converting module 82 is further configured to: converting the image signal in the RGB color space into an XYZ color space to obtain an image signal in the XYZ color space; and converting the image signal in the XYZ color space into the Lab color space to obtain the image signal in the Lab color space.
In a specific implementation, the signal extracting module 83 is configured to: filtering the image signal in the designated color space to obtain a filtered signal; and performing CMOR4-2 wavelet transform on the filtered signal to obtain a blood volume pulse signal.
Specifically, when the designated color space is an LAB color space, the signal extraction module 83 is further configured to: and performing band-pass filtering on the image signal corresponding to the channel a in the Lab color space to obtain a filtering signal.
Further, the heart rate determining module 84 is configured to: performing time-frequency analysis on the blood volume pulse signal to obtain an initial heart rate value; and carrying out verification processing on the initial heart rate value to obtain the heart rate of the target object.
The implementation principle and the generated technical effect of the non-contact heart rate detection device provided by the embodiment of the invention are the same as those of the method embodiment, and for brief description, no part of the embodiment of the device is mentioned, and reference may be made to the corresponding content in the method embodiment.
An embodiment of the present invention further provides an electronic device, as shown in fig. 12, where the electronic device includes a processor 101 and a memory 100, where the memory 100 stores machine executable instructions capable of being executed by the processor 101, and the processor 101 executes the machine executable instructions to implement the above-mentioned non-contact heart rate detection method.
Further, the electronic device shown in fig. 12 further includes a bus 102 and a communication interface 103, and the processor 101, the communication interface 103, and the memory 100 are connected through the bus 102.
The Memory 100 may include a high-speed Random Access Memory (RAM) and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used. The bus 102 may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 12, but that does not indicate only one bus or one type of bus.
The processor 101 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 101. The Processor 101 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 100, and the processor 101 reads the information in the memory 100, and completes the steps of the method of the foregoing embodiment in combination with the hardware thereof.
The functions may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method of non-contact heart rate detection, the method comprising:
acquiring a target image containing a target object through a preset camera;
determining a region of interest based on the target image;
performing color space conversion on the image area corresponding to the region of interest to obtain an image signal in a specified color space;
performing signal extraction on the image signal in the appointed color space to obtain a blood volume pulse signal; wherein, the blood volume pulse signal comprises time-frequency information of the signal;
determining a heart rate of the target subject based on the blood volume pulse signal.
2. The method of claim 1, wherein the step of determining a region of interest based on the target image comprises:
carrying out face recognition on the target image to obtain a face image;
and determining the region of interest from the face image.
3. The method according to claim 1 or 2, wherein after the step of determining a region of interest based on the target image, the method further comprises:
determining a target offset within the region of interest;
and correcting the region of interest according to the target offset to obtain a corrected region of interest, and taking the corrected region of interest as a final region of interest.
4. The method of claim 3, wherein the step of determining the target offset within the region of interest comprises:
performing target tracking on the target image through a sparse optical flow tracking algorithm;
and calculating the target offset in the region of interest according to the target tracking result.
5. The method of claim 1, wherein the specified color space is a Lab color space; the step of performing color space conversion on the image area corresponding to the region of interest to obtain an image signal in a specified color space includes:
extracting RGB channel characteristic values of an image area corresponding to the region of interest to obtain image signals in RGB color space;
and carrying out color space conversion on the image signal in the RGB color space to obtain the image signal in the Lab color space.
6. The method as claimed in claim 5, wherein the step of performing color space conversion on the image signal in the RGB color space to obtain the image signal in the Lab color space comprises:
converting the image signal in the RGB color space into an XYZ color space to obtain an image signal in the XYZ color space;
and converting the image signal in the XYZ color space into the Lab color space to obtain the image signal in the Lab color space.
7. The method according to claim 1, wherein the step of extracting the image signal in the designated color space to obtain the blood volume pulse signal comprises:
filtering the image signal in the specified color space to obtain a filtered signal;
and performing CMOR4-2 wavelet transform on the filtered signal to obtain the blood volume pulse signal.
8. The method of claim 7, wherein the specified color space is a Lab color space; the step of performing filtering processing on the image signal in the designated color space to obtain a filtered signal includes:
and performing band-pass filtering on the image signal corresponding to the channel a in the Lab color space to obtain the filtering signal.
9. The method of claim 1, wherein the step of determining the heart rate of the target subject based on the blood volume pulse signal comprises:
performing time-frequency analysis on the blood volume pulse signal to obtain an initial heart rate value;
and carrying out verification processing on the initial heart rate value to obtain the heart rate of the target object.
10. A non-contact heart rate detection device, the device comprising:
the image acquisition module is used for acquiring a target image containing a target object through a preset camera;
a region-of-interest determination module for determining a region of interest based on the target image;
the color space conversion module is used for carrying out color space conversion on the image area corresponding to the interested area to obtain an image signal under a specified color space;
the signal extraction module is used for carrying out signal extraction on the image signal in the specified color space to obtain a blood volume pulse signal; wherein, the blood volume pulse signal comprises time-frequency information of the signal;
a heart rate determination module to determine a heart rate of the target subject based on the blood volume pulse signal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210222186.XA CN114569101A (en) | 2022-03-09 | 2022-03-09 | Non-contact heart rate detection method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210222186.XA CN114569101A (en) | 2022-03-09 | 2022-03-09 | Non-contact heart rate detection method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114569101A true CN114569101A (en) | 2022-06-03 |
Family
ID=81778813
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210222186.XA Pending CN114569101A (en) | 2022-03-09 | 2022-03-09 | Non-contact heart rate detection method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114569101A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116559818A (en) * | 2023-07-04 | 2023-08-08 | 南昌大学 | Human body posture recognition method, system, computer and readable storage medium |
| CN116999044A (en) * | 2023-09-07 | 2023-11-07 | 南京云思创智信息科技有限公司 | Real-time motion full-connection bidirectional consistent optical flow field heart rate signal extraction method |
| CN118587225A (en) * | 2024-08-07 | 2024-09-03 | 沈阳康泰电子科技股份有限公司 | Non-contact physiological parameter monitoring method and system |
-
2022
- 2022-03-09 CN CN202210222186.XA patent/CN114569101A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116559818A (en) * | 2023-07-04 | 2023-08-08 | 南昌大学 | Human body posture recognition method, system, computer and readable storage medium |
| CN116559818B (en) * | 2023-07-04 | 2023-09-12 | 南昌大学 | Human body posture recognition method, system, computer and readable storage medium |
| CN116999044A (en) * | 2023-09-07 | 2023-11-07 | 南京云思创智信息科技有限公司 | Real-time motion full-connection bidirectional consistent optical flow field heart rate signal extraction method |
| CN116999044B (en) * | 2023-09-07 | 2024-04-16 | 南京云思创智信息科技有限公司 | Real-time motion full-connection bidirectional consistent optical flow field heart rate signal extraction method |
| CN118587225A (en) * | 2024-08-07 | 2024-09-03 | 沈阳康泰电子科技股份有限公司 | Non-contact physiological parameter monitoring method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110269600B (en) | Non-contact video heart rate detection method based on multivariate empirical mode decomposition and combined blind source separation | |
| CN109977858B (en) | Heart rate detection method and device based on image analysis | |
| Feng et al. | Dynamic ROI based on K-means for remote photoplethysmography | |
| CN114569101A (en) | Non-contact heart rate detection method and device and electronic equipment | |
| Feng et al. | Motion artifacts suppression for remote imaging photoplethysmography | |
| US10016166B2 (en) | Contactless detection method with noise limination for information of physiological and physical activities | |
| CN111243739A (en) | Anti-interference physiological parameter telemetering method and system | |
| JP6371410B2 (en) | Respiratory state estimation apparatus, portable device, wearable device, program, medium, and respiratory state estimation method | |
| CN111938622B (en) | Heart rate detection method, device and system and readable storage medium | |
| US9662023B2 (en) | Robust heart rate estimation | |
| CN112233813A (en) | Non-contact non-invasive heart rate and respiration measurement method and system based on PPG | |
| CN113693573B (en) | Video-based non-contact multi-physiological-parameter monitoring system and method | |
| CN110236511A (en) | A kind of noninvasive method for measuring heart rate based on video | |
| CN107334469A (en) | Non-contact more people's method for measuring heart rate and device based on SVMs | |
| WO2021111436A1 (en) | System and method for physiological measurements from optical data | |
| Li et al. | An improvement for video-based heart rate variability measurement | |
| CN113361526B (en) | Non-contact respiration rate monitoring method fusing shoulder and chest area information | |
| CN113591769B (en) | Non-contact heart rate detection method based on photoplethysmography | |
| CN112826483A (en) | Fingertip video-based heart rate detection method, system and device | |
| CN117136027A (en) | Method and system for extracting heart rate from RGB image | |
| KR20170004804A (en) | A method for estimating respiratory and heart rate using dual cameras on a smart phone | |
| WO2023165482A1 (en) | Method and apparatus for heart rate detection | |
| Hu et al. | Study on Real-Time Heart Rate Detection Based on Multi-People. | |
| Wang et al. | KLT algorithm for non-contact heart rate detection based on image photoplethysmography | |
| CN104688199A (en) | Non-contact type pulse measurement method based on skin pigment concentration difference |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |