CN115294027A - Heart rate detection method and device, electronic equipment and storage medium - Google Patents
Heart rate detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115294027A CN115294027A CN202210804138.1A CN202210804138A CN115294027A CN 115294027 A CN115294027 A CN 115294027A CN 202210804138 A CN202210804138 A CN 202210804138A CN 115294027 A CN115294027 A CN 115294027A
- Authority
- CN
- China
- Prior art keywords
- heart rate
- corner
- determining
- speckle images
- speckle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 82
- 238000006073 displacement reaction Methods 0.000 claims abstract description 57
- 238000000034 method Methods 0.000 claims abstract description 44
- 230000003287 optical effect Effects 0.000 claims abstract description 42
- 238000001914 filtration Methods 0.000 claims abstract description 29
- 238000012163 sequencing technique Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 claims description 5
- 230000035515 penetration Effects 0.000 claims description 3
- 230000036541 health Effects 0.000 abstract description 3
- 238000012544 monitoring process Methods 0.000 abstract description 2
- 210000003491 skin Anatomy 0.000 description 13
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 239000002245 particle Substances 0.000 description 5
- 208000024172 Cardiovascular disease Diseases 0.000 description 3
- 210000002615 epidermis Anatomy 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 210000004369 blood Anatomy 0.000 description 2
- 239000008280 blood Substances 0.000 description 2
- 210000000748 cardiovascular system Anatomy 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 210000002808 connective tissue Anatomy 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000001678 irradiating effect Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 239000006096 absorbing agent Substances 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 238000010009 beating Methods 0.000 description 1
- 230000004071 biological effect Effects 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 210000004207 dermis Anatomy 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000001121 heart beat frequency Effects 0.000 description 1
- 238000009532 heart rate measurement Methods 0.000 description 1
- 108010036302 hemoglobin AS Proteins 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000284 resting effect Effects 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0082—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence adapted for particular medical purposes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/024—Detecting, measuring or recording pulse rate or heart rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30088—Skin; Dermal
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Pathology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Heart & Thoracic Surgery (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Cardiology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Physiology (AREA)
- Multimedia (AREA)
- Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
Abstract
The invention relates to the technical field of human health monitoring, and provides a heart rate detection method, a heart rate detection device, electronic equipment and a storage medium, wherein the method comprises the following steps: collecting speckle images of a human body; performing pixel reconstruction on the speckle images according to an image super-resolution reconstruction algorithm, and filtering the reconstructed speckle images; tracking the angular points in the speckle images after filtering according to angular point detection and a pyramid optical flow method, and determining angular point displacement according to a tracking result; and determining the heart rate of the human body according to the angular point displacement. According to the embodiment of the invention, the angular points on the speckle images are selected by adopting angular point detection as target tracking points for optical flow tracking, so that the angular point displacement in the speckle images of adjacent frames is quickly obtained, and then the heart rate of the detected human body can be quickly and effectively obtained, thereby the accuracy and the efficiency of heart rate detection are realized.
Description
Technical Field
The invention relates to the technical field of human health monitoring, in particular to a heart rate detection method and device, electronic equipment and a storage medium.
Background
At present, with the acceleration of population aging, the incidence of cardiovascular diseases of the nation is promoted year by year, and the cardiovascular diseases are one of the chief causes of harming human health, and are particularly common to people over 50 years old. If the relevant parameters of the cardiovascular system can be accurately measured and judged, the incidence rate of cardiovascular diseases can be reduced to a great extent, so that the accurate prediction of the heart rate and pulse wave information of a human body is particularly important.
The heartbeat per minute of the human body in a normal resting state is called as a heart rate, the heart rate of the human body is different due to individual differences, and the pulse wave signals contain important physiological information of a cardiovascular system and can effectively reflect the beating rule of the heart, so the heart rate can be estimated and measured by adopting the pulse wave signals under most conditions, but the accuracy of estimating and measuring the heart rate by adopting the pulse wave signals is lower at present.
Disclosure of Invention
The invention provides a heart rate detection method, a heart rate detection device, electronic equipment and a storage medium, which are used for solving the problem of low heart rate detection accuracy.
The invention provides a heart rate detection method, which comprises the following steps:
collecting speckle images of a human body;
performing pixel reconstruction on the speckle images according to an image super-resolution reconstruction algorithm, and filtering the reconstructed speckle images;
tracking the angular points in the speckle images after filtering according to angular point detection and a pyramid optical flow method, and determining angular point displacement according to a tracking result;
and determining the heart rate of the human body according to the angular point displacement.
In one embodiment, the tracking the corner points in the filtered speckle images according to the corner point detection and pyramid optical flow method includes:
determining a corner to be tracked in the speckle image after filtering according to the corner detection;
and tracking the corner points to be tracked according to the pyramid optical flow method.
In one embodiment, the determining a corner to be tracked in the filtered speckle image according to the corner detection includes:
determining the minimum characteristic value of the corner points, and deleting the corner points with characteristic values smaller than the minimum characteristic value;
and sequencing the corner points with the characteristic values larger than the minimum characteristic value, and determining the corner point to be tracked according to a sequencing result.
In one embodiment, the tracking the corner points to be tracked according to the pyramid optical flow method includes:
determining the state of each corner point in the current frame according to the pyramid optical flow method, the corner point to be tracked of the previous frame and the current frame;
and tracking the angular points to be tracked according to the state of each angular point in the current frame.
In one embodiment, the determining the heart rate of the human body according to the corner point displacement includes:
determining a corner displacement image according to the corner displacement;
and determining the heart rate of the human body according to the angular point displacement graph and Fourier transform.
In one embodiment, the collecting speckle images of a human body includes:
determining the penetrability of light sources with different wavelengths to human skin;
determining the effect of light sources of different wavelengths on heart rate from the penetration;
determining a target light source according to the influence of light sources with different wavelengths on the heart rate, and acquiring the speckle image of the human body according to the target light source.
In one embodiment, the pixel reconstruction of the speckle image according to an image super-resolution reconstruction algorithm comprises:
inserting pixel points into the speckle images according to the image super-resolution reconstruction algorithm;
and determining the brightness value of the pixel point, and filling the brightness of the pixel point according to the brightness value.
The invention provides a heart rate detection device, comprising:
the acquisition module is used for acquiring speckle images of a human body;
the reconstruction module is used for carrying out pixel reconstruction on the speckle images according to an image super-resolution reconstruction algorithm and filtering the reconstructed speckle images;
the tracking module is used for tracking the angular points in the speckle images after filtering according to angular point detection and a pyramid optical flow method and determining angular point displacement according to a tracking result;
and the determining module is used for determining the heart rate of the human body according to the angular point displacement.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the heart rate detection method.
The invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a heart rate detection method as described in any one of the above.
The heart rate detection method, the heart rate detection device, the electronic equipment and the storage medium provided by the invention collect speckle images of a human body; performing pixel reconstruction on the speckle images according to an image super-resolution reconstruction algorithm, and filtering the reconstructed speckle images; tracking the angular points in the speckle images after filtering according to angular point detection and a pyramid optical flow method, and determining angular point displacement according to a tracking result; and determining the heart rate of the human body according to the angular point displacement. The embodiment of the invention adopts the angular point detection to select the angular point on the speckle image as the target tracking point for optical flow tracking, quickly obtains the angular point displacement in the speckle images of the adjacent frames, and can quickly and effectively acquire the heart rate of the detected human body, thereby realizing the accuracy and the efficiency of heart rate detection.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a heart rate detection method provided by the present invention;
FIG. 2 is a second flowchart of a heart rate detection method according to the present invention;
FIG. 3 is a third schematic flow chart of a heart rate detection method provided by the present invention;
FIG. 4 is a schematic diagram of a time domain waveform of the corner point displacement provided by the present invention;
FIG. 5 is a schematic frequency domain waveform of the corner displacement provided by the present invention;
FIG. 6 is a schematic structural diagram of a heart rate detecting device provided by the present invention;
fig. 7 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The heart rate detection method, apparatus, electronic device and storage medium of the present invention are described below with reference to fig. 1-7.
Specifically, the present invention provides a heart rate detection method, and referring to fig. 1, fig. 1 is one of the flow diagrams of the heart rate detection method provided by the present invention.
It should be noted that although a logical order is shown in the flow chart, under certain data, the steps shown or described may be performed in an order different than that shown or described herein.
The heart rate detection method provided by the embodiment of the invention comprises the following steps:
s10, collecting speckle images of a human body;
it should be noted that when laser light (or coherent light) is applied to the surface of a living body, the light scattered from the surface exhibits the form of speckle on the receiving plane, and this phenomenon is called dynamic speckle, and is also called biological speckle because it is generated by biological activity. Research proves that the intensity and the shape of the biological speckle are changed constantly along with time, the speckle is continuously moved, deformed, disappeared and reappeared to have a certain correlation with the biological characteristics of the detected object, and based on the correlation, the heart rate (heartbeat) detection can be realized.
Specifically, a laser is used to irradiate a light source with a set wavelength to the surface of human skin, and then a CCD (Charge Coupled Devices) camera is used to record dynamic biological speckle video formed in space, i.e. speckle images of the human body.
Because the heart rate detection is interfered by ambient light, the distinguishing and the extraction of heart rate signals are seriously influenced, and based on the method, the embodiment of the invention adopts light sources with different wavelengths to irradiate the skin of an experimental area to obtain speckle images, and then adopts the light source with the best effect to carry out data acquisition experiments. Specifically, the penetrability of light sources with different wavelengths on the skin of a human body is determined, then the influence of the light sources with different wavelengths on the heart rate is determined according to the penetrability, finally, a target light source is determined according to the influence of the light sources with different wavelengths on the heart rate, and the speckle image of the human body is collected according to the target light source. It should be noted that the human epidermis typically has a depth in the range of 0 to 80 μm, because the thickness of the skin epidermis is body-part dependent, and the human epidermis is transparent and functional to all wavelengths of light, but absorbs mainly blue light (< 490 nm). Human dermis (depth range 50-200 μm) has hemoglobin as the primary absorber and wavelengths above > 500nm are achievable. In the experimental process, the light path of the blue laser and the experimental light path of the red laser are kept consistent, the same exposure time and frame rate are kept, then a speckle image of 30s is shot, the obtained speckle image is processed, heart rate graphs under the irradiation of the blue laser and the red laser light paths are obtained respectively, the influence of light sources with different wavelengths on the heart rate can be determined by comparing the heart rate graphs, the light source with the minimum influence on the heart rate is determined as a target light source, and the speckle image of a human body is acquired based on the target light source. That is, the influence of the deformation of blood and connective tissue on the measured pulse wave results is verified by comparing the different penetrability of the light sources of different wavelengths into the skin, and the light source of the wavelength best for obtaining the speckle image is selected based on the influence.
The speckle images are obtained by irradiating the skin of the experimental area by adopting the light sources with different wavelengths, the target light source is determined based on the speckle images, and then the target light source is adopted to collect the speckle images of the human body, so that the condition that the discrimination and extraction of heart rate signals are influenced by the interference of ambient light is avoided, and the accuracy of heart rate detection is improved.
S20, performing pixel reconstruction on the speckle images according to an image super-resolution reconstruction algorithm, and filtering the reconstructed speckle images;
it should be noted that the super-resolution reconstruction technique of images refers to restoring a given low-resolution image into a corresponding high-resolution image through a specific algorithm, and aims to overcome or compensate the problems of imaging image blurring, low quality, insignificant region of interest, and the like caused by the limitations of an image acquisition system or an acquisition environment. In brief, the super-resolution reconstruction is to change a small-size image into a large-size image, so that the image is more "clear".
After the speckle images of the human body are collected, in order to improve the resolution of the speckle images, the speckle images need to be subjected to pixel reconstruction processing, and specifically, the speckle images are subjected to pixel reconstruction according to an image super-resolution reconstruction algorithm, wherein the image super-resolution reconstruction algorithm comprises super-resolution reconstruction based on interpolation, model-based super-resolution reconstruction and learning-based super-resolution reconstruction. The embodiment of the invention takes super-resolution reconstruction based on interpolation as an example for analysis and explanation, specifically, pixel points are inserted into speckle images according to an image super-resolution reconstruction algorithm, then the brightness values of the pixel points are determined, and the brightness of the pixel points is filled according to the brightness values, for example, a 2*2 pixel image is amplified by two times to 4*4, and the specific steps are as follows:
1. inserting pixel points;
2. filling the inserted pixel points with brightness values, taking dest (I, J) as an example, the specific steps are as follows:
(1) According to the mapping relation, finding the closest pixel point ori (u, v) of dest (I, J) in the original image origin;
(2) Taking pixel ori (u, v) as a center, radiating 16 points outwards, wherein coordinates of the 16 points are ori (i + u, j + v), and the value range of i, j is-1 or more and i is less than or equal to 2; j is more than or equal to-1 and less than or equal to 2, if the original image does not have the pixel points, filling and completion can be carried out;
(3) These 16 points are operated on according to some relation (similar to convolution) to obtain the brightness value at dest (I, J).
Further, in order to extract the target more accurately, a filter is further required to filter the reconstructed speckle image, and specifically, band-pass filtering is used to filter out interference noise in the reconstructed speckle image, so as to extract the target more accurately.
Step S30, tracking the corner points in the speckle images after filtering according to corner point detection and a pyramid optical flow method, and determining corner point displacement according to a tracking result;
it should be noted that, in the embodiment of the present invention, the shit-Tomasi corner detection and the pyramid LK (Lucas-Kanade) optical flow method are adopted to track the corners in the filtered speckle image, and it can be understood that the instantaneous speed of the pixel motion of the spatial moving object on the observation imaging plane is a method for finding the corresponding relationship between the previous frame and the current frame by using the change of the pixel in the image sequence in the time domain and the correlation between the adjacent frames, so as to calculate the motion information of the object between the adjacent frames. For example, the corner to be tracked is determined based on the corner detection, then the corner to be tracked is tracked by adopting a pyramid optical flow method, and the corner displacement is determined according to the tracking result, for example, the position information (such as coordinate value) of the relevant corner between the adjacent frames is determined according to the tracking result, and the corner displacement is determined based on the position information.
In one embodiment, the data set is trained by pre-preparation, when the person to be detected passes through (some infrared probe is shielded), the corners in the speckle images of the adjacent frames are tracked according to the learned data set, and then the corner displacement is determined.
And S40, determining the heart rate of the human body according to the angular point displacement.
It should be noted that, the heart beat of the human body may cause the vibration of the skin surface of the human body, and the speckle particles in the dynamic biological speckle video collected by the CCD camera may also vibrate at the same frequency, which is specifically indicated that the displacement amounts of the speckle particles within the same time are different, that is, the displacement speeds of the speckle particles within the image are different, so that the vibration frequency of the speckle particles corresponds to the heart beat frequency of the human body to be measured. Because the motion condition of speckle particles in the image can be reflected by the filtered data signals, the heart rate and pulse wave signals of the detected human body can be obtained by analyzing the frequency of the filtered data signal curve.
Specifically, an angular point displacement image is determined according to angular point displacement, and then the heart rate of the human body is determined according to the angular point displacement image and Fourier transform. For example, a time domain map of the corner displacement is determined according to the corner displacement, as shown in fig. 4, and a frequency domain map is obtained by performing frequency domain analysis on the time domain map by using Fast Fourier Transform (FFT), as shown in fig. 5, assuming that the sampling frequency of the original signal is f s And the signal length is N, then the minimum frequency interval (spectrum resolution) after the FFT of the original signal is known as:
from this, the resolution of the measured heart rate can be calculated to be 60 · f.
According to the heart rate detection method provided by the embodiment of the invention, speckle images of a human body are collected; performing pixel reconstruction on the speckle images according to an image super-resolution reconstruction algorithm, and filtering the reconstructed speckle images; tracking the angular points in the speckle images after filtering according to angular point detection and a pyramid optical flow method, and determining angular point displacement according to a tracking result; and determining the heart rate of the human body according to the angular point displacement. According to the embodiment of the invention, the angular points on the speckle images are selected by adopting angular point detection as target tracking points for optical flow tracking, so that the angular point displacement in the speckle images of adjacent frames is quickly obtained, and then the heart rate of the detected human body can be quickly and effectively obtained, thereby the accuracy and the efficiency of heart rate detection are realized.
Referring to fig. 2, fig. 2 is a second schematic flow chart of the heart rate detection method provided by the present invention. In an embodiment of the present invention, the tracking the corner points in the filtered speckle image according to corner point detection and a pyramid optical flow method includes:
step S31, determining a corner to be tracked in the speckle image after filtering according to the corner detection;
and S32, tracking the corner points to be tracked according to the pyramid optical flow method.
Specifically, the minimum feature value of the corner points is determined, the corner points with the feature values smaller than the minimum feature value are deleted, then the corner points with the feature values larger than the minimum feature value are sequenced, and the corner points to be tracked are determined according to the sequencing result. For example, in the Shi-Tomasi corner detection method, the image used for detection is a gray scale image, and the Shi-Tomasi corner detection is performed on the gray scale image, and the specific detection steps are as follows:
1. introducing a speckle image, and converting the speckle image into a gray level image;
2. the number of corner points to be found is determined and the minimum feature quality-level acceptable for corner point detection, which is between 0 and 1, represents the minimum quality of a corner point below which a corner point will be rejected.
3. Determining the minimum Euclidean distance of the detected corner points, deleting all corner points with the characteristic values smaller than the defined minimum characteristic value quality-level in a region specified by a certain Euclidean distance, arranging the rest corner points (namely the corner points with the characteristic values larger than the minimum characteristic value) in a descending manner based on the quality-level, and taking the first strongest corner point by a function, so as to form a detected corner point in a region, wherein the corner point is taken as the corner point to be tracked.
Further, according to the pyramid optical flow method, the corner points to be tracked of the previous frame and the current frame, the state of each corner point in the current frame is determined, and then the corner points to be tracked are tracked according to the state of each corner point in the current frame. For example, a first frame is selected, a Shi-Tomasi corner point is detected in an image of the first frame, and then an LK algorithm is used to iteratively track a feature point (i.e., a corner point to be tracked), wherein the feature point (i.e., the corner point to be tracked) of a previous frame of picture and a current frame of picture are continuously transmitted into an optical flow tracking function, and the optical flow tracking function returns a point of the current frame, where the point has a state of 1 or 0, and if a point in the previous frame is found in the current frame, the state of the point is 1, otherwise, the state of the point is 0.
According to the embodiment of the invention, the corner point to be tracked is determined in the filtered speckle image according to the corner point detection, and then the corner point to be tracked is tracked according to the pyramid optical flow method, so that the optical flow method is utilized to track the speckle displacement of the adjacent frames, the efficiency is high, the speed is high, the accuracy is high, and the efficiency and the accuracy of heart rate detection are improved.
Referring to fig. 3, fig. 3 is a third schematic flow chart of a heart rate detection method provided by the present invention. In the embodiment of the invention, according to the rule that the skin surface of a human body deforms along with the heart beat, laser with the best effect wavelength is selected to irradiate the skin surface of the human body, a CCD camera is used for recording a dynamic biological speckle video formed in a space, and heart rate information of the human body is extracted from the obtained speckle video through an image processing algorithm by means of computer software, so that the non-contact measurement of the heart rate of the human body is realized.
Specifically, the embodiment of the invention attaches to laser speckle optical path equipment and acquires speckle images of human skin, wherein the main hardware is as follows:
(1) A computer: the adopted notebook computer has the CPU main frequency of 2.6Hz and the memory of 8G and is used for collecting the dynamic speckle video and processing the dynamic speckle video by using the algorithm based on the Jupyter book webpage;
(2) A laser: lasers with different wavelengths are used for selecting the laser with the best dynamic speckle result;
(3) A CCD camera: for recording dynamic biological speckle video formed in space.
The specific steps of heart rate detection based on the equipment are as follows:
(1) Comparing and verifying the influence of the deformation of blood and connective tissues on the measured pulse wave result through the different penetrability of the light sources with different wavelengths on the skin, and selecting the light source with the best wavelength for obtaining the speckle image;
(2) Irradiating the selected light source on the surface of the skin of a human body, extracting a dynamic speckle video through a CCD (charge coupled device) camera, and then performing pixel reconstruction on the speckle image by using a super-resolution reconstruction technology of the image so as to improve the resolution of the image;
(3) Then, filtering out interference noise by adopting band-pass filtering so as to more accurately extract a target;
(4) And (3) combining Shi-Tomasi corner detection with a pyramid LK (Lucas-Kanade) optical flow method, tracking the most relevant corner points in the adjacent frame speckle images, calculating the displacement of the corner points and converting the displacement into the heart rate.
According to the non-contact human body heart rate measurement mode based on the laser speckles, the angular points on the speckle images are selected by using shi-Tomashi angular point detection to serve as target tracking points for optical flow tracking, angular point displacement in the speckle images of adjacent frames is quickly obtained, then the heart rate of a person can be quickly and effectively obtained, and a quick and stable heart rate obtaining result can be achieved. Based on this, the embodiment of the present invention has the following advantages:
1. performing ultrahigh resolution reconstruction on the acquired speckle images by using an interpolation method, so that corner points of the acquired speckle images are captured more easily;
2. the heart rate is acquired by laser, non-contact measurement is realized, the application range is wider, the efficiency is higher, and the mode is simple and convenient;
3. the optical flow method is utilized to track the speckle displacement of the adjacent frames, so that the efficiency is high, the speed is high, and the accuracy is high;
4. the optical flow method is utilized to track the angular point displacement of the obtained speckle images, and then the heart rate is calculated, so that the experimental device is simple, high in cost performance, high in efficiency and uncomplicated.
Fig. 6 is a schematic structural diagram of a heart rate detection apparatus provided by the present invention, and referring to fig. 6, an embodiment of the present invention provides a heart rate detection apparatus, which includes an acquisition module 601, a reconstruction module 602, a tracking module 603, and a determination module 604.
The acquisition module 601 is used for acquiring speckle images of a human body;
the reconstruction module 602 is configured to perform pixel reconstruction on the speckle images according to an image super-resolution reconstruction algorithm, and filter the reconstructed speckle images;
the tracking module 603 is configured to track the corner in the filtered speckle image according to corner detection and a pyramid optical flow method, and determine corner displacement according to a tracking result;
the determining module 604 is configured to determine a heart rate of the human body according to the corner point displacement.
The heart rate detection device provided by the embodiment of the invention collects speckle images of a human body; performing pixel reconstruction on the speckle images according to an image super-resolution reconstruction algorithm, and filtering the reconstructed speckle images; tracking the corner points in the speckle images after filtering according to corner point detection and a pyramid optical flow method, and determining corner point displacement according to a tracking result; and determining the heart rate of the human body according to the angular point displacement. According to the embodiment of the invention, the angular points on the speckle images are selected by adopting angular point detection as target tracking points for optical flow tracking, so that the angular point displacement in the speckle images of adjacent frames is quickly obtained, and then the heart rate of the detected human body can be quickly and effectively obtained, thereby the accuracy and the efficiency of heart rate detection are realized.
In one embodiment, the tracking module 603 is specifically configured to:
determining a corner to be tracked in the speckle image after filtering according to the corner detection;
and tracking the corner points to be tracked according to the pyramid optical flow method.
In one embodiment, the tracking module 603 is specifically configured to:
determining the minimum characteristic value of the corner points, and deleting the corner points with characteristic values smaller than the minimum characteristic value;
and sequencing the corner points with the characteristic values larger than the minimum characteristic value, and determining the corner point to be tracked according to a sequencing result.
In an embodiment, the tracking module 603 is specifically configured to:
determining the state of each corner point in the current frame according to the pyramid optical flow method, the corner point to be tracked of the previous frame and the current frame;
and tracking the angular points to be tracked according to the state of each angular point in the current frame.
In one embodiment, the determining module 604 is specifically configured to:
determining a corner displacement graph according to the corner displacement;
and determining the heart rate of the human body according to the angular point displacement graph and Fourier transform.
In an embodiment, the acquisition module 601 is specifically configured to:
determining the penetrability of light sources with different wavelengths to human skin;
determining the effect of light sources of different wavelengths on heart rate based on the penetration;
determining a target light source according to the influence of light sources with different wavelengths on the heart rate, and acquiring the speckle image of the human body according to the target light source.
In an embodiment, the reconstruction module 602 is specifically configured to:
inserting pixel points into the speckle images according to the image super-resolution reconstruction algorithm;
and determining the brightness value of the pixel point, and filling the brightness of the pixel point according to the brightness value.
Fig. 7 illustrates a physical structure diagram of an electronic device, and as shown in fig. 7, the electronic device may include: a processor (processor) 710, a communication Interface (Communications Interface) 720, a memory (memory) 730, and a communication bus 740, wherein the processor 710, the communication Interface 720, and the memory 730 communicate with each other via the communication bus 740. Processor 710 may invoke logic instructions in memory 730 to perform a heart rate detection method comprising:
collecting speckle images of a human body;
performing pixel reconstruction on the speckle images according to an image super-resolution reconstruction algorithm, and filtering the reconstructed speckle images;
tracking the angular points in the speckle images after filtering according to angular point detection and a pyramid optical flow method, and determining angular point displacement according to a tracking result;
and determining the heart rate of the human body according to the angular point displacement.
In addition, the logic instructions in the memory 730 can be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention or a part thereof which substantially contributes to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
In another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements a method for heart rate detection provided by the above methods, the method comprising:
collecting speckle images of a human body;
performing pixel reconstruction on the speckle images according to an image super-resolution reconstruction algorithm, and filtering the reconstructed speckle images;
tracking the angular points in the speckle images after filtering according to angular point detection and a pyramid optical flow method, and determining angular point displacement according to a tracking result;
and determining the heart rate of the human body according to the angular point displacement.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, and not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method of heart rate detection, comprising:
collecting speckle images of a human body;
performing pixel reconstruction on the speckle images according to an image super-resolution reconstruction algorithm, and filtering the reconstructed speckle images;
tracking the corner points in the speckle images after filtering according to corner point detection and a pyramid optical flow method, and determining corner point displacement according to a tracking result;
and determining the heart rate of the human body according to the angular point displacement.
2. The heart rate detection method according to claim 1, wherein the tracking the corner points in the filtered speckle images according to the corner point detection and the pyramid optical flow method comprises:
determining a corner to be tracked in the speckle image after filtering according to the corner detection;
and tracking the corner points to be tracked according to the pyramid optical flow method.
3. The heart rate detection method according to claim 2, wherein the determining a corner to be tracked in the filtered speckle image according to the corner detection comprises:
determining the minimum characteristic value of the corner points, and deleting the corner points with characteristic values smaller than the minimum characteristic value;
and sequencing the corner points with the characteristic values larger than the minimum characteristic value, and determining the corner point to be tracked according to a sequencing result.
4. The heart rate detection method according to claim 2, wherein the tracking the corner points to be tracked according to the pyramid optical flow method comprises:
determining the state of each corner point in the current frame according to the pyramid optical flow method, the corner point to be tracked of the previous frame and the current frame;
and tracking the angular points to be tracked according to the state of each angular point in the current frame.
5. The heart rate detection method according to claim 1, wherein the determining the heart rate of the human body according to the corner point displacement comprises:
determining a corner displacement graph according to the corner displacement;
and determining the heart rate of the human body according to the angular point displacement graph and Fourier transform.
6. The heart rate detection method of claim 1, wherein the acquiring speckle images of the human body comprises:
determining the penetrability of light sources with different wavelengths to human skin;
determining the effect of light sources of different wavelengths on heart rate based on the penetration;
determining a target light source according to the influence of light sources with different wavelengths on the heart rate, and acquiring the speckle image of the human body according to the target light source.
7. The heart rate detection method according to claim 1, wherein the pixel reconstruction of the speckle image according to an image super-resolution reconstruction algorithm comprises:
inserting pixel points into the speckle images according to the image super-resolution reconstruction algorithm;
and determining the brightness value of the pixel point, and filling the brightness of the pixel point according to the brightness value.
8. A heart rate detection device, comprising:
the acquisition module is used for acquiring speckle images of a human body;
the reconstruction module is used for carrying out pixel reconstruction on the speckle images according to an image super-resolution reconstruction algorithm and filtering the reconstructed speckle images;
the tracking module is used for tracking the angular points in the speckle images after filtering according to angular point detection and a pyramid optical flow method and determining angular point displacement according to a tracking result;
and the determining module is used for determining the heart rate of the human body according to the angular point displacement.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements a heart rate detection method according to any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, implements the heart rate detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210804138.1A CN115294027A (en) | 2022-07-07 | 2022-07-07 | Heart rate detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210804138.1A CN115294027A (en) | 2022-07-07 | 2022-07-07 | Heart rate detection method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115294027A true CN115294027A (en) | 2022-11-04 |
Family
ID=83822003
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210804138.1A Pending CN115294027A (en) | 2022-07-07 | 2022-07-07 | Heart rate detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115294027A (en) |
-
2022
- 2022-07-07 CN CN202210804138.1A patent/CN115294027A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110740688B (en) | Method for microvascular super-resolution ultrasound imaging | |
| Abd-Elmoniem et al. | Real-time speckle reduction and coherence enhancement in ultrasound imaging via nonlinear anisotropic diffusion | |
| US20230263463A1 (en) | Osteoporosis diagnostic support apparatus | |
| JP6107537B2 (en) | Imaging system and image processing method thereof, image processing apparatus and image processing method thereof, and program | |
| KR101121396B1 (en) | System and method for providing 2-dimensional ct image corresponding to 2-dimensional ultrasound image | |
| CN108670239B (en) | Three-dimensional blood flow imaging method and system based on feature space | |
| JP5948831B2 (en) | Measuring device, measuring method, program, and recording medium | |
| KR101842043B1 (en) | Apparatus and Method for analyzing ultrasound images | |
| JP2007313360A (en) | X-ray image processing | |
| CN111938622B (en) | Heart rate detection method, device and system and readable storage medium | |
| CN112949349A (en) | Method and system for displaying pulse condition waveform in real time based on face video | |
| US10667704B2 (en) | Apparatus and method for measuring the quality of an extracted signal | |
| US8500646B2 (en) | Color Doppler mode image processing in an ultrasound system | |
| Gökdağ et al. | Image denoising using 2-D wavelet algorithm for Gaussian-corrupted confocal microscopy images | |
| JP5990906B2 (en) | Measuring device, measuring method, program, and recording medium | |
| CN111513701A (en) | Heart rate detection method and device, computer equipment and readable storage medium | |
| CN113017593B (en) | Blood vessel tail artifact removing method and system based on blood flow signal intensity layered filtering | |
| JP7124974B2 (en) | Blood volume pulse signal detection device, blood volume pulse signal detection method, and program | |
| CN115294027A (en) | Heart rate detection method and device, electronic equipment and storage medium | |
| CN116350198A (en) | Remote moving target heart rate real-time monitoring method | |
| Zhang et al. | Heart rate measurement based on video acceleration magnification | |
| CN104688199A (en) | Non-contact type pulse measurement method based on skin pigment concentration difference | |
| CN113706567A (en) | Blood flow imaging quantitative processing method and device combining blood vessel morphological characteristics | |
| JP7237768B2 (en) | Biological information detector | |
| Daniel et al. | Retinal image enhancement using wavelet domain edge filtering and scaling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |