WO2023165482A1

WO2023165482A1 - Method and apparatus for heart rate detection

Info

Publication number: WO2023165482A1
Application number: PCT/CN2023/078835
Authority: WO
Inventors: 张智瑞; 许天骄; 邱翰
Original assignee: 虹软科技股份有限公司
Priority date: 2022-03-02
Filing date: 2023-02-28
Publication date: 2023-09-07
Also published as: CN114601437A

Abstract

The present disclosure provides a method and an apparatus for heart rate detection, and the method comprises: acquiring a fingerprint image sequence; determining, according to an effective fingerprint area of each frame of fingerprint image in the fingerprint image sequence, an initial signal sequence corresponding to the fingerprint image sequence, and performing time-frequency analysis on the initial signal sequence to obtain a heart rate. According to the disclosed solution, the heart rate detection is realized on the premise of an existing optical imaging system for fingerprint identification, such that the detection accuracy is not easily interfered with noise, and the application scenario is broadened.

Description

A heart rate detection method and device

This disclosure claims the priority of the Chinese patent application with the application number 202210199507.9 and the application name "a heart rate detection method and device" submitted to the China Patent Office on March 2, 2022, the entire contents of which are incorporated by reference in this disclosure.

technical field

The present disclosure relates to heart rate detection technology, in particular to a heart rate detection method and device.

Background technique

Electrocardiography (ECG) is a technology that uses an electrocardiograph to record the point activity change graphics produced by each cardiac cycle of the heart from the body surface. Although this method has high accuracy, the instrument is expensive, and requires professionals to operate, the equipment is cumbersome, and the use scenarios are extremely limited.

Photoplethysmography (ppg) is a non-invasive detection method for monitoring blood volume changes in living tissue by means of photoelectric means. When light of a certain wavelength is used to irradiate the skin surface such as between fingers, the light will be captured by the photoelectric receiver through transmission or reflection. During the whole process, the light is weakened due to the absorption of skin, muscle, blood, etc., and the light intensity reaching the photoelectric receiver will decrease. The weakening effect of the skin, muscle, etc. on the light intensity is constant, while the weakening effect of the blood in the blood vessel on the light will show pulsating changes with the beating of the heart. When the heart contracts, the volume of blood in the blood vessel increases, the intensity of the light absorbed increases, and the light intensity received by the photoelectric receiver decreases accordingly, and the opposite happens when the heart relaxes. By converting the light intensity into an electrical signal, changes in the volumetric pulse and blood flow can be obtained. This method requires the sensor to be in close contact with a fixed body part, which has many restrictions on the user's use method and scene.

Existing heart rate detection relies on special light sources and specific sensors on the one hand, and additional modules need to be added to the hardware. On the other hand, it only uses the time-domain counting method to estimate the heart rate, and the accuracy of the result is easily affected by noise. For the above problems, no effective method has been proposed so far. solution.

Contents of the invention

The present disclosure provides a heart rate detection method and device, which can realize heart rate detection on the premise of an existing fingerprint identification optical imaging system, so that the detection accuracy is not easily disturbed by noise, and the application scene is broadened.

The present disclosure provides a heart rate detection method, comprising: collecting a sequence of fingerprint images; determining an initial signal sequence corresponding to the sequence of fingerprint images according to the effective fingerprint area of each frame of the fingerprint image sequence in the sequence of fingerprint images; Time-frequency domain analysis was performed on the sequence to obtain heart rate.

In the present disclosure, the collection of fingerprint image sequences by the collection device may include:

The screen light source emits light of a preset color to irradiate the fingerprint area;

Image acquisition is performed based on the light signal returned by the fingerprint area, and the fingerprint image sequence is acquired.

In the present disclosure, before determining the initial signal sequence corresponding to the fingerprint image sequence according to the valid fingerprint area of each frame of the fingerprint image sequence in the fingerprint image sequence, the method may further include:

Determine the effective fingerprint area of each frame of fingerprint image in the fingerprint image sequence.

In the present disclosure, the determination of the effective fingerprint area of each frame of the fingerprint image in the fingerprint image sequence includes: performing the following operations on each frame of the fingerprint image:

Filter out the region containing the fingerprint from the frame of the fingerprint image;

An area meeting the first condition is selected from the areas containing fingerprints as the effective fingerprint area.

In the present disclosure, determining the initial signal sequence corresponding to the fingerprint image sequence according to the valid fingerprint area of each frame of the fingerprint image sequence in the fingerprint image sequence may include:

Calculate the initial signal corresponding to the current frame fingerprint image according to the valid fingerprint area of the current frame fingerprint image in the fingerprint image sequence;

The initial signal corresponding to the fingerprint image of the current frame and the initial signal corresponding to the fingerprint image of the previous N frames in history constitute the initial signal sequence.

In the present disclosure, the calculation of the initial signal corresponding to the current frame fingerprint image according to the effective fingerprint area of the current fingerprint frame image in the fingerprint image sequence may include:

Perform data processing on the pixel values of the plurality of valid fingerprint areas in the current frame fingerprint image to obtain an initial signal corresponding to the current frame fingerprint image.

In the present disclosure, after acquiring the initial signal sequence, the method may further include:

performing preprocessing on the initial signal sequence in the time domain to obtain a time domain signal of the initial signal sequence;

Wherein, the pretreatment may include any one or more of the following:

Detrending, sliding filtering, bandpass filtering, and normalization.

In the present disclosure, before performing time-frequency domain analysis on the initial signal sequence to obtain the heart rate, the method may further include: performing validity judgment on the initial signal sequence.

In the present disclosure, judging the validity of the initial signal sequence may include:

extracting eigenvalues of preset features from the initial signal sequence;

comparing the characteristic value with a preset standard characteristic value;

When the characteristic value conforms to the preset floating range of the standard characteristic value, it is determined that the initial signal sequence is a valid signal;

When the characteristic value does not meet the preset floating range of the standard characteristic value, it is determined that the initial signal sequence is an invalid signal.

In the present disclosure, the preset features may include any one or more of the following: area, entropy, skewness and kurtosis.

In the present disclosure, the time-frequency domain analysis of the initial signal sequence to obtain the heart rate may include:

converting the initial signal sequence from a time-domain signal to a first signal, wherein the first signal is a frequency-domain signal or a time-frequency domain signal;

performing secondary frequency-domain filtering and post-processing on the first signal to obtain a second signal;

According to the type of the second signal, the second signal is analyzed and counted to obtain the heart rate, wherein the second signal is a frequency domain signal or a time-frequency domain signal.

In the present disclosure, when the category of the second signal is a frequency domain signal, analyzing and counting the second signal and obtaining the heart rate may include:

using a preset peak detection algorithm to obtain a peak of the second signal within a preset heart rate range;

Sorting all the obtained peaks according to the peak size, obtaining the highest peak with the largest peak, and calculating the confidence of the highest peak;

When the confidence degree is greater than or equal to a first threshold, the mode of the frequencies of all peaks is obtained as the heart rate.

In the present disclosure, the calculating the confidence degree of the highest peak may include:

Taking the ratio of the energy of the highest peak to the sum of the energies of other peaks in all the peaks except the highest peak as the confidence of the highest peak; or,

The ratio of the peak value of the highest peak to the peak value of the second highest peak is taken as the confidence level of the highest peak.

In the present disclosure, when the category of the second signal is a time-frequency domain signal, analyzing and counting the second signal to obtain the heart rate may include:

performing frequency maximum detection on the second signal according to the time domain to obtain a plurality of frequency domain signals within a preset heart rate range;

calculating confidence levels for the plurality of frequency domain signals;

When the confidence degree is greater than or equal to a second threshold, the mode of frequencies of the multiple frequency domain signals is acquired as the heart rate.

In the present disclosure, the detecting the frequency maximum value of the second signal according to the time domain to obtain a plurality of frequency domain signals within a preset heart rate range may include:

acquiring a time-frequency coordinate diagram corresponding to the second signal;

For each preset time point in the time axis, a plurality of heart rate candidate frequency domain signals are obtained from all frequency domain signals corresponding to the preset time point, and a frequency response maximum frequency response is selected from the plurality of heart rate candidate frequency domain signals. frequency domain signal;

Accumulate a plurality of preset time points, and respond to the frequency response corresponding to each preset time point The largest frequency-domain signal is recorded, and multiple frequency-domain signals varying along the time axis within the possible range of the heart rate are obtained.

In the present disclosure, the calculating the confidence levels of the multiple frequency domain signals may include:

Calculate the standard deviation of the multiple frequency domain signals as the confidence of the multiple frequency domain signals.

In the present disclosure, the method may also include:

When the confidence degree is less than the preset threshold value, discard the earliest initial signal obtained in the initial signal sequence, and add the initial signal corresponding to the fingerprint image of the next frame to the initial signal sequence, so as to implement the initial signal sequence update;

The updated time-domain signal of the initial signal sequence is obtained, and time-frequency domain analysis is performed on the updated time-domain signal of the initial signal sequence to obtain the heart rate.

The peak detection is performed on the second signal according to the time domain, and the heart rate is calculated according to the number of detected peaks and a preset heart rate calculation formula.

The present disclosure also provides a heart rate detection device, which may include a processor and a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the instructions are executed by the processor, any of the above-mentioned A described heart rate detection method.

The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the heart rate detection method described in any one of the foregoing is implemented.

Compared with related technologies, the present disclosure may include: collecting a sequence of fingerprint images when performing heart rate detection; determining an initial signal sequence corresponding to the sequence of fingerprint images according to the valid fingerprint area of each frame of the fingerprint image in the sequence of fingerprint images; Time-frequency domain analysis is performed on the initial signal sequence to obtain the heart rate. Through the disclosed embodiment, the heart rate detection is realized on the premise of the existing fingerprint identification optical imaging system, so that the detection accuracy is not easily disturbed by noise, and the application scene is broadened.

Additional features and advantages of the disclosure will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the disclosure. Other advantages of the present disclosure can be realized and obtained through the solutions described in the specification and the accompanying drawings.

Description of drawings

The accompanying drawings are used to provide an understanding of the technical solutions of the present disclosure, and constitute a part of the description, and are set together with the embodiments of the embodiments of the present disclosure to explain the technical solutions of the present disclosure, and do not constitute limitations to the technical solutions of the embodiments of the present disclosure.

FIG. 1 is a flowchart of a heart rate detection method according to an embodiment of the present disclosure;

2 is a flowchart of a method for determining an initial signal sequence corresponding to a fingerprint image sequence according to an effective fingerprint area of each frame of the fingerprint image sequence in an embodiment of the present disclosure;

FIG. 3 is a flow chart of the first solution for obtaining heart rate by performing time-frequency domain analysis on the time-domain signal of the initial signal sequence according to an embodiment of the present disclosure;

FIG. 4 is a flow chart of a second solution for performing time-frequency domain analysis on the time-domain signal of the initial signal sequence to obtain heart rate according to an embodiment of the present disclosure;

FIG. 5 is a flow chart of a third scheme for performing time-frequency domain analysis on the time-domain signal of the initial signal sequence to obtain heart rate according to an embodiment of the present disclosure;

FIG. 6 is a block diagram of a heart rate detection device according to an embodiment of the disclosure.

Detailed ways

The present disclosure describes a number of embodiments, but the description is illustrative rather than restrictive, and it will be apparent to those of ordinary skill in the art that within the scope encompassed by the described embodiments of the present disclosure, There are many more embodiments and implementations. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are possible. Except where expressly limited, any feature or element of any embodiment may be used in combination with, or substituted for, any other feature or element of any other embodiment.

This disclosure includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The disclosed embodiments, features and elements of this disclosure may also be combined with any conventional feature or element to form unique inventive solutions as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive solutions to form another embodiment as defined by the claims. unique invention. It is therefore to be understood that any of the features shown and/or discussed in this disclosure can be implemented alone or in any suitable combination. Accordingly, the embodiments are not to be limited except in accordance with the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

Furthermore, in describing representative embodiments, the specification may have presented a method and/or process as a particular sequence of steps. However, to the extent the method or process is not dependent on the specific order of steps described herein, the method or process should not be limited to the specific order of steps described. Other sequences of steps are also possible, as will be appreciated by those of ordinary skill in the art. Therefore, the specific order of the steps set forth in the specification should not be construed as limitations on the claims. Furthermore, claims to the method and/or process should not be limited to performing their steps in the order written, as those skilled in the art can readily appreciate that such order can be varied and still remain within the spirit and scope of the disclosed embodiments Inside.

An embodiment of the present disclosure provides a heart rate detection method, as shown in FIG. 1, the method may include steps S101-S103:

S101. Collect fingerprint image sequences;

S102. Determine an initial signal sequence corresponding to the fingerprint image sequence according to the valid fingerprint area of each frame of the fingerprint image sequence in the fingerprint image sequence;

S103. Perform time-frequency domain analysis on the initial signal sequence to obtain the heart rate.

In an exemplary embodiment of the present disclosure, there is no limit to the manner of collecting the sequence of fingerprint images, for example, it may be collected by an under-screen camera. The solutions of the embodiments of the present disclosure can realize heart rate detection through an off-screen fingerprint heart rate estimation algorithm. The off-screen fingerprint heart rate estimation algorithm proposed by the embodiments of the present disclosure mainly includes two modules: an off-screen camera module in the form of hardware and a signal processing module in the form of software. Wherein, the camera module under the screen may include: a light source under the screen and a camera under the screen, the light source under the screen is set to irradiate the fingerprint area, and the camera collects the light signal returned by the user's finger to obtain the fingerprint image sequence (ie finger reflection imaging video); And use the signal processing module to detect and locate the fingerprint area of the fingerprint signal in the obtained fingerprint image sequence, perform denoising processing, and convert it into a digital signal. Specifically, the form of the digital signal is not limited in this disclosure. The digital signal can be The photoplethysmography ppg signal (referred to as ppg signal) is analyzed by an algorithm to obtain an estimated value of the current heart rate.

In the exemplary embodiment of the present disclosure, the input of sensor data required by the solution of the embodiment of the present disclosure is relatively simple, just the sequence of fingerprint images collected by the camera under the screen, usually greater than 20fps.

In an exemplary embodiment of the present disclosure, collecting a sequence of fingerprint images may include:

Image acquisition is performed based on the light signal returned by the fingerprint area to obtain a sequence of fingerprint images.

In an exemplary embodiment of the present disclosure, the light of a preset color emitted by the screen light source may include but not limited to green light. Specifically, the screen light source is set to provide a light signal for fingerprint detection, and a built-in light source or an external light source may be used. light source. The fingerprint area is the area where the user's finger can be pressed. This disclosure does not limit the position of the fingerprint area on the detection device. It ensures that the screen light source emits light of a preset color and can illuminate the fingerprint area. The light signal is scattered or reflected by the finger to the collection device. Finally, a sequence of fingerprint images can be obtained.

In an exemplary embodiment of the present disclosure, before determining the initial signal sequence corresponding to the fingerprint image sequence according to the valid fingerprint area of each frame of the fingerprint image sequence in the fingerprint image sequence, the method may include:

Determine the valid fingerprint area of each frame of fingerprint image in the sequence of fingerprint images.

In an exemplary embodiment of the present disclosure, determining the effective fingerprint area of each frame of the fingerprint image in the fingerprint image sequence may include: performing the following operations on each frame of the fingerprint image:

An area meeting the first condition (for example, an area with a preset color) is selected from the areas containing fingerprints as valid fingerprint areas.

In the exemplary embodiment of the present disclosure, due to factors affecting data validity such as noise and finger gestures in fingerprint collection, it is necessary to divide and screen out valid fingerprint regions for fingerprint image sequences collected from fingerprint regions, that is, to effectively extract different The fingerprint characteristics of the position are used to counteract the influence of the above factors, and finally based on the effective fingerprint area. The effective fingerprint area above refers to an area with a strong heart rate signal, which is more conducive to subsequent heart rate detection. In an exemplary embodiment of the present disclosure, firstly, the area containing the fingerprint is screened out in the frame of the fingerprint image, that is, the area where the finger actually touches the touch screen; combined with the first condition, an effective fingerprint area is selected from the area containing the fingerprint Specifically, the first condition may include a finger pressing state and/or blood vessel distribution.

In an exemplary embodiment of the present disclosure, the blood vessels on the finger are rich in distribution, but the blood flow distribution is variable, and when the initial signal is a ppg signal, the information contained in the blood flow is used. Depending on the degree of pressing, there are different treatments: if the pressing is too light, the contact area is not enough, and there will be no fingerprints in the untouched area; if the pressing is heavy, the capillary blood flow in the area with the highest pressure will be blocked (that is, some areas of the fingers will be blocked). White without blood), does not meet the conditions for measuring the initial signal. Therefore, the area containing the fingerprint (that is, the actual contact area) and rich in blood flow (which can be judged by whether there is blood color) can be selected as the effective fingerprint area according to the actual situation.

In an exemplary embodiment of the present disclosure, determining the initial signal sequence corresponding to the fingerprint image sequence according to the valid fingerprint area of each frame of the fingerprint image sequence in the fingerprint image sequence may include:

According to the effective fingerprint area of the current frame fingerprint image in the fingerprint image sequence and calculate the initial signal corresponding to the current frame fingerprint image;

The initial signal corresponding to the fingerprint image of the current frame and the initial signal corresponding to the fingerprint image of the previous N frames constitute an initial signal sequence.

In an exemplary embodiment of the present disclosure, as shown in FIG. 2, the detailed steps of the above solution may include S201-S204:

S201. Calculate an initial signal corresponding to the fingerprint image of the current frame.

In an exemplary embodiment of the present disclosure, calculating the initial signal corresponding to the fingerprint image of the current frame may include:

Data processing is performed on the pixel values of multiple valid fingerprint areas in the current frame fingerprint image to obtain an initial signal corresponding to the current frame fingerprint image.

In an exemplary embodiment of the present disclosure, the data processing may refer to weighted average, which performs weighted average on the pixel values of multiple effective fingerprint regions to obtain the sampling point value of the initial signal of the fingerprint image of this frame. In the weighted average, the weighted values of different effective fingerprint regions can be determined according to pre-experimentation, for example, the weight of the region with strong heart rate signal is relatively high, and the weight of the region with low heart rate signal is relatively low.

S202. Accumulate the initial signal corresponding to the fingerprint image of the current frame and the initial signals corresponding to the fingerprint images of multiple frames before the fingerprint image of the current frame.

S203. Detect whether the number of initial signals has accumulated to the preset number N+1, when the number of initial signals When the amount has accumulated to the preset number N+1, enter step S204; when the amount of the initial signal has not accumulated to the preset number N+1, obtain the next frame of fingerprint image as the current frame fingerprint image, and return to step S201; N is a positive integer.

S204. Form the accumulated N+1 initial signals into an initial signal sequence.

In an exemplary embodiment of the present disclosure, the ppg sampling point values of the current frame fingerprint image and the ppg sampling point values of the previous N frames of fingerprint images form a set of ppg signals, that is, an initial signal sequence.

In an exemplary embodiment of the present disclosure, the historical frame is relative to the current frame, and the previous N frames in the history mean that up to the current frame, N frames of fingerprint images have been accumulated in the current frame, excluding the current frame. For example: the number of the fingerprint image of the current frame is M, and the number of the fingerprint image of the previous N frames is [M-N, M-N+1, M-N+2, ..., M-1].

In an exemplary embodiment of the present disclosure, a sampling point value is obtained by performing weighted average on the effective fingerprint area of the current frame fingerprint image, and combined with signals obtained by the same method in previous N frames of history to obtain an initial signal sequence. Since multiple effective fingerprint areas are screened out in each frame of fingerprint image, it can be considered that there are multiple large effective fingerprint areas on one fingerprint image, and the pixels in each effective fingerprint area are required by the scheme of the embodiment of the present disclosure. Pixels in the valid fingerprint area are not needed. The ppg signal of the sampling point of the fingerprint image in this frame is also the weighted average of the pixel values in all valid fingerprint areas in the current fingerprint image to obtain a ppg value, that is, the sampling value of the ppg signal. When the fingerprint image of each frame in the fingerprint image sequence is obtained When the ppg signal sampling values are connected into an array (forming a 1×(N+1) one-dimensional signal array), an initial signal sequence is generated, which is the original ppg signal.

In an exemplary embodiment of the present disclosure, after acquiring the time-domain signal of the initial signal sequence, the method may further include:

The initial signal sequence is preprocessed in the time domain to obtain the time domain signal of the initial signal sequence.

In an exemplary embodiment of the present disclosure, preprocessing may include any one or more of the following:

Detrending, sliding filtering, bandpass filtering, and normalization.

In an exemplary embodiment of the present disclosure, through this preprocessing, the time domain signal after denoising can be obtained number, which improves the quality of the data used to estimate the heart rate, thereby ensuring the accuracy of subsequent heart rate estimates.

In an exemplary embodiment of the present disclosure, before performing time-frequency domain analysis on the initial signal sequence to obtain the heart rate, the method may further include: performing validity judgment on the initial signal sequence.

In an exemplary embodiment of the present disclosure, the solution of this embodiment can combat problems such as noise and finger gestures that affect data validity.

In an exemplary embodiment of the present disclosure, the validity judgment of the initial signal sequence may include:

Extracting eigenvalues of preset features from the initial signal sequence;

Compare the eigenvalues with preset standard eigenvalues;

When the eigenvalue meets the preset floating range of the standard eigenvalue, it is determined that the initial signal sequence is a valid signal;

When the eigenvalue does not meet the preset floating range of the standard eigenvalue, it is determined that the initial signal sequence is an invalid signal.

In an exemplary embodiment of the present disclosure, the preset features may include any one or more of the following: area, entropy, skewness, and kurtosis.

In an exemplary embodiment of the present disclosure, the area feature extraction is the closed area where the ppg sequence signal and the mean value intersect (that is, the curve corresponding to the ppg sequence signal intersects with the straight line corresponding to the ppg mean value calculated by the ppg sequence signal, area enclosed by curves and lines). Features such as entropy, skewness, and kurtosis directly process the initial signal sequence as a whole, and are classic feature indicators.

In an exemplary embodiment of the present disclosure, performing time-frequency domain analysis on the initial signal sequence to obtain the heart rate may include:

converting the initial signal sequence from a time-domain signal to a first signal, where the first signal is a frequency-domain signal or a time-frequency domain signal;

In an exemplary embodiment of the present disclosure, the initial signal sequence is converted from a time-domain signal to a first signal. Since the first signal is a frequency-domain signal or a time-frequency domain signal, the second signal obtained after filtering and post-processing is also a frequency domain signal. Domain signal or time-frequency domain signal, depending on the type of the signal, performing time-frequency domain analysis on the signal sequence to obtain the heart rate may include various schemes, and three schemes of embodiments are given below.

Option One

In an exemplary embodiment of the present disclosure, as shown in FIG. 3, when the category of the second signal is a frequency domain signal, analyzing and counting the second signal to obtain the heart rate may include steps S301-S303:

S301. Obtain the peak of the power spectrum signal (second signal) within the preset heart rate range by using a preset peak detection algorithm;

S302. Sort all the obtained peaks according to the peak size, obtain the highest peak with the largest peak, and calculate the confidence of the highest peak;

S303. When the confidence level is greater than or equal to the first threshold (which is a preset threshold), acquire the mode of frequencies of all peaks as the heart rate.

In an exemplary embodiment of the present disclosure, the present disclosure does not limit the time-frequency domain conversion method, and the denoised time-domain signal of the initial signal sequence may be converted into a frequency-domain signal through fast Fourier transform. Fast Fourier transform (FFT, fast fourier transform) is a classic algorithm that inputs a time-domain signal and obtains a frequency-domain signal after FFT.

In an exemplary embodiment of the present disclosure, the obtained frequency-domain signal may be subjected to secondary frequency-domain filtering and post-processing to obtain a power spectrum signal with obvious peaks. Use peak detection to obtain the peak of the power spectrum signal within the possible range of heart rate, and use peak sorting and peak size ratio to calculate the confidence of the highest peak.

In an exemplary embodiment of the present disclosure, calculating the confidence of the highest peak may include:

The ratio of the energy of the highest peak to the sum of the energies of all other peaks except the highest peak in all peaks is taken as the confidence level of the highest peak; or,

The confidence of the highest peak is taken as the ratio of the peak value of the highest peak to the peak value of the second highest peak.

In an exemplary embodiment of the present disclosure, the ratio of the energy of the highest peak to the sum of the energies of other peaks is the signal-to-noise ratio, and the signal-to-noise ratio may be used as the confidence of the highest peak.

In an exemplary embodiment of the present disclosure, when the calculated confidence is greater than or equal to a preset confidence threshold (the confidence threshold may be the first threshold), it may be considered that the initial signal sequence is not polluted by noise, and the result It is credible, and the frequency of the peak can be obtained, and after conversion (for example, obtaining the mode of the frequency of all peaks) to obtain the main frequency corresponding to the input initial signal sequence, that is, the heart rate.

In an exemplary embodiment of the present disclosure, the method may further include:

When the confidence is less than the preset confidence threshold (for example, the first threshold), discard the earliest initial signal obtained in the initial signal sequence, and obtain the initial signal corresponding to the next frame of fingerprint image and add it to the initial signal sequence, so as to realize the initial signal sequence update;

The time-domain signal of the updated initial signal sequence is obtained, and the time-frequency domain analysis is performed on the time-domain signal of the updated initial signal sequence to obtain the heart rate.

In an exemplary embodiment of the present disclosure, when the calculated confidence is less than the preset threshold, it indicates that the initial signal sequence is seriously polluted by noise (the signal confidence calculated from the highest peak of the signal is one of the initial signal sequences Inspection standard. If this standard is not qualified, it is not credible. Only when the standard is qualified can the process of obtaining heart rate according to the initial signal sequence be entered), and the current result can be discarded to enter the next frame of fingerprint image. On the basis of the accumulated initial signal sequence, continue to process the sampling point signal of the next frame of fingerprint image.

In an exemplary embodiment of the present disclosure, each frame of fingerprint image can obtain a ppg sampling point, or obtain a ppg signal, and it is necessary to continuously obtain N ppg signals to form an initial signal sequence, and then perform Signal processing, in the case of unqualified standards, you can go back to the initial step to obtain a new frame of fingerprint image in the fingerprint image sequence, calculate the ppg signal of the frame fingerprint image, and add it to the end of the initial signal sequence as a new ppg signal. The oldest ppg signal at the head end of the signal sequence is eliminated, and then the updated constant n-length initial signal sequence is denoised and validity judged, and the heart rate is obtained according to the updated initial signal sequence.

Option II

In an exemplary embodiment of the present disclosure, as shown in FIG. 4, when the category of the second signal is a time-frequency domain signal, analyzing and counting the second signal to obtain the heart rate may include steps S401-S403:

S401. Perform frequency maximum detection on the second signal according to the time domain, and obtain heart rate within the preset heart rate range Multiple frequency domain signals within;

S402. Calculate confidence levels of multiple frequency domain signals;

S403. When the confidence level is greater than or equal to the second threshold (which is a preset threshold), acquire the mode of the frequencies of the multiple frequency domain signals as the heart rate.

In an exemplary embodiment of the present disclosure, the denoised time-domain signal is decomposed using a time-frequency analysis method, and then the frequency-domain signal is subjected to secondary frequency-domain filtering and post-processing to obtain a "clean" time-frequency signal, The present disclosure does not limit the time-frequency analysis method, and the time-domain signal can be converted into a time-frequency signal through short-time Fourier transform. In order to obtain higher time-frequency resolution and better distinguish random noise and signal, the present disclosure can also use wavelet synchronous compression transform for time-frequency domain conversion. In addition, the present disclosure performs frequency maximum detection on a "clean" time-frequency signal according to the time domain to obtain a signal within the possible range of heart rate.

In an exemplary embodiment of the present disclosure, according to the time domain, the frequency maximum value detection is performed on the frequency domain signal after frequency domain filtering and post-processing, and multiple frequency domain signals within the possible heart rate range are obtained, which may include:

Obtain the time-frequency coordinate diagram corresponding to the frequency domain signal; the horizontal axis of the time-frequency coordinate diagram is the time axis, the vertical axis is the frequency domain axis, and each point in the time-frequency coordinate diagram is the frequency domain signal size;

For each preset time point in the time axis, multiple heart rate candidate frequency domain signals are obtained from all frequency domain signals corresponding to the preset time point, and the frequency domain signal with the largest frequency response is selected from the multiple heart rate candidate frequency domain signals ;

Accumulate multiple preset time points, and record the frequency domain signal with the largest frequency response corresponding to each preset time point, and obtain multiple frequency domain signals that change along the time axis within the possible range of heart rate.

In an exemplary embodiment of the present disclosure, the frequency response is the strength of the signal, not equal to the frequency.

In an exemplary embodiment of the present disclosure, calculating confidence levels of multiple frequency domain signals may include:

Compute the standard deviation of multiple frequency domain signals as the confidence level of multiple frequency domain signals.

When the degree of confidence is less than the degree of confidence threshold (the second threshold), discard the earliest initial signal obtained in the initial signal sequence, and obtain the corresponding initial signal of the next frame of fingerprint image to join the initial signal sequence, so as to realize the update of the initial signal sequence;

In an exemplary embodiment of the present disclosure, the processing solution when the confidence level is less than the confidence level threshold is the same as the processing solution in Solution 1, and will not be repeated here.

third solution

In an exemplary embodiment of the present disclosure, as shown in FIG. 5, when the category of the second signal is a time-frequency domain signal, analyzing and counting the second signal to obtain the heart rate may include step S501:

S501. Perform peak detection on the second signal according to the time domain, and calculate the heart rate according to the number of detected peaks and a preset heart rate calculation formula.

In an exemplary embodiment of the present disclosure, the denoised time-domain signal is decomposed using a time-frequency analysis method to obtain a frequency-domain signal, and then the frequency-domain signal is subjected to secondary frequency-domain filtering and post-processing to obtain a "clean" time-frequency signal. The present disclosure does not limit the time-frequency analysis method, such as converting a time-domain signal into a time-frequency signal by short-time Fourier transform. In addition, the present disclosure performs peak detection on a "clean" time-frequency signal according to the time domain to calculate the number of peaks of the initial signal sequence.

In an exemplary embodiment of the present disclosure, the present disclosure may take a peak and a trough in the waveform of the initial signal sequence as a cycle, and calculate the number of cycles, that is, the number of peaks in the initial signal sequence.

In an exemplary embodiment of the present disclosure, calculating the heart rate according to the detected number of peaks and a preset heart rate calculation formula includes: calculating the heart rate according to the following heart rate calculation formula:

HR=P/T*M;

Wherein, HR is the heart rate, P is the number of peaks, and M is the unit duration (for example, 1 minute).

The embodiment of the present disclosure also provides a heart rate detection device 1, as shown in FIG. When executed by the processor 11, the heart rate detection method is realized.

In the exemplary embodiments of the present disclosure, any of the aforementioned embodiments of the heart rate detection method are applicable to the embodiment of the heart rate detection device, and will not be repeated here.

An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the heart rate detection method is implemented.

In the exemplary embodiments of the present disclosure, any of the aforementioned embodiments of the heart rate detection method are applicable to the embodiment of the computer-readable storage medium, and will not be repeated here.

Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, the functional modules/units in the system, and the device can be implemented as software, firmware, hardware, and an appropriate combination thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components. Components cooperate to execute. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. permanent, removable and non-removable media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, tape, magnetic disk storage or other magnetic storage devices, or can Any other medium used to store desired information and which can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

Claims

A heart rate detection method, comprising:

Collect fingerprint image sequence;

determining an initial signal sequence corresponding to the fingerprint image sequence according to the effective fingerprint area of each frame of the fingerprint image in the fingerprint image sequence;

Time-frequency domain analysis is performed on the initial signal sequence to obtain the heart rate.
The heart rate detection method according to claim 1, wherein said collection of fingerprint image sequences comprises:

The screen light source emits light of a preset color to irradiate the fingerprint area;

Image acquisition is performed based on the light signal returned by the fingerprint area, and the fingerprint image sequence is acquired.
The heart rate detection method according to claim 1, wherein, before determining the initial signal sequence corresponding to the fingerprint image sequence according to the valid fingerprint area of each frame of the fingerprint image sequence in the fingerprint image sequence, the method comprises:

Determine the effective fingerprint area of each frame of fingerprint image in the fingerprint image sequence.
The heart rate detection method according to claim 3, wherein said determining the effective fingerprint area of each frame of the fingerprint image in the fingerprint image sequence comprises: performing the following operations on each frame of the fingerprint image:

Filter out the region containing the fingerprint from the frame of the fingerprint image;

An area meeting the first condition is selected from the areas containing fingerprints as the effective fingerprint area.
The heart rate detection method according to claim 1, wherein said determining the initial signal sequence corresponding to the fingerprint image sequence according to the valid fingerprint area of each frame of the fingerprint image sequence in the fingerprint image sequence comprises:

calculating an initial signal corresponding to the current frame fingerprint image according to the valid fingerprint area of the current frame fingerprint image in the fingerprint image sequence;

The initial signal corresponding to the fingerprint image of the current frame and the initial signal corresponding to the fingerprint image of previous N frames constitute the initial signal sequence.
The heart rate detection method according to claim 5, wherein said calculating the initial signal corresponding to the fingerprint image of the current frame according to the effective fingerprint area of the fingerprint image of the current frame in the fingerprint image sequence comprises:

Perform data processing on the pixel values of the plurality of valid fingerprint areas in the current frame fingerprint image to obtain an initial signal corresponding to the current frame fingerprint image.
The heart rate detection method according to claim 1, wherein, after acquiring the initial signal sequence, the method further comprises:

Preprocessing the initial signal sequence in the time domain to obtain a time domain signal of the initial signal sequence;

Wherein, the pretreatment includes any one or more of the following:

Detrending, sliding filtering, bandpass filtering, and normalization.
The heart rate detection method according to claim 1, wherein, before performing time-frequency domain analysis on the initial signal sequence to obtain the heart rate, the method further includes: performing a validity judgment on the initial signal sequence.
The heart rate detection method according to claim 8, wherein said determining the validity of said initial signal sequence comprises:

extracting eigenvalues of preset features from the initial signal sequence;

comparing the characteristic value with a preset standard characteristic value;

When the characteristic value conforms to the preset floating range of the standard characteristic value, it is determined that the initial signal sequence is a valid signal;

When the characteristic value does not meet the preset floating range of the standard characteristic value, it is determined that the initial signal sequence is an invalid signal.
The heart rate detection method according to claim 9, wherein the preset features include any one or more of the following: area, entropy, skewness and kurtosis.
The heart rate detection method according to claim 1, wherein the time-frequency domain analysis of the initial signal sequence to obtain the heart rate includes:

converting the initial signal sequence from a time-domain signal to a first signal, wherein the first signal is a frequency-domain signal or a time-frequency domain signal;

performing secondary frequency-domain filtering and post-processing on the first signal to obtain a second signal;

According to the type of the second signal, the second signal is analyzed and counted to obtain the heart rate, wherein the type of the second signal is a frequency domain signal or a time-frequency domain signal.
The heart rate detection method according to claim 11, wherein when the category of the second signal is a frequency domain signal, analyzing and counting the second signal to obtain the heart rate includes:

using a preset peak detection algorithm to obtain the peak of the second signal within a preset heart rate range;

Sorting all the obtained peaks according to the peak size, obtaining the highest peak with the largest peak, and calculating the confidence of the highest peak;

When the confidence degree is greater than or equal to a first threshold, the mode of the frequencies of all peaks is obtained as the heart rate.
The heart rate detection method according to claim 12, wherein said calculating the confidence level of said highest peak comprises:

Taking the ratio of the energy of the highest peak to the sum of the energies of other peaks in all the peaks except the highest peak as the confidence of the highest peak; or,

The ratio of the peak value of the highest peak to the peak value of the second highest peak is taken as the confidence level of the highest peak.
The heart rate detection method according to claim 11, wherein when the category of the second signal is a time-frequency domain signal, analyzing and counting the second signal to obtain the heart rate includes:

performing frequency maximum detection on the second signal according to the time domain to obtain a plurality of frequency domain signals within a preset heart rate range;

calculating confidence levels for the plurality of frequency domain signals;

When the confidence degree is greater than or equal to a second threshold, the mode of frequencies of the multiple frequency domain signals is acquired as the heart rate.
The heart rate detection method according to claim 14, wherein the frequency maximum detection of the second signal according to the time domain to obtain multiple frequency domain signals within a preset heart rate range includes:

acquiring a time-frequency coordinate diagram corresponding to the second signal;

For each preset time point in the time axis, a plurality of heart rate candidate frequency domain signals are obtained from all frequency domain signals corresponding to the preset time point, and a frequency response maximum frequency response is selected from the plurality of heart rate candidate frequency domain signals. frequency domain signal;

Accumulate multiple preset time points, and record the frequency domain signal corresponding to each preset time point with the largest frequency response, and obtain multiple heart rate changes along the time axis within the possible heart rate range. frequency domain signal.
The heart rate detection method according to claim 14, wherein said calculating the confidence of said multiple frequency domain signals comprises:

Calculate the standard deviation of the multiple frequency domain signals as the confidence of the multiple frequency domain signals.
The heart rate detection method according to claim 12 or 14, wherein the method further comprises:

When the confidence degree is less than the preset threshold, the earliest obtained signal in the initial signal sequence is discarded, and the signal corresponding to the fingerprint image of the next frame is added to the initial signal sequence, so as to implement the initial signal sequence renew;

The updated time-domain signal of the initial signal sequence is obtained, and time-frequency domain analysis is performed on the updated time-domain signal of the initial signal sequence to obtain the heart rate.
The heart rate detection method according to claim 11, wherein when the category of the second signal is a time-frequency domain signal, analyzing and counting the second signal to obtain the heart rate includes:

Perform peak detection on the second signal according to the time domain, according to the number of detected peaks and the predicted The heart rate calculation formula is used to calculate the heart rate.
A heart rate detection device, comprising a processor and a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the instructions are executed by the processor, any one of claims 1-18 can be realized. The heart rate detection method described in the item.
A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the heart rate detection method according to any one of claims 1-18 is realized.