CN111310584A - Heart rate information acquisition method and device, computer equipment and storage medium - Google Patents

Heart rate information acquisition method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN111310584A
CN111310584A CN202010061378.8A CN202010061378A CN111310584A CN 111310584 A CN111310584 A CN 111310584A CN 202010061378 A CN202010061378 A CN 202010061378A CN 111310584 A CN111310584 A CN 111310584A
Authority
CN
China
Prior art keywords
sub
sequence
pulse wave
weight
subframe
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010061378.8A
Other languages
Chinese (zh)
Inventor
周康明
杨昭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Eye Control Technology Co Ltd
Original Assignee
Shanghai Eye Control Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Eye Control Technology Co Ltd filed Critical Shanghai Eye Control Technology Co Ltd
Priority to CN202010061378.8A priority Critical patent/CN111310584A/en
Publication of CN111310584A publication Critical patent/CN111310584A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024—Detecting, measuring or recording pulse rate or heart rate
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7253—Details of waveform analysis characterised by using transforms
    • A61B5/726—Details of waveform analysis characterised by using transforms using Wavelet transforms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/10—Image enhancement or restoration using non-spatial domain filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20064—Wavelet transform [DWT]

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Surgery (AREA)
  • Medical Informatics (AREA)
  • Veterinary Medicine (AREA)
  • Physiology (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Psychiatry (AREA)
  • Multimedia (AREA)
  • Cardiology (AREA)
  • Human Computer Interaction (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

The application relates to a heart rate information acquisition method and device, computer equipment and a storage medium. The method comprises the following steps: dividing the region of interest of each frame image in the plurality of frame images according to a preset dividing mode to obtain a plurality of subframe image sequences; acquiring a sub-pulse wave signal of each sub-frame image sequence; determining weight sequence information of the corresponding divided regions according to the sub-pulse wave signals; obtaining pulse wave signals of the plurality of frame images according to the weight sequence information and the sub pulse wave signals; and determining a heart rate estimation value according to the frequency corresponding to the maximum power spectral density of the pulse wave signal in a preset time period. By adopting the method, the accuracy of heart rate estimation can be improved.

Description

Heart rate information acquisition method and device, computer equipment and storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a heart rate information obtaining method and apparatus, a computer device, and a storage medium.
Background
The heart rate is an important index reflecting the physiological health state of a human body, and the heart rate of the human body needs to be detected in daily life of the human body, such as physical exercise. However, the traditional electrocardiogram used for representing the heart rate of a person is limited by the environment and cannot be used anytime and anywhere.
Therefore, research on the manner in which heart rate is obtained has begun. At present, the existing heart rate detection method can obtain heart rate related data based on recognition and certain analysis of a face image. However, the traditional method cannot exclude external interference, such as interference caused by facial movement or expression change, so that the obtained heart rate data is inaccurate.
Disclosure of Invention
In view of the above, it is necessary to provide a method, an apparatus, a computer device and a storage medium for acquiring heart rate information, which can be more accurate.
A heart rate information acquisition method, the method comprising:
dividing the region of interest of each frame image in the plurality of frame images according to a preset dividing mode to obtain a plurality of subframe image sequences; wherein the plurality of frame images include a plurality of images arranged in time sequence, and each of the sub-frame image sequences includes a plurality of sub-frame images arranged in time sequence in a corresponding divided region;
acquiring a sub-pulse wave signal of each sub-frame image sequence;
determining weight sequence information of the corresponding divided regions according to the sub-pulse wave signals; wherein the weight sequence information comprises a frequency domain weight sequence, a gradient weight sequence and a mask function representing the sub-frame image quality;
obtaining pulse wave signals of the plurality of frame images according to the weight sequence information and the sub pulse wave signals;
and determining a heart rate estimation value according to the frequency corresponding to the maximum power spectral density of the pulse wave signal in a preset time period.
In one embodiment, the acquiring a sub-pulse wave signal of each of the sub-frame image sequences includes:
carrying out chrominance feature extraction on the subframe images in the subframe image sequence to obtain the chrominance feature of each subframe image and form a chrominance feature sequence of the subframe image sequence;
and obtaining a sub-pulse wave signal of the sub-frame image sequence according to the chrominance characteristics of the sub-frame image and the corresponding pixel number of the divided area.
In one embodiment, the obtaining the sub-pulse wave signals of the sub-frame image sequence according to the chrominance features of the sub-frame images and the corresponding pixel numbers of the division areas includes:
and taking the ratio of the sum obtained by accumulating the chrominance characteristics of each pixel point of the subframe image to the pixel number of the subframe image as the point pulse wave of the corresponding moment of the subframe image so as to form the sub pulse wave signal corresponding to the subframe image sequence in which the subframe image is positioned.
In one embodiment, the determining the weight sequence information of the corresponding divided regions according to the sub-pulse wave signals includes:
performing wavelet transformation on the chrominance characteristic sequence of the subframe image sequence to obtain a wavelet transformation coefficient of the subframe image;
and obtaining the mask function corresponding to each sub-frame image sequence according to the belonged relationship between the maximum frequency of the wavelet transform coefficients of the sub-frame images in the frequency domain and a preset normal heart rate frequency range.
In one embodiment, the determining the weight sequence information of the corresponding divided regions according to the sub-pulse wave signals includes:
performing wavelet transformation on the chrominance characteristic sequence of the subframe image sequence to obtain a wavelet transformation coefficient of the subframe image;
acquiring the maximum amplitude and the second maximum amplitude of the wavelet transform coefficient of the subframe image at each moment in a frequency domain;
and obtaining the frequency domain weight of each moment by the ratio of the difference between the maximum amplitude and the secondary maximum amplitude of the wavelet transform coefficient of the subframe image of each moment to the secondary maximum amplitude so as to form the frequency domain weight sequence corresponding to each subframe image sequence.
In one embodiment, the determining the weight sequence information of the corresponding divided regions according to the sub-pulse wave signals includes:
solving an average gradient of the sub-frame images in the sub-frame image sequence to obtain an average gradient sequence of each sub-frame image sequence;
and obtaining the gradient weight sequence corresponding to each sub-frame image sequence according to the average gradient sequence and a preset adjusting hyper-parameter.
In one embodiment, the obtaining the pulse wave signals of the plurality of frame images according to the weight sequence information and the sub-pulse wave signals includes:
multiplying the frequency domain weight sequence, the gradient weight sequence and the mask function corresponding to the subframe image sequence according to the time correspondence to obtain the comprehensive weight of the subframe image sequence;
and accumulating the products of the comprehensive weight of each sub-frame image sequence and the corresponding sub-pulse wave signals, and obtaining the pulse wave signals of the plurality of frame images according to the ratio of the comprehensive weight sum of each sub-frame image sequence.
In one embodiment, the determining the heart rate estimate according to the frequency corresponding to the maximum power spectral density of the pulse wave signal within the preset time period includes:
filtering and frequency domain conversion are carried out on the pulse wave signals, and the frequency corresponding to the obtained maximum power spectral density is taken as a target frequency;
and determining the preset multiple of the target frequency as the heart rate estimated value.
A heart rate information acquisition apparatus, the apparatus comprising:
the dividing module is used for dividing the region of interest of each frame image in the plurality of frame images according to a preset dividing mode to obtain a plurality of subframe image sequences; wherein the plurality of frame images include a plurality of images arranged in time sequence, and each of the sub-frame image sequences includes a plurality of sub-frame images arranged in time sequence in a corresponding divided region;
the processing module is used for acquiring a sub-pulse wave signal of each sub-frame image sequence, determining weight sequence information of the corresponding divided area according to the sub-pulse wave signal, obtaining pulse wave signals of the plurality of frame images according to the weight sequence information and the sub-pulse wave signal, and determining a heart rate estimation value according to a frequency corresponding to the maximum power spectral density of the pulse wave signals in a preset time period; wherein the weight sequence information comprises a frequency domain weight sequence, a gradient weight sequence and a mask function representing the sub-frame image quality.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
dividing the region of interest of each frame image in the plurality of frame images according to a preset dividing mode to obtain a plurality of subframe image sequences; wherein the plurality of frame images include a plurality of images arranged in time sequence, and each of the sub-frame image sequences includes a plurality of sub-frame images arranged in time sequence in a corresponding divided region;
acquiring a sub-pulse wave signal of each sub-frame image sequence;
determining weight sequence information of the corresponding divided regions according to the sub-pulse wave signals; wherein the weight sequence information comprises a frequency domain weight sequence, a gradient weight sequence and a mask function representing the sub-frame image quality;
obtaining pulse wave signals of the plurality of frame images according to the weight sequence information and the sub pulse wave signals;
and determining a heart rate estimation value according to the frequency corresponding to the maximum power spectral density of the pulse wave signal in a preset time period.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
dividing the region of interest of each frame image in the plurality of frame images according to a preset dividing mode to obtain a plurality of subframe image sequences; wherein the plurality of frame images include a plurality of images arranged in time sequence, and each of the sub-frame image sequences includes a plurality of sub-frame images arranged in time sequence in a corresponding divided region;
acquiring a sub-pulse wave signal of each sub-frame image sequence;
determining weight sequence information of the corresponding divided regions according to the sub-pulse wave signals; wherein the weight sequence information comprises a frequency domain weight sequence, a gradient weight sequence and a mask function representing the sub-frame image quality;
obtaining pulse wave signals of the plurality of frame images according to the weight sequence information and the sub pulse wave signals;
and determining a heart rate estimation value according to the frequency corresponding to the maximum power spectral density of the pulse wave signal in a preset time period.
According to the heart rate information obtaining method, the heart rate information obtaining device, the computer equipment and the storage medium, the computer equipment obtains a plurality of sub-frame image sequences by dividing the region of interest of each frame image in the plurality of frame images according to a preset dividing mode, obtains the sub-pulse wave signals of each sub-frame image sequence, and then determines the weight sequence information of the corresponding divided region according to the sub-pulse wave signals. The pulse wave signals of the plurality of frame images are obtained according to the weight sequence information and the sub-pulse wave signals corresponding to different divided areas, so that the obtained pulse wave signals can be fused with the quality degrees of the sub-frame images of the different divided areas at different moments, and further, according to the frequency corresponding to the maximum power spectral density of the pulse wave signals in a preset time period, the determined heart rate estimation value can be estimated by fusing the quality degrees of the sub-frame images of the different divided areas at different moments, so that the estimation is more reasonable and accurate. In a real environment with sudden motion interference, the weight of a high-quality subframe image is increased based on a fusion strategy of subframe images of different division areas at different moments, so that the interference of facial motion and expression change is effectively processed, and the estimated heart rate estimated value is more accurate.
Drawings
FIG. 1 is a schematic flow chart of a method for acquiring heart rate information according to an embodiment;
FIG. 2 is a schematic flow chart of a heart rate information acquisition method in another embodiment;
FIG. 3 is a schematic flow chart of a heart rate information acquisition method in yet another embodiment;
FIG. 3a is a time-frequency diagram of a wavelet transform performed on a pulse wave signal according to an embodiment;
FIG. 4 is a schematic flow chart of a heart rate information acquisition method in yet another embodiment;
FIG. 5 is a flow chart illustrating a heart rate information obtaining method according to yet another embodiment;
FIG. 6 is a flow chart illustrating a heart rate information obtaining method according to yet another embodiment;
FIG. 7 is a flow chart illustrating a method for obtaining heart rate information according to yet another embodiment;
FIG. 8 is a flow chart illustrating a method for obtaining heart rate information according to yet another embodiment;
fig. 9 is a block diagram showing the structure of a heart rate information acquisition device in one embodiment;
FIG. 10 is a diagram showing an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The following describes the technical solutions of the present application and how to solve the above technical problems with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
It should be noted that the execution subject of the method embodiments described below may be a heart rate information obtaining apparatus, which may be implemented by software, hardware, or a combination of software and hardware as part or all of the above computer device. The following method embodiments are described by taking the execution subject as the computer device as an example.
Fig. 1 is a schematic flow chart of a heart rate information obtaining method according to an embodiment. The embodiment relates to a specific process. As shown in fig. 1, includes:
s10, dividing the region of interest of each frame image in the plurality of frame images according to a preset dividing mode to obtain a plurality of subframe image sequences; wherein the plurality of frame images include a plurality of images arranged in time sequence, and each of the sub-frame image sequences includes a plurality of sub-frame images arranged in time sequence in a corresponding divided region.
It should be noted that the computer device may acquire a plurality of frame images, may be a plurality of video frame images acquired from a video, such as a video frame image stored in a read memory, or receive a video frame image shot by a camera, and the plurality of frame images may be frame images in a video shot based on a human face, and thus, the plurality of frame images are images arranged in a time sequence.
Specifically, the computer device may perform face detection on each of the plurality of video frame images by using a detection algorithm to obtain a face roi of each frame image, for example, the image may be detected by using an artificial neural network model to obtain a region of interest (ROI) of a human face, or the region of interest of a human face is detected in a first frame by using a Viola-Jones face detector as an initial ROI, then using discriminant response image fitting algorithm to position 66 coordinates of the human face mark points in the ROI, including eyes, nose, mouth and face contour shape, then using Kanade-Lucas-Tomasi algorithm to track the human face mark points in the video frame image to obtain the ROI in each frame or using tracking algorithm, such as the Kanade-Lucas-Tomasi algorithm, estimates the ROI of the next frame image based on the detection result of the previous frame image. The ROI may be a rectangular region. Then, the computer device divides the ROI of each frame image according to a preset division mode, for example, the ROI of each frame image may be divided into a plurality of subframe images with a size of m × n, where m and n may be the same or different, and when m and n are the same, the subsequent processing is facilitated, and computer resources may be saved. The manner of dividing each frame image by the computer device may be the same, and therefore, the computer device takes a sequence of subframe images at the same position in each frame image in time sequence as a subframe image sequence, thereby obtaining a subframe image sequence corresponding to each divided region.
And S20, acquiring the sub-pulse wave signal of each sub-frame image sequence.
Specifically, the computer device may perform chrominance feature extraction on the subframe images in each subframe image sequence, may perform normalization based on different color channels, and obtain a sub-pulse wave signal representing the pixel average level of each subframe image based on a functional relationship of the normalized color channels.
S30, determining the weight sequence information of the corresponding divided areas according to the sub pulse wave signals; wherein the weight sequence information comprises a frequency domain weight sequence, a gradient weight sequence and a mask function representing the sub-frame image quality.
Specifically, since each sub-pulse wave signal represents the image state of the sub-frame image sequence of the corresponding divided region, the computer device may respectively determine the weight sequence information of the corresponding divided region according to the sub-pulse wave signal based on a wavelet time-frequency analysis method. It should be noted that, based on wavelet time-frequency transformation, the corresponding frequency domain weight sequence, gradient weight sequence and mask function are respectively determined according to the sub-pulse wave signals of each divided region, and then the frequency domain weight sequence, gradient weight sequence and mask function are combined together, for example, the values of the frequency domain weight, gradient weight and eye mask function at each moment are fused according to a certain functional relationship and arranged according to a time sequence, so as to obtain weight sequence information.
It should be noted that the mask function may be a function representing image qualities of different subframe images at different time instants, for example, if a frequency component corresponding to an i-th subframe image in a T-frame image is in the middle of a normal range, the mask function value may be determined to be 1, and if a frequency component corresponding to the i-th subframe image is out of the normal range, the mask function value may be determined to be 0, and if a frequency component corresponding to the i-th subframe image is at an edge of the normal range, the mask function value may be determined to be a number between 0 and 1, for example, 0.8, which is not limited in this embodiment. Through the mask function representing the sub-frame image instruction, the computer equipment can quantify the image change conditions of the sub-frame images corresponding to different divided areas at different moments according to the frequency components of the sub-frame images, so that the advantages and disadvantages of the images at different positions at different moments can be accurately positioned based on the mask function, and the quality of the sub-frame images at different positions and different moments in different regions can be accurately described.
And S40, obtaining the pulse wave signals of the plurality of frame images according to the weight sequence information and the sub pulse wave signals.
Specifically, the computer device can fuse the weight sequence information of each sub-frame image sequence and the sub-pulse wave signals through a certain functional relationship, so as to obtain the pulse wave signals representing the overall characteristics of the plurality of frame images.
And S50, determining a heart rate estimation value according to the frequency corresponding to the maximum power spectral density of the pulse wave signal in a preset time period.
Specifically, the computer device may obtain the pulse wave signal, perform frequency domain transformation, such as fast fourier transformation, to convert the pulse wave signal into a frequency domain, obtain a frequency corresponding to a maximum power spectral density from the frequency, and then perform heart rate estimation based on the frequency to obtain a psychometric estimation value.
In this embodiment, the computer device may divide the region of interest of each frame image in the plurality of frame images according to a preset dividing manner to obtain a plurality of subframe image sequences, obtain a sub-pulse wave signal of each subframe image sequence, and determine weight sequence information of the corresponding divided region according to the sub-pulse wave signal. The pulse wave signals of the plurality of frame images are obtained according to the weight sequence information and the sub-pulse wave signals corresponding to different divided areas, so that the obtained pulse wave signals can be fused with the quality degrees of the sub-frame images of the different divided areas at different moments, and further, according to the frequency corresponding to the maximum power spectral density of the pulse wave signals in a preset time period, the determined heart rate estimation value can be estimated by fusing the quality degrees of the sub-frame images of the different divided areas at different moments, so that the estimation is more reasonable and accurate. In a real environment with sudden motion interference, the weight of a high-quality subframe image is increased based on a fusion strategy of subframe images of different division areas at different moments, so that the interference of facial motion and expression change is effectively processed, and the estimated heart rate estimated value is more accurate.
Optionally, on the basis of the foregoing embodiment, a possible implementation manner of the foregoing step S20 may be as shown in fig. 2, and includes:
s21, extracting the chromaticity characteristics of the sub-frame images in the sub-frame image sequence to obtain the chromaticity characteristics of each sub-frame image, and forming the chromaticity characteristic sequence of the sub-frame image sequence.
Specifically, the computer device may perform chrominance feature extraction on each subframe image to obtain chrominance features of each subframe image, and then arrange the chrominance features of each subframe image corresponding to the same division region according to a time sequence to obtain a chrominance feature sequence of the subframe image sequence.
It should be noted that, the process of extracting the chrominance feature of each subframe image by the computer device may include the following steps: for a self-frame imageMay calculate the chrominance characteristic c-u of a pixelf-λvfWherein λ ═ σ (u)f)/σ(vf) And σ (u)f) Representing a signal ufStandard deviation of (d), σ (v)f) Representing a signal vfStandard deviation of (2). Wherein the signal ufSum signal vfThe color channel signal r can be normalized by the sub-frame image in three color channelsn、gn、bn. Converting the signal u to 3rn-2gnBand-pass filtering as signal ufSetting the signal v to 1.5rn+2gn-1.5bnBand-pass filtering as signal vf. The combination coefficient of the normalized color channel signals in the signals u and v can be calculated by adopting a skin tone standardization method.
And S22, obtaining the sub-pulse wave signals of the sub-frame image sequence according to the chrominance characteristics of the sub-frame images and the corresponding pixel numbers of the divided areas.
Specifically, the computer device may determine the sub-pulse wave signal corresponding to the sub-frame image sequence according to the chrominance feature of each sub-frame image in the sub-frame image sequence and the number of pixels of the corresponding division area.
Optionally, a ratio of a sum obtained by accumulating chrominance characteristics of each pixel point of the subframe image to a pixel number of the subframe image is used as a point pulse wave of a corresponding moment of the subframe image to form the sub-pulse wave signal corresponding to the subframe image sequence in which the subframe image is located. Specifically, taking the kth division area as an example, the chroma characteristic of the ith pixel point of the k division area is ck,iIndicating that the chroma feature sequence corresponding to the divided region can be represented by ck,i(t), in this case, the computer device can use the formula
Figure BDA0002374613050000091
Or determining the sub-pulse wave signal h corresponding to the sub-frame image sequence by the deformation of the formulak(t) of (d). Wherein K is [1, K ]]Wherein K represents the number of sub-frame images divided by each frame image, and N representsThe number of pixels per sub-frame image. Using a formula
Figure BDA0002374613050000092
The determination of the sub-pulse wave signals corresponding to the sub-frame image sequence is simpler and more accurate.
In this embodiment, the computer device performs chrominance feature extraction on the subframe images in the subframe image sequence to obtain the chrominance feature of each subframe image, so as to form a chrominance feature sequence of the subframe image sequence, and then obtains the sub-pulse wave signal of the subframe image sequence according to the chrominance feature of the subframe images and the number of pixels of the corresponding division area, thereby implementing a process of obtaining the sub-pulse wave signal based on the chrominance feature of the subframe images. In the method, the chrominance characteristics can accurately represent the chrominance changes such as brightness, brightness and the like of the image, so that the chrominance characteristic extraction based on the subframe image can acquire the chrominance changes aiming at different divided regions, and can accurately extract the chrominance of the corresponding divided regions when the chrominance characteristic extraction is applied to face recognition under the condition of motion or expression change, thereby enabling the sub-pulse wave signals to more accurately represent the change condition of the local region of the face and further ensuring the accuracy of heart rate estimation.
Optionally, on the basis of the foregoing embodiments, the step S30 may specifically include a process of acquiring mask functions corresponding to different divided regions, as shown in fig. 3, where the process includes:
s311, performing wavelet transformation on the chrominance characteristic sequence of the subframe image sequence to obtain a wavelet transformation coefficient of the subframe image.
Specifically, the computer device performs wavelet transform on the chrominance characteristic sequence of the subframe image sequence to obtain a wavelet transform coefficient WT of the subframe imagek(a, b), optionally, a formula may be employed
Figure BDA0002374613050000101
Or a variation of this equation. Wherein the wavelet basis function psi*(t) is obtained by scaling and shifting the wavelet mother function ψ (t), and optionally, a formula
Figure BDA0002374613050000102
Or a variation of this formula, where a is the scaling factor and a ∈ R+*B is a translation factor, and b ∈ R, wherein R represents a real number field, R+*For positive real number domain, optionally, a time-frequency diagram of the pulse wave signal subjected to wavelet transform can be seen in fig. 3 a.
S312, obtaining the mask function corresponding to each sub-frame image sequence according to the maximum frequency of the wavelet transform coefficients of the sub-frame images in the frequency domain and the preset normal heart rate frequency range.
Specifically, the computer device may obtain the mask function corresponding to each sub-frame image sequence according to an affiliated relationship between a maximum frequency of a wavelet transform coefficient of a frame image in a frequency domain and a preset normal heart rate frequency range. Alternatively, the computer device may use a binary mask function that generates a time-varying function to select good quality sub-frame images in each frame image. If the frequency component corresponding to the wavelet transform coefficient of the subframe image is located in the preset frequency range, the subframe image is determined to be a high-quality image, and the value of the binary mask function is 1 at this time, and if the frequency component corresponding to the wavelet transform coefficient of the frame image is located outside the frequency range, the subframe image is determined to be a low-quality image, and the value of the binary mask function is 0 at this time. In addition, the frequency range [ f ]l,fh]A normal physiological parameter, such as heart rate, corresponds to a normal range of frequencies.
Alternatively, the computer device may use a formula
Figure BDA0002374613050000111
Or a modification of the formula representing the mask function mk(t) of (d). Wherein f ismax,k(t) represents the maximum frequency component of the wavelet transform coefficient of the kth sub-frame image of the tth frame converted in the frequency domain.
In this embodiment, the computer device performs wavelet transform on the chrominance feature sequence of the subframe image sequence to obtain a wavelet transform coefficient of the subframe image, and obtains a mask function corresponding to each subframe image sequence according to the belonged relationship between the maximum frequency of the wavelet transform coefficient of the subframe image in the frequency domain and the preset normal heart rate frequency range, so that the selection of a high-quality subframe image can be realized based on the frequency domain characteristics of the wavelet transform in the frequency domain. The mask function obtained by the method can distinguish the advantages and disadvantages of the subframe images of different divided regions at different moments, so that the determined weight sequence information can be based on the advantages and disadvantages of the subframe images corresponding to the different divided regions at different moments, the information of the frame images is more accurately expressed, and further the heart rate estimation is more reasonable and accurate.
Optionally, on the basis of the foregoing embodiments, the step S30 may further include an obtaining process of the frequency domain weight sequences corresponding to different divided regions, as shown in fig. 4, where the obtaining process includes:
s321, performing wavelet transformation on the chrominance characteristic sequence of the subframe image sequence to obtain a wavelet transformation coefficient of the subframe image.
And S322, acquiring the maximum amplitude and the second maximum amplitude of the wavelet transform coefficient of the sub-frame image at each moment in the frequency domain.
S323, obtaining the frequency domain weight of each moment by the ratio of the maximum amplitude and the secondary large amplitude of the wavelet transform coefficient of the sub-frame image of each moment to the secondary large amplitude, so as to form the frequency domain weight sequence corresponding to each sub-frame image sequence.
It should be noted that the result of continuous wavelet transform of the sub-pulse wave signal can be used to estimate the quality of the sub-frame image. For each sub-frame image in the same frame image, a frequency spectrum with different wavelet transform coefficients can be obtained from the continuous wavelet transform result, wherein the frequency with the highest peak is regarded as the frequency component with the largest heartbeat frequency, and other frequencies with non-zero amplitude are generally caused by noise. That is, higher amplitudes of other frequencies indicate more noise components, which provides lower confidence for fusing the estimated pulse wave signals. Therefore, we use the maximum amplitude and the sub-maximum amplitude of the wavelet transform coefficients to determine the frequency domain weights.
Specifically, the computer device may acquire the wavelet transform coefficient of the subframe image by using the manner of S311, which is not described in detail in this embodiment. Then, the computer device frequency-transforms the wavelet transform coefficients of the sub-frame image at each of the above-mentioned time instants, and finds the maximum amplitude A of the frequency domaink(t) and a second largest amplitude Bk(t) of (d). Maximum amplitude A of computer equipmentk(t) and a second largest amplitude Bk(t) difference from said sub-maximum amplitude Bk(t) ratio to obtain the frequency domain weight at each moment
Figure BDA0002374613050000121
And forming the frequency domain weight sequence corresponding to each subframe image sequence by the frequency domain weight of each moment of one divided region according to the time sequence. Alternatively, the computer device may employ a formula
Figure BDA0002374613050000122
Or a variant of this formula results in a sequence of frequency domain weights.
In this embodiment, the computer device performs wavelet transform on the chrominance feature sequence of the subframe image sequence to obtain a wavelet transform coefficient of the subframe image, obtains a maximum amplitude and a second maximum amplitude of the wavelet transform coefficient of the subframe image at each time in the frequency domain, and obtains a frequency domain weight at each time by using a ratio of a difference between the maximum amplitude and the second maximum amplitude of the wavelet transform coefficient of the subframe image at each time to the second maximum amplitude to form a frequency domain weight sequence corresponding to each subframe image sequence. According to the method, the frame image can be partitioned to obtain the frequency domain weights corresponding to different transform areas, so that the obtained frequency domain weights are more convenient to determine the corresponding frequency domain weights respectively by judging the quality degrees of the subframe images corresponding to the different partition areas, the described weight information is more accurate and reasonable, the information of the frame image is more accurately expressed, and further the heart rate estimation is more reasonable and accurate.
Optionally, on the basis of the foregoing embodiments, the step S30 may further include an obtaining process of a gradient weight sequence corresponding to different divided regions, as shown in fig. 5, where the obtaining process includes:
s331, solving an average gradient of the sub-frame images in the sub-frame image sequence to obtain an average gradient sequence of each sub-frame image sequence.
S332, obtaining the gradient weight sequence corresponding to each subframe image sequence according to the average gradient sequence and a preset adjusting hyper-parameter.
It should be noted that the quality of the sub-frame image in the ROI also depends on the local structural uniformity, the potential sub-pulse wave signals in the sub-frame image with low uniformity are often destroyed by the local structural changes (e.g. hair, skin wrinkles, shadows, etc.), some non-skin regions (e.g. nostrils, clothes, background, etc.) in the sub-frame image do not even contain any pulse wave signals, so the structural uniformity of a given sub-frame image can be measured by calculating the average gradient.
In particular, the computer device may be a bulletin
Figure BDA0002374613050000131
Or a variation of this formula yields a sequence of gradient weights, wherein
Figure BDA0002374613050000133
The mean absolute gradient of the sub-frame image k at the time t, namely the image of the t frame, can be obtained by deriving pixels, wherein α is a regulation hyperparameter which can be set according to experience, and α can influence the size range of gradient weights
Figure BDA0002374613050000132
The change degree of the model can be enhanced by adopting an inverse exponential model, so that the influence of the gradient weight is more prominent, and the face surface is further enabled to beThe subtle changes of the conditions can be more obviously reflected, and the accuracy of heart rate estimation is further enhanced.
In this embodiment, the average gradient of the subframe images in the subframe image sequence is solved to obtain an average gradient sequence of each subframe image sequence, and a gradient weight sequence corresponding to each subframe image sequence is obtained according to the average gradient sequence and a preset adjustment hyper-parameter. According to the method, the frame image can be partitioned to obtain the gradient weights corresponding to different transformation areas, so that the obtained gradient weights are more convenient to determine the corresponding gradient weights respectively by judging the quality degrees of the subframe images corresponding to the different partition areas, the described weight information is more accurate and reasonable, the information of the frame image is more accurately expressed, and further the heart rate estimation is more reasonable and accurate.
Optionally, on the basis of the foregoing embodiments, a possible implementation manner of the step S40 may also be as shown in fig. 6, and includes:
s41, multiplying the frequency domain weight sequence, the gradient weight sequence and the mask function corresponding to the subframe image sequence according to time to obtain the comprehensive weight of the subframe image sequence.
And S42, accumulating the products of the comprehensive weight of each sub-frame image sequence and the corresponding sub-pulse wave signals, and comparing the products with the sum of the comprehensive weights of each sub-frame image sequence to obtain the pulse wave signals of the plurality of frame images.
Specifically, the computer device multiplies the frequency domain weight sequence, the gradient weight sequence and the mask function corresponding to the subframe image sequence according to the time to obtain the comprehensive weight of the subframe image sequence. May be by the formula
Figure BDA0002374613050000141
Or the deformation of the formula obtains the comprehensive weight w of the subframe image sequencek(t) of (d). Then, the computer device accumulates the product of the integrated weight of each sub-frame image sequence and the corresponding sub-pulse wave signal, and the ratio of the integrated weight of each sub-frame image sequence to the sum of the integrated weights of each sub-frame image sequenceAnd obtaining pulse wave signals of a plurality of frame images. May be by the formula
Figure BDA0002374613050000142
Or a modification of this formula results in the pulse wave signal s (t).
In this embodiment, the computer device multiplies the frequency domain weight sequence, the gradient weight sequence and the mask function corresponding to the subframe image sequence according to the time to obtain the integrated weight of the subframe image sequence, and then accumulates the product based on the integrated weight of each subframe image sequence and the corresponding sub-pulse wave signal, and compares the accumulated product with the sum of the integrated weights of each subframe image sequence, so that the obtained pulse wave signal can fuse the goodness and badness of the subframe image of different divisional areas at different times and fuse different frequency domain weights and gradient weights corresponding to the subframe images of different divisional areas, thereby making the heart rate estimation value more reasonable and accurate. In a real environment with sudden motion interference, the weight of a high-quality subframe image is increased based on a fusion strategy of subframe images of different division areas at different moments, so that the interference of facial motion and expression change is effectively processed, and the estimated heart rate estimated value is more accurate.
Optionally, on the basis of the foregoing embodiments, a possible implementation manner of the step S50 may also be as shown in fig. 7, and includes:
and S51, filtering and frequency domain converting the pulse wave signal, and taking the frequency corresponding to the obtained maximum power spectral density as a target frequency.
And S52, determining the preset multiple of the target frequency as the heart rate estimation value.
In particular, the computer device may employ de-trend filtering to reduce slow non-stationary trends in the signal using a frequency range of [ f ]l,fh]The band-pass filter is used for filtering stray waves to obtain a filtered pulse wave signal, and then the filtered pulse wave signal adopts fast Fourier transformTransforming and frequency converting, and taking the frequency corresponding to the maximum power spectral density in the obtained frequency domain, namely the frequency with the highest amplitude as the target frequency fHRThen a preset multiple of the target frequency, for example 60, is determined as the heart rate estimate HR. Alternatively, the formula HR 60f may be usedHROr a variation of this equation to derive the heart rate estimate. Alternatively, the preset multiple may be another value, such as 61, 62, or 58, 59, and the like, which is not limited in this embodiment.
In this embodiment, computer equipment carries out the filtering with pulse wave signal, can filter the clutter influence, has ensured the accuracy of rhythm of the heart estimated value, through frequency domain conversion, regards the frequency that the maximum power spectral density that obtains corresponds as the target frequency, then the preset multiple of target frequency confirms as rhythm of the heart estimated value, has realized pulse wave signal and rhythm of the heart estimated value and is worth converting for the expression of estimation result is abundanter, the user of being convenient for discerns.
In order to express the embodiments of the present application more clearly, the following description is made with reference to a flowchart shown in fig. 8. As shown in fig. 8, the computer device performs face detection and tracking on a video frame image to obtain an ROI, then divides the video frame image into a plurality of sub-frame images, performs chrominance feature extraction to obtain sub-pulse wave signals of different divided regions, and then performs continuous wavelet transform on the sub-pulse wave signals to obtain a wavelet transform coefficient and a binary mask function. The computer equipment further obtains a frequency domain weight sequence and a gradient weight sequence based on the sub-pulse wave signals, and finally fuses the frequency domain weight sequence, the gradient weight sequence and the mask function to obtain a comprehensive gradient, and further obtains the pulse wave signals according to the comprehensive gradient. And finally, performing trend filtering, band-pass filtering and frequency transformation on the pulse wave signals, thereby realizing the estimation of the heart rate estimated value based on wavelet time-frequency analysis.
It should be understood that although the various steps in the flow charts of fig. 1-8 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1-8 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.
In one embodiment, as shown in fig. 9, there is provided a heart rate information acquiring apparatus including:
the dividing module 100 is configured to divide an interesting region of each frame image in the plurality of frame images according to a preset dividing manner to obtain a plurality of subframe image sequences; wherein the plurality of frame images include a plurality of images arranged in time sequence, and each of the sub-frame image sequences includes a plurality of sub-frame images arranged in time sequence in a corresponding divided region;
a processing module 200, configured to obtain a sub-pulse wave signal of each sub-frame image sequence, determine weight sequence information of the corresponding divided area according to the sub-pulse wave signal, obtain pulse wave signals of the plurality of frame images according to the weight sequence information and the sub-pulse wave signal, and determine a heart rate estimation value according to a frequency corresponding to a maximum power spectral density of the pulse wave signals within a preset time period; wherein the weight sequence information comprises a frequency domain weight sequence, a gradient weight sequence and a mask function representing the sub-frame image quality.
In an embodiment, the processing module 200 is specifically configured to perform chrominance feature extraction on the subframe images in the subframe image sequence to obtain a chrominance feature of each subframe image, so as to form a chrominance feature sequence of the subframe image sequence; and obtaining a sub-pulse wave signal of the sub-frame image sequence according to the chrominance characteristics of the sub-frame image and the corresponding pixel number of the divided area.
In an embodiment, the processing module 200 is specifically configured to use a ratio of a sum obtained by accumulating chrominance features of each pixel point of the subframe image and a pixel number of the subframe image as a point pulse wave of a corresponding time of the subframe image, so as to form the sub-pulse wave signal corresponding to the subframe image sequence in which the subframe image is located.
In an embodiment, the processing module 200 is specifically configured to perform wavelet transform on the chrominance feature sequence of the subframe image sequence to obtain a wavelet transform coefficient of the subframe image; and obtaining the mask function corresponding to each sub-frame image sequence according to the belonged relationship between the maximum frequency of the wavelet transform coefficients of the sub-frame images in the frequency domain and a preset normal heart rate frequency range.
In an embodiment, the processing module 200 is specifically configured to perform wavelet transform on the chrominance feature sequence of the subframe image sequence to obtain a wavelet transform coefficient of the subframe image; acquiring the maximum amplitude and the second maximum amplitude of the wavelet transform coefficient of the subframe image at each moment in a frequency domain; and obtaining the frequency domain weight of each moment by the ratio of the difference between the maximum amplitude and the secondary maximum amplitude of the wavelet transform coefficient of the subframe image of each moment to the secondary maximum amplitude so as to form the frequency domain weight sequence corresponding to each subframe image sequence.
In an embodiment, the processing module 200 is specifically configured to solve an average gradient of the subframe images in the subframe image sequence to obtain an average gradient sequence of each subframe image sequence; and obtaining the gradient weight sequence corresponding to each sub-frame image sequence according to the average gradient sequence and a preset adjusting hyper-parameter.
In an embodiment, the processing module 200 is specifically configured to multiply the frequency domain weight sequence, the gradient weight sequence, and the mask function corresponding to the subframe image sequence according to time to obtain a comprehensive weight of the subframe image sequence; and accumulating the products of the comprehensive weight of each sub-frame image sequence and the corresponding sub-pulse wave signals, and obtaining the pulse wave signals of the plurality of frame images according to the ratio of the comprehensive weight sum of each sub-frame image sequence.
In an embodiment, the processing module 200 is specifically configured to perform filtering and frequency domain conversion on the pulse wave signal, and use a frequency corresponding to the obtained maximum power spectral density as a target frequency; and determining the preset multiple of the target frequency as the heart rate estimated value.
For specific limitations of the heart rate information acquisition device, reference may be made to the above limitations of the heart rate information acquisition method, which are not described herein again. The modules in the heart rate information acquisition device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 10. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is for storing a plurality of frame images. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a heart rate information acquisition method.
Those skilled in the art will appreciate that the architecture shown in fig. 10 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
dividing the region of interest of each frame image in the plurality of frame images according to a preset dividing mode to obtain a plurality of subframe image sequences; wherein the plurality of frame images include a plurality of images arranged in time sequence, and each of the sub-frame image sequences includes a plurality of sub-frame images arranged in time sequence in a corresponding divided region;
acquiring a sub-pulse wave signal of each sub-frame image sequence;
determining weight sequence information of the corresponding divided regions according to the sub-pulse wave signals; wherein the weight sequence information comprises a frequency domain weight sequence, a gradient weight sequence and a mask function representing the sub-frame image quality;
obtaining pulse wave signals of the plurality of frame images according to the weight sequence information and the sub pulse wave signals;
and determining a heart rate estimation value according to the frequency corresponding to the maximum power spectral density of the pulse wave signal in a preset time period.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
carrying out chrominance feature extraction on the subframe images in the subframe image sequence to obtain the chrominance feature of each subframe image and form a chrominance feature sequence of the subframe image sequence;
and obtaining a sub-pulse wave signal of the sub-frame image sequence according to the chrominance characteristics of the sub-frame image and the corresponding pixel number of the divided area.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
and taking the ratio of the sum obtained by accumulating the chrominance characteristics of each pixel point of the subframe image to the pixel number of the subframe image as the point pulse wave of the corresponding moment of the subframe image so as to form the sub pulse wave signal corresponding to the subframe image sequence in which the subframe image is positioned.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
performing wavelet transformation on the chrominance characteristic sequence of the subframe image sequence to obtain a wavelet transformation coefficient of the subframe image;
and obtaining the mask function corresponding to each sub-frame image sequence according to the belonged relationship between the maximum frequency of the wavelet transform coefficients of the sub-frame images in the frequency domain and a preset normal heart rate frequency range.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
performing wavelet transformation on the chrominance characteristic sequence of the subframe image sequence to obtain a wavelet transformation coefficient of the subframe image;
acquiring the maximum amplitude and the second maximum amplitude of the wavelet transform coefficient of the subframe image at each moment in a frequency domain;
and obtaining the frequency domain weight of each moment by the ratio of the difference between the maximum amplitude and the secondary maximum amplitude of the wavelet transform coefficient of the subframe image of each moment to the secondary maximum amplitude so as to form the frequency domain weight sequence corresponding to each subframe image sequence.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
solving an average gradient of the sub-frame images in the sub-frame image sequence to obtain an average gradient sequence of each sub-frame image sequence;
and obtaining the gradient weight sequence corresponding to each sub-frame image sequence according to the average gradient sequence and a preset adjusting hyper-parameter.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
multiplying the frequency domain weight sequence, the gradient weight sequence and the mask function corresponding to the subframe image sequence according to the time correspondence to obtain the comprehensive weight of the subframe image sequence;
and accumulating the products of the comprehensive weight of each sub-frame image sequence and the corresponding sub-pulse wave signals, and obtaining the pulse wave signals of the plurality of frame images according to the ratio of the comprehensive weight sum of each sub-frame image sequence.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
filtering and frequency domain conversion are carried out on the pulse wave signals, and the frequency corresponding to the obtained maximum power spectral density is taken as a target frequency;
and determining the preset multiple of the target frequency as the heart rate estimated value.
It should be clear that, in the embodiments of the present application, the process of executing the computer program by the processor is consistent with the process of executing the steps in the above method, and specific reference may be made to the description above.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
dividing the region of interest of each frame image in the plurality of frame images according to a preset dividing mode to obtain a plurality of subframe image sequences; wherein the plurality of frame images include a plurality of images arranged in time sequence, and each of the sub-frame image sequences includes a plurality of sub-frame images arranged in time sequence in a corresponding divided region;
acquiring a sub-pulse wave signal of each sub-frame image sequence;
determining weight sequence information of the corresponding divided regions according to the sub-pulse wave signals; wherein the weight sequence information comprises a frequency domain weight sequence, a gradient weight sequence and a mask function representing the sub-frame image quality;
obtaining pulse wave signals of the plurality of frame images according to the weight sequence information and the sub pulse wave signals;
and determining a heart rate estimation value according to the frequency corresponding to the maximum power spectral density of the pulse wave signal in a preset time period.
In one embodiment, the computer program when executed by the processor further performs the steps of:
carrying out chrominance feature extraction on the subframe images in the subframe image sequence to obtain the chrominance feature of each subframe image and form a chrominance feature sequence of the subframe image sequence;
and obtaining a sub-pulse wave signal of the sub-frame image sequence according to the chrominance characteristics of the sub-frame image and the corresponding pixel number of the divided area.
In one embodiment, the computer program when executed by the processor further performs the steps of:
and taking the ratio of the sum obtained by accumulating the chrominance characteristics of each pixel point of the subframe image to the pixel number of the subframe image as the point pulse wave of the corresponding moment of the subframe image so as to form the sub pulse wave signal corresponding to the subframe image sequence in which the subframe image is positioned.
In one embodiment, the computer program when executed by the processor further performs the steps of:
performing wavelet transformation on the chrominance characteristic sequence of the subframe image sequence to obtain a wavelet transformation coefficient of the subframe image;
and obtaining the mask function corresponding to each sub-frame image sequence according to the belonged relationship between the maximum frequency of the wavelet transform coefficients of the sub-frame images in the frequency domain and a preset normal heart rate frequency range.
In one embodiment, the computer program when executed by the processor further performs the steps of:
performing wavelet transformation on the chrominance characteristic sequence of the subframe image sequence to obtain a wavelet transformation coefficient of the subframe image;
acquiring the maximum amplitude and the second maximum amplitude of the wavelet transform coefficient of the subframe image at each moment in a frequency domain;
and obtaining the frequency domain weight of each moment by the ratio of the difference between the maximum amplitude and the secondary maximum amplitude of the wavelet transform coefficient of the subframe image of each moment to the secondary maximum amplitude so as to form the frequency domain weight sequence corresponding to each subframe image sequence.
In one embodiment, the computer program when executed by the processor further performs the steps of:
solving an average gradient of the sub-frame images in the sub-frame image sequence to obtain an average gradient sequence of each sub-frame image sequence;
and obtaining the gradient weight sequence corresponding to each sub-frame image sequence according to the average gradient sequence and a preset adjusting hyper-parameter.
In one embodiment, the computer program when executed by the processor further performs the steps of:
multiplying the frequency domain weight sequence, the gradient weight sequence and the mask function corresponding to the subframe image sequence according to the time correspondence to obtain the comprehensive weight of the subframe image sequence;
and accumulating the products of the comprehensive weight of each sub-frame image sequence and the corresponding sub-pulse wave signals, and obtaining the pulse wave signals of the plurality of frame images according to the ratio of the comprehensive weight sum of each sub-frame image sequence.
In one embodiment, the computer program when executed by the processor further performs the steps of:
filtering and frequency domain conversion are carried out on the pulse wave signals, and the frequency corresponding to the obtained maximum power spectral density is taken as a target frequency;
and determining the preset multiple of the target frequency as the heart rate estimated value.
It should be clear that, in the embodiments of the present application, the process executed by the processor by the computer program is consistent with the execution process of each step in the above method, and specific reference may be made to the description above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (11)

1. A heart rate information acquisition method, characterized in that the method comprises:
dividing the region of interest of each frame image in the plurality of frame images according to a preset dividing mode to obtain a plurality of subframe image sequences; wherein the plurality of frame images include a plurality of images arranged in time sequence, and each of the sub-frame image sequences includes a plurality of sub-frame images arranged in time sequence in a corresponding divided region;
acquiring a sub-pulse wave signal of each sub-frame image sequence;
determining weight sequence information of the corresponding divided regions according to the sub-pulse wave signals; wherein the weight sequence information comprises a frequency domain weight sequence, a gradient weight sequence and a mask function representing the sub-frame image quality;
obtaining pulse wave signals of the plurality of frame images according to the weight sequence information and the sub pulse wave signals;
and determining a heart rate estimation value according to the frequency corresponding to the maximum power spectral density of the pulse wave signal in a preset time period.
2. The method of claim 1, wherein said acquiring a sub-pulse wave signal for each of said sequence of sub-frame images comprises:
carrying out chrominance feature extraction on the subframe images in the subframe image sequence to obtain the chrominance feature of each subframe image and form a chrominance feature sequence of the subframe image sequence;
and obtaining a sub-pulse wave signal of the sub-frame image sequence according to the chrominance characteristics of the sub-frame image and the corresponding pixel number of the divided area.
3. The method according to claim 2, wherein said deriving sub-pulse wave signals of said sequence of sub-frame images from chrominance features of said sub-frame images and corresponding number of pixels of said divided regions comprises:
and taking the ratio of the sum obtained by accumulating the chrominance characteristics of each pixel point of the subframe image to the pixel number of the subframe image as the point pulse wave of the corresponding moment of the subframe image so as to form the sub pulse wave signal corresponding to the subframe image sequence in which the subframe image is positioned.
4. The method according to claim 1, wherein said determining weight sequence information of the corresponding divided regions from the sub-pulse wave signals comprises:
performing wavelet transformation on the chrominance characteristic sequence of the subframe image sequence to obtain a wavelet transformation coefficient of the subframe image;
and obtaining the mask function corresponding to each sub-frame image sequence according to the belonged relationship between the maximum frequency of the wavelet transform coefficients of the sub-frame images in the frequency domain and a preset normal heart rate frequency range.
5. The method according to claim 1, wherein said determining weight sequence information of the corresponding divided regions from the sub-pulse wave signals comprises:
performing wavelet transformation on the chrominance characteristic sequence of the subframe image sequence to obtain a wavelet transformation coefficient of the subframe image;
acquiring the maximum amplitude and the second maximum amplitude of the wavelet transform coefficient of the subframe image at each moment in a frequency domain;
and obtaining the frequency domain weight of each moment by the ratio of the difference between the maximum amplitude and the secondary maximum amplitude of the wavelet transform coefficient of the subframe image of each moment to the secondary maximum amplitude so as to form the frequency domain weight sequence corresponding to each subframe image sequence.
6. The method according to claim 1, wherein said determining weight sequence information of the corresponding divided regions from the sub-pulse wave signals comprises:
solving an average gradient of the sub-frame images in the sub-frame image sequence to obtain an average gradient sequence of each sub-frame image sequence;
and obtaining the gradient weight sequence corresponding to each sub-frame image sequence according to the average gradient sequence and a preset adjusting hyper-parameter.
7. The method according to any one of claims 1 to 6, wherein obtaining the pulse wave signals of the plurality of frame images according to the weight sequence information and the sub-pulse wave signals comprises:
multiplying the frequency domain weight sequence, the gradient weight sequence and the mask function corresponding to the subframe image sequence according to the time correspondence to obtain the comprehensive weight of the subframe image sequence;
and accumulating the products of the comprehensive weight of each sub-frame image sequence and the corresponding sub-pulse wave signals, and obtaining the pulse wave signals of the plurality of frame images according to the ratio of the comprehensive weight sum of each sub-frame image sequence.
8. The method according to any one of claims 1 to 6, wherein the determining the heart rate estimate from the frequency corresponding to the maximum power spectral density of the pulse wave signal within a preset time period comprises:
filtering and frequency domain conversion are carried out on the pulse wave signals, and the frequency corresponding to the obtained maximum power spectral density is taken as a target frequency;
and determining the preset multiple of the target frequency as the heart rate estimated value.
9. A heart rate information acquisition apparatus, characterized in that the apparatus comprises:
the dividing module is used for dividing the region of interest of each frame image in the plurality of frame images according to a preset dividing mode to obtain a plurality of subframe image sequences; wherein the plurality of frame images include a plurality of images arranged in time sequence, and each of the sub-frame image sequences includes a plurality of sub-frame images arranged in time sequence in a corresponding divided region;
the processing module is used for acquiring a sub-pulse wave signal of each sub-frame image sequence, determining weight sequence information of the corresponding divided area according to the sub-pulse wave signal, obtaining pulse wave signals of the plurality of frame images according to the weight sequence information and the sub-pulse wave signal, and determining a heart rate estimation value according to a frequency corresponding to the maximum power spectral density of the pulse wave signals in a preset time period; wherein the weight sequence information comprises a frequency domain weight sequence, a gradient weight sequence and a mask function representing the sub-frame image quality.
10. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 8 when executing the computer program.
11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 8.
CN202010061378.8A 2020-01-19 2020-01-19 Heart rate information acquisition method and device, computer equipment and storage medium Pending CN111310584A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010061378.8A CN111310584A (en) 2020-01-19 2020-01-19 Heart rate information acquisition method and device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010061378.8A CN111310584A (en) 2020-01-19 2020-01-19 Heart rate information acquisition method and device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
CN111310584A true CN111310584A (en) 2020-06-19

Family

ID=71158266

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010061378.8A Pending CN111310584A (en) 2020-01-19 2020-01-19 Heart rate information acquisition method and device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN111310584A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111938622A (en) * 2020-07-16 2020-11-17 启航汽车有限公司 Heart rate detection method, device and system and readable storage medium
CN112315440A (en) * 2020-10-26 2021-02-05 青岛歌尔智能传感器有限公司 Heart rate detection method, wearable device and readable storage medium
CN112733791A (en) * 2021-01-21 2021-04-30 展讯通信（上海）有限公司 Living body detection method and device, storage medium and terminal
CN113995387A (en) * 2021-10-28 2022-02-01 上海掌门科技有限公司 Method, apparatus, medium, and program product for detecting pulse waveform

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105654031A (en) * 2014-10-22 2016-06-08 通用汽车环球科技运作有限责任公司 Systems and methods for object detection
CN105989357A (en) * 2016-01-18 2016-10-05 合肥工业大学 Human face video processing-based heart rate detection method
CN109480808A (en) * 2018-09-27 2019-03-19 深圳市君利信达科技有限公司 A kind of heart rate detection method based on PPG, system, equipment and storage medium
WO2019203106A1 (en) * 2018-04-17 2019-10-24 Nec Corporation Pulse rate estimation apparatus, pulse rate estimation method, and computer-readable storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105654031A (en) * 2014-10-22 2016-06-08 通用汽车环球科技运作有限责任公司 Systems and methods for object detection
CN105989357A (en) * 2016-01-18 2016-10-05 合肥工业大学 Human face video processing-based heart rate detection method
WO2019203106A1 (en) * 2018-04-17 2019-10-24 Nec Corporation Pulse rate estimation apparatus, pulse rate estimation method, and computer-readable storage medium
CN109480808A (en) * 2018-09-27 2019-03-19 深圳市君利信达科技有限公司 A kind of heart rate detection method based on PPG, system, equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHAO YANG等: "Motion-resistant heart rate measurement from face videos using patch-based fusion", 《SIGNAL, IMAGE AND VIDEO PROCESSING》 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111938622A (en) * 2020-07-16 2020-11-17 启航汽车有限公司 Heart rate detection method, device and system and readable storage medium
CN111938622B (en) * 2020-07-16 2022-08-30 启航汽车有限公司 Heart rate detection method, device and system and readable storage medium
CN112315440A (en) * 2020-10-26 2021-02-05 青岛歌尔智能传感器有限公司 Heart rate detection method, wearable device and readable storage medium
CN112733791A (en) * 2021-01-21 2021-04-30 展讯通信（上海）有限公司 Living body detection method and device, storage medium and terminal
CN112733791B (en) * 2021-01-21 2022-09-30 展讯通信（上海）有限公司 Living body detection method and device, storage medium and terminal
CN113995387A (en) * 2021-10-28 2022-02-01 上海掌门科技有限公司 Method, apparatus, medium, and program product for detecting pulse waveform
CN113995387B (en) * 2021-10-28 2024-04-12 上海掌门科技有限公司 Method, device, medium and program product for detecting pulse waveform

Similar Documents

Publication Publication Date Title
CN111310584A (en) Heart rate information acquisition method and device, computer equipment and storage medium
Bobbia et al. Unsupervised skin tissue segmentation for remote photoplethysmography
CN110269600B (en) Non-contact video heart rate detection method based on multivariate empirical mode decomposition and combined blind source separation
JP6957929B2 (en) Pulse wave detector, pulse wave detection method, and program
CN107635457B (en) Identifying living skin tissue in a video sequence
CN112784773B (en) Image processing method and device, storage medium and terminal
CN110706826B (en) Non-contact real-time multi-person heart rate and blood pressure measuring method based on video image
CN111134650A (en) Heart rate information acquisition method and device, computer equipment and storage medium
CN111938622B (en) Heart rate detection method, device and system and readable storage medium
CN111513701A (en) Heart rate detection method and device, computer equipment and readable storage medium
CN117237591A (en) Intelligent removal method for heart ultrasonic image artifacts
CN112069986A (en) Machine vision tracking method and device for eye movements of old people
CN117036310A (en) DICOM image peripheral outline identification and extraction method
CN112102348A (en) Image processing apparatus
Birla et al. ALPINE: Improving remote heart rate estimation using contrastive learning
CN110321781B (en) Signal processing method and device for non-contact measurement
CN115153473B (en) Non-contact heart rate detection method based on multivariate singular spectrum analysis
CN115187507A (en) Heart rate and respiratory rate detection system and method based on thermal imaging technology
He et al. Graph-based clustering for identifying region of interest in eye tracker data analysis
CN112507877A (en) System and method for detecting heart rate under condition of partial video information loss
Rios et al. Parametric Study of Performance of Remote Photopletysmography System
CN117710242B (en) Non-contact physiological parameter extraction method for resisting illumination and motion interference
Li et al. ECG signal reconstruction based on facial videos via combined explicit and implicit supervision
CN113408406B (en) Sight tracking method and system
Kurihara et al. Blood volume pulse signal extraction based on spatio-temporal low-rank approximation for heart rate estimation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200619

RJ01 Rejection of invention patent application after publication