WO2023061042A1

WO2023061042A1 - Heart rate measurement method and apparatus, and electronic device and storage medium

Info

Publication number: WO2023061042A1
Application number: PCT/CN2022/113619
Authority: WO
Inventors: 李若岱; 葛学人; 李泉录; 陈朝军; 马堃
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-10-15
Filing date: 2022-08-19
Publication date: 2023-04-20
Also published as: CN114140616A

Abstract

The present disclosure relates to a heart rate measurement method and apparatus, and an electronic device and a storage medium. The method comprises: determining a thermal-imaged image sequence that comprises a plurality of thermal-imaged images, which are sequentially collected, wherein the thermal-imaged images are images for representing a temperature distribution in a target region of a human body; determining a feature value of a region, in which a vessel is located, in at least one thermal-imaged image, so as to determine a temperature change signal according to a positional order and a corresponding feature value, and obtain a temperature change of a vessel in the target region of the human body; and on the basis of a characteristic of the temperature of a vessel changing along with the diastole and systole of a heart, further determining, according to the temperature change signal, the heart rate of a human body that is subjected to thermal-imaged image collection. By means of the embodiments of the present disclosure, a thermal-imaged image of a human body can be collected, and the heart rate can be determined according to a temperature change of a vessel therein, so as to realize non-contact measurement of the heart rate. Moreover, the thermal-imaged image of the human body can be collected, only by means of a thermal-imaged image collection apparatus, for heart rate measurement, such that the manufacturing cost is low, and the universality and the portability are good.

Description

Heart rate detection method and device, electronic equipment and storage medium

This application claims the priority of the Chinese patent application filed on October 15, 2021, with the application number 202111204042.3, and the title of the invention is "heart rate detection method and device, electronic equipment and storage medium", the entire content of which is incorporated in this application by reference .

technical field

The present disclosure relates to the technical field of computers, and in particular to a heart rate detection method and device, electronic equipment and a storage medium.

Background technique

Heart rate is of great significance to measure the health of a person's heart. It refers to the number of heart beats between units, and is a physiological parameter for routine clinical diagnosis. Existing heart rate detection methods are usually determined by detecting changes in attenuation of reflected light or attenuation of transmitted light in blood. Such detection devices are usually expensive, poor in popularity and portability.

Contents of the invention

The disclosure proposes a heart rate detection method and device, electronic equipment and a storage medium, realizes non-contact measurement information, reduces manufacturing cost and improves universality and portability.

According to a first aspect of the present disclosure, a heart rate detection method is provided, including:

Determining a sequence of thermal imaging images, the sequence of thermal imaging images includes a plurality of thermal imaging images sequentially collected within a preset time interval, and the thermal imaging images are used to characterize the temperature distribution of the target area of the human body;

determining at least one eigenvalue corresponding to the thermal imaging image, the eigenvalue being used to characterize the blood vessel temperature in the target area of the human body;

Determining a temperature change signal according to a time-series position of the thermal imaging image corresponding to at least one of the characteristic values, the temperature change signal being used to characterize the temperature change of blood vessels in the target area of the human body within a preset time interval;

A target heart rate is determined according to the temperature change signal, and the target heart rate represents the heart rate of the human body corresponding to the sequence of thermal imaging images.

In a possible implementation manner, the determining at least one feature value corresponding to the thermal imaging image includes:

determining at least one region-of-interest image corresponding to the thermal imaging image, where the region-of-interest image is used to characterize the temperature distribution of blood vessels in the target region of the human body;

A feature value of a corresponding thermal imaging image is determined according to pixel values of at least one image of the region of interest.

In a possible implementation manner, the determining at least one ROI image corresponding to the thermal imaging image includes:

Determining at least one blood vessel region image in the thermal imaging image, where the blood vessel region image is an image of a region where blood vessels are located in the thermal imaging image;

Image processing is performed on the image of the blood vessel region according to a preset image processing strategy to obtain an image of the region of interest.

In a possible implementation manner, performing image processing on the image of the blood vessel region according to a preset image processing strategy to obtain the image of the region of interest includes:

Performing image processing on the image of the blood vessel region at least including two operations of top-hat operation and adaptive binarization to obtain an image of the region of interest.

Performing at least one anisotropic filter, top-hat operation, adaptive binarization and skeleton extraction on the image of the blood vessel region in sequence to obtain the image of the region of interest.

In a possible implementation manner, the determining the temperature change signal according to the time-series position of at least one of the characteristic values corresponding to the thermal imaging image includes;

Draw a candidate change curve according to the position of at least one of the characteristic values corresponding to the thermal imaging image in time series;

The candidate change curve is filtered according to the heart rate frequency band, which is predetermined according to the heart rate range of the human body, to obtain the temperature change signal.

In a possible implementation manner, the determining the target heart rate according to the temperature change signal includes:

Perform Fourier transform on the temperature change signal, and determine the frequency corresponding to the highest amplitude in the transform result as the target heart rate.

According to a second aspect of the present disclosure, a heart rate detection device is provided, comprising:

A sequence determination module, configured to determine a sequence of thermal imaging images, the sequence of thermal imaging images includes multiple thermal imaging images sequentially collected within a preset time interval, and the thermal imaging images are used to characterize the temperature distribution of the target area of the human body;

A temperature extraction module, configured to determine at least one feature value corresponding to the thermal imaging image, where the feature value is used to characterize the blood vessel temperature in the target area of the human body;

A signal determination module, configured to determine a temperature change signal according to the time series position of at least one of the characteristic values corresponding to the thermal imaging image, and the temperature change signal is used to characterize the blood vessel in the target area of the human body within a preset time interval Preset time interval for temperature change;

The heart rate determination module is configured to determine a target heart rate according to the temperature change signal, and the target heart rate represents the heart rate of the human body corresponding to the thermal imaging image sequence.

In a possible implementation manner, the temperature extraction module includes:

An image determining submodule, configured to determine at least one region-of-interest image corresponding to the thermal imaging image, where the region-of-interest image is used to characterize the temperature distribution of blood vessels in the target region of the human body;

The feature value determining submodule is configured to determine the feature value of the corresponding thermal imaging image according to the pixel value of at least one image of the region of interest.

In a possible implementation manner, the image determination submodule includes:

An image determining unit, configured to determine at least one blood vessel region image in the thermal imaging image, where the blood vessel region image is an image of a region where blood vessels are located in the thermal imaging image;

The image processing unit is configured to perform image processing on the image of the blood vessel region according to a preset image processing strategy to obtain an image of the region of interest.

In a possible implementation manner, the image processing unit includes:

The first image processing subunit is used to perform image processing on the blood vessel region image at least including two operations of top-hat operation and adaptive binarization to obtain the region-of-interest image.

In a possible implementation manner, the image processing unit includes:

The second image processing subunit is configured to sequentially perform at least one anisotropic filter, top-hat operation, adaptive binarization and skeleton extraction on the image of the blood vessel region to obtain the image of the region of interest.

In a possible implementation manner, the signal determination module includes;

The curve drawing submodule is used to draw a candidate change curve according to the position of at least one of the characteristic values corresponding to the thermal imaging image in time series;

The filtering sub-module is configured to filter the candidate change curves according to the heart rate frequency band, which is predetermined according to the heart rate range of the human body, to obtain the temperature change signal.

In a possible implementation manner, the heart rate determination module includes:

The time-frequency conversion sub-module is used to perform Fourier transformation on the temperature change signal, and determine the frequency with the highest corresponding amplitude in the transformation result as the target heart rate.

According to a third aspect of the present disclosure, there is provided an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented.

According to a fifth aspect of the present disclosure, there is provided a computer program product, including computer readable codes, or a non-volatile computer readable storage medium bearing computer readable codes, when the computer readable codes are stored in an electronic device When running in the processor of the electronic device, the processor in the electronic device is used to implement the above method.

The embodiment of the present disclosure can collect human body thermal imaging images, and based on the characteristic that the temperature of human blood vessels varies with the diastole and contraction of the heart, determine the heart rate according to the temperature changes of blood vessels in multiple human body thermal imaging images, so as to realize non-contact heart rate measurement. At the same time, the thermal imaging image of the human body can be collected only through the thermal imaging image acquisition device, and then transmitted to any other electronic equipment capable of image and signal processing for heart rate measurement, which is low in cost, good in universality and high in portability.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

The accompanying drawings here are incorporated into the description and constitute a part of the present description. These drawings show embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure.

FIG. 1 shows a schematic diagram of an application scenario of a heart rate detection method according to an embodiment of the present disclosure;

FIG. 2 shows a flow chart of a heart rate detection method according to an embodiment of the present disclosure;

Fig. 3 shows a schematic diagram of image positions of blood vessel regions according to an embodiment of the present disclosure;

Fig. 4 shows a schematic diagram of a temperature change signal according to an embodiment of the present disclosure;

Fig. 5 shows a schematic diagram of a process of determining a target heart rate according to an embodiment of the present disclosure;

Fig. 6 shows a schematic diagram of a heart rate detection device according to an embodiment of the present disclosure;

Fig. 7 shows a block diagram of an electronic device according to an embodiment of the present disclosure;

Fig. 8 shows a block diagram of an electronic device according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. In addition, the term "at least one" herein means any one of a variety or any combination of at least two of the more, for example, including at least one of A, B, and C, which may mean including from A, Any one or more elements selected from the set formed by B and C.

In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific implementation manners. It will be understood by those skilled in the art that the present disclosure may be practiced without some of the specific details. In some instances, methods, means, components and circuits that are well known to those skilled in the art have not been described in detail so as to obscure the gist of the present disclosure.

Fig. 1 shows a schematic diagram of an application scenario of a heart rate detection method according to an embodiment of the present disclosure. As shown in FIG. 1 , in a possible implementation manner, a system applying the heart rate detection method of the embodiment of the present disclosure may include a first electronic device 10 and a second electronic device 11 connected through a network. Wherein, the second electronic device 11 is used to collect thermal imaging images of the target area of the human body multiple times within a preset time interval to obtain a sequence of thermal imaging images and send them to the first electronic device 10 through the network. The first electronic device 10 is configured to, after receiving the thermal imaging image sequence, execute the heart rate detection method of the embodiment of the present disclosure to obtain the target heart rate of the human body in the thermal imaging image sequence.

Optionally, the system applying the heart rate detection method of the embodiment of the present disclosure may also only include the second electronic device 11, that is, the second electronic device 11 collects thermal imaging images of the target area of the human body multiple times within a preset time interval, and obtains Sequence of thermal imaging images. Further, the heart rate detection method of the embodiment of the present disclosure is directly executed based on the thermal imaging image sequence to obtain the target heart rate of the human body in the thermal imaging image sequence.

When the heart rate detection method is implemented by a system composed of the first electronic device 10 and the second electronic device 11, the first electronic device 10 may be an electronic device such as a terminal device or a server. Wherein, the terminal device may be user equipment (User Equipment, UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant, PDA), handheld device, computing device, vehicle-mounted device, Wearable devices and other devices capable of image and signal processing. The server can be a single server or a server cluster composed of multiple servers. The second electronic device 11 may be an image acquisition device capable of acquiring thermal imaging images, or a terminal device having a thermal imaging image acquisition function. The heart rate detection method in the embodiment of the present disclosure can be implemented by the processor in the first electronic device 10 calling the computer-readable instructions stored in the memory.

When the heart rate detection method is implemented only by the second electronic device 11, the second electronic device 11 may be a terminal device having a thermal imaging image acquisition function and capable of image processing and signal processing. Optionally, it may be user equipment (User Equipment, UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant, PDA), handheld device, computing device, vehicle-mounted device, Wearable equipment and other equipment.

Embodiments of the present disclosure may be applied to any application scenario where heart rate is detected by collecting thermal imaging images of a human body. For example, the target heart rate obtained by executing the heart rate detection method may be displayed for the user to view after the image of the user's body part is collected by a thermal imager with image processing and signal processing functions. Alternatively, after collecting images of parts of the user's body with a thermal imager, they can be sent to other electronic devices in a wired or wireless manner to perform heart rate detection and return to the target heart rate, and the thermal imager will display the received target heart rate for the user to view .

Fig. 2 shows a flow chart of a heart rate detection method according to an embodiment of the present disclosure. As shown in Figure 2, the heart rate detection method includes:

Step S10, determining a sequence of thermal imaging images.

In a possible implementation manner, the sequence of thermal imaging images includes multiple thermal imaging images sequentially collected within a preset time interval, and at least one thermal imaging image is used to characterize the temperature distribution of the target area of the human body. Optionally, thermal imaging images may be acquired through preset acquisition frequency. A thermal imaging image may have a time window characterizing the corresponding acquisition time. Further, the target area of the human body may be a pre-determined area of relatively dense blood vessels in the human body, such as the face area, the back of the hand area, or the wrist area.

That is to say, thermal imaging images of a specific area on a user's body can be collected multiple times according to a preset image acquisition frequency within a preset time interval by an image acquisition device with a thermal imaging image acquisition function, and the images arranged in the order of acquisition time are obtained. sequence of thermal imaging images.

Step S20, determining at least one feature value corresponding to the thermal imaging image.

In a possible implementation manner, for at least one thermal imaging image in the sequence of thermal imaging images, its corresponding feature value is respectively determined. Among them, the feature value is used to represent the blood vessel temperature in the target area of the human body, which can be the temperature value of the blood vessel in the target area of the human body, or the pixel value corresponding to the temperature of the blood vessel in the target area of the human body based on the pixel value-temperature mapping relationship in the thermal imaging image . For example, when the thermal imaging image is obtained by collecting the back of the user's hand with a thermal imager, the characteristic value may be the temperature in blood vessels on the back of the user's hand.

Optionally, in this embodiment of the present disclosure, the manner of determining the corresponding feature value of the thermal imaging image may be: determining an ROI image corresponding to at least one thermal imaging image, and the ROI image is used to characterize the temperature distribution of blood vessels in the target area of the human body. A feature value corresponding to the thermal imaging image is determined according to the pixel values of the at least one region-of-interest image. That is to say, by performing image processing on at least one thermal imaging image, extracting the region representing the blood vessel temperature distribution as the region of interest image, and calculating the feature value according to at least one pixel value in the region of interest image. Alternatively, at least one pixel value in the image of the region of interest may be converted into a corresponding temperature value according to the preset mapping relationship between the pixel value and the temperature in the thermal imaging image, and then the feature value may be obtained by calculating the at least one temperature value. Further, the feature value can be any value that characterizes the temperature characteristics of blood vessels in the target area of the human body, for example, it can be at least one pixel value or the average, variance, median, etc. of the temperature value corresponding to at least one pixel value.

Further, in the embodiment of the present disclosure, the manner of determining the ROI image corresponding to at least one thermal imaging image may further include: determining a blood vessel region image in at least one thermal imaging image, where the blood vessel region image is an image of a region where blood vessels are located in the thermal imaging image. Image processing is performed on the image of the blood vessel region according to a preset image processing strategy to obtain an image of the region of interest. That is to say, first obtain the image of the blood vessel area in the thermal imaging image as the blood vessel area image, and then remove the interference caused by other positions in the blood vessel area image except the blood vessel position through further image processing, and obtain only the blood vessel position temperature The distributed image is used as the ROI image.

In a possible implementation manner, the image of the blood vessel area may be an image obtained by cropping the thermal imaging image, and the position of the image of the blood vessel area in the thermal imaging is the cropping area. Optionally, the clipping area of the image of the blood vessel region may be preset according to the location of dense blood vessels in the human body. For example, when the target area of the human body is the hand area, the image of the blood vessel area may be the area where the back of the hand is located, that is, the clipping area is the back area of the hand. When the target area of the human body is the face area, the image of the blood vessel area may be the area where the forehead is located, that is, the clipped area is the forehead area. Further, when the target area of the human body is the back of the hand or the forehead area, the thermal imaging image may not be cropped, and the thermal imaging image may be directly determined as the image of the blood vessel area.

Fig. 3 shows a schematic diagram of image positions of blood vessel regions according to an embodiment of the present disclosure. As shown in Figure 3, since the blood vessel area above the forehead of the human body is more obvious and relatively easy to extract, when the thermal imaging image is an image obtained by collecting the face area of the human body, the position of the blood vessel area image can be set as the forehead position in the face area 31. That is to say, for at least one thermal imaging image, a partial image of the forehead position 31 is obtained by cropping as a corresponding blood vessel region image.

After the image of the blood vessel area is obtained, further image processing is performed to remove the interference caused by positions other than the position of the blood vessel, and the image of the position of the blood vessel is extracted as the image of the region of interest. Optionally, the image processing process in this embodiment of the present disclosure may be to perform image processing on the blood vessel region image at least including two operations of top-hat operation and adaptive binarization to obtain the region-of-interest image. That is, the top-hat operation is first performed on the image to enlarge the texture area, and then the image with the highlighted texture area is obtained by subtracting the image after the opening operation from the original image. The enlarged image of the blood vessel position is subtracted to obtain an image including only image data near the blood vessel position. The image of the region of interest can be obtained by further enlarging the blood vessel position in the extracted image by means of adaptive binarization.

In order to improve the accuracy of the image processing result, the embodiments of the present disclosure may also add other processing manners in the image processing process. For example, at least one of anisotropic filtering, top-hat operation, adaptive binarization and skeleton extraction may be performed sequentially on the image of the blood vessel region to obtain the image of the region of interest.

Wherein, at least one anisotropic filter is equivalent to retaining the texture information of the image while suppressing the image noise, that is, suppressing the image noise of the blood vessel region while retaining the texture of the blood vessel position. The top-hat operation first performs an opening operation on the image to enlarge the texture area, and then subtracts the image after the opening operation from the original image to obtain an image that highlights the texture area, that is, first enlarges the position of the blood vessel in the image of the blood vessel area, and then subtracts and enlarges it through the image of the blood vessel area After the image of the blood vessel location, an image including only image data near the blood vessel location is obtained. Since the extracted image at this time is relatively blurred, the positions of blood vessels in the extracted image are enlarged by means of adaptive binarization. In order to reduce the possibility of introducing more noise during the zoom-in process, through the skeleton extraction of the zoomed-in image, the position of the blood vessel is refined, and the image of the region of interest including the refined blood vessel position is obtained.

After obtaining the region of interest image corresponding to the at least one thermal imaging image, at least one pixel value included in the at least one region of interest image, or the temperature corresponding to the at least one pixel value is calculated to obtain the feature value. Optionally, the feature value may be determined by calculating an average value of at least one pixel value in the region-of-interest image, or by calculating an average value of temperature values corresponding to at least one pixel value in the region-of-interest image.

In the embodiment of the present disclosure, when determining the corresponding feature value of at least one thermal imaging image, the fine extraction of the blood vessel part is realized through image processing, which effectively reduces the noise, improves the accuracy of the feature value, and further improves the accuracy of the final determination of the target heart rate. .

Step S30 , determining a temperature change signal according to a time-series position of at least one of the characteristic values corresponding to the thermal imaging image.

In a possible implementation manner, after the feature value corresponding to the at least one thermal imaging image is determined, the time-series position of the at least one feature value is sequentially determined according to the position sequence of the at least one thermal imaging image in time sequence, that is, the acquisition time sequence. Further, a corresponding temperature change signal is generated according to the at least one eigenvalue and the time series position of the at least one eigenvalue. The temperature change signal is used to characterize the temperature change of blood vessels in the target area of the human body within a preset time interval, and can be a continuous curve signal drawn according to at least one thermal imaging image corresponding to the eigenvalue, or a plurality of eigenvalues formed in time series discrete signal.

Fig. 4 shows a schematic diagram of a temperature change signal according to an embodiment of the disclosure. As shown in FIG. 4 , the embodiment of the present disclosure may generate a temperature change curve 40 as a temperature change signal according to at least one eigenvalue and the time-series position of the at least one eigenvalue. Wherein, the abscissa of the temperature change curve 40 represents the time, and the ordinate represents the characteristic value, and any point on the temperature change curve 40 represents the temperature characteristics of blood vessels in the target area of the human body at the corresponding time.

Further, in the case that the temperature change signal is a continuous curve signal, in order to reduce the possibility of noise affecting the accuracy of the temperature change signal, in a possible implementation, the method of determining the temperature change signal can be based on at least one characteristic The value corresponds to the position of the thermal imaging image in the time series to draw a candidate change curve, and then filter the candidate change curve according to the heart rate frequency band to obtain a temperature change signal. The heart rate frequency band is predetermined according to the heart rate range of the human body. That is to say, first generate a candidate change curve based on at least one eigenvalue and the time-series position of at least one eigenvalue, and then filter the candidate change curve based on the possible heart rate range of the human body to remove noise to obtain an accurate representation of the human body within the preset time interval The temperature change curve of the temperature change situation is used as a temperature change signal. Optionally, since a person's heart rate is usually 50-160 beats per minute, it can be determined according to the heart rate range that the corresponding heart rate frequency band is 5/6-8/3HZ.

Step S40, determining the target heart rate according to the temperature change signal.

Usually, the temperature of the blood is higher when the heart is diastolic, and the temperature of the blood is lower when the heart is systolic. Therefore, the temperature in the blood vessels of the human body will change with the diastole and contraction of the heart, that is, every heartbeat will cause the temperature in the blood vessels to fluctuate, and the target heart rate can be determined based on the temperature change signal that characterizes the temperature change in the blood vessels. That is to say, the temperature change signal is a continuous signal or a discrete signal that characterizes the beating frequency of the heart based on the relationship between the blood vessel temperature change and the beating heart.

In a possible implementation manner, the target heart rate represents the heart rate of the human body corresponding to the thermal imaging image sequence, that is, the heart rate of the user whose thermal imaging image sequence is obtained from the captured target area of the human body. Optionally, the manner of determining the target heart rate according to the temperature change signal may be to perform Fourier transform on the temperature change signal, and determine the frequency corresponding to the highest amplitude in the transformation result as the target heart rate. That is, the temperature change signal is converted from the time domain to the frequency domain, and the component with the highest amplitude is determined as the target heart rate, and the other components are noise components.

Fig. 5 shows a schematic diagram of a process of determining a target heart rate according to an embodiment of the present disclosure. As shown in FIG. 5 , the embodiment of the present disclosure collects thermal imaging images of target areas on the user's body within a preset time period, and obtains a thermal imaging image sequence 50 composed of N thermal imaging images arranged in chronological order. For at least one thermal imaging image 51 included in the thermal imaging image sequence 50 , the region representing the temperature distribution of the region where the user's blood vessel is located is respectively intercepted to obtain the blood vessel region image 52 . By performing image processing on the blood vessel region image 52 , an ROI image 53 including only the temperature distribution of the blood vessel is extracted, and a feature value 54 is calculated according to at least one pixel value in at least one ROI image 53 .

Further, a candidate variation curve 55 is drawn according to the arrangement order of the at least one thermal imaging image 51 in the thermal imaging image sequence 50 and its corresponding feature value 54 . In order to ensure the authenticity and accuracy of the candidate change curve 55, the heart rate frequency band is determined according to the possible heart rate range of the human body, and the candidate change curve 55 is filtered to remove other noises to obtain a temperature change signal 56 with high accuracy. Optionally, the temperature change signal 56 can be converted from the time domain to the frequency domain by way of Fourier transform to obtain the transformation result 57 . Wherein, the change result 57 includes multiple components constituting the temperature change signal 56 , and the frequency corresponding to the component with the highest amplitude is determined as the target heart rate 58 , that is, the user's heart rate for capturing the thermal imaging image 51 .

Further, in the embodiment of the present disclosure, when determining the corresponding feature value of at least one thermal imaging image, the fine extraction of the blood vessel part is realized through image processing, which effectively reduces the noise, improves the accuracy of the feature value, and further improves the final determination of the target heart rate. accuracy.

It can be understood that at least one of the above method embodiments mentioned in this disclosure can be combined with each other to form a combined embodiment without violating the principle and logic. Due to space limitations, this disclosure will not repeat them. Those skilled in the art can understand that, in the above-mentioned method in the specific implementation manner, the specific execution order of at least one step should be determined by its function and possible internal logic.

In addition, the present disclosure also provides a heart rate detection device, electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any heart rate detection method provided in the present disclosure. For the corresponding technical solutions and descriptions, refer to the corresponding records in the method section ,No longer.

Fig. 6 shows a schematic diagram of a heart rate detection device according to an embodiment of the present disclosure. As shown in Fig. 6, the device includes:

A sequence determination module 60, configured to determine a sequence of thermal imaging images, the sequence of thermal imaging images includes multiple thermal imaging images sequentially collected within a preset time interval, and the thermal imaging images are used to characterize the temperature distribution of the target area of the human body ;

A temperature extraction module 61, configured to determine at least one feature value corresponding to the thermal imaging image, where the feature value is used to characterize the blood vessel temperature in the target area of the human body;

A signal determining module 62, configured to determine a temperature change signal according to the time series position of at least one of the characteristic values corresponding to the thermal imaging image, and the temperature change signal is used to characterize the blood vessels in the target area of the human body within a preset time interval The preset time interval for temperature changes;

The heart rate determination module 63 is configured to determine a target heart rate according to the temperature change signal, and the target heart rate represents the heart rate of the human body corresponding to the thermal imaging image sequence.

In a possible implementation manner, the image processing unit includes:

The first image processing subunit is configured to perform image processing on the image of the blood vessel region at least including two operations of top-hat operation and adaptive binarization to obtain an image of the region of interest.

In a possible implementation manner, the image processing unit includes:

In a possible implementation manner, the signal determination module includes;

In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the method embodiments above, and its specific implementation can refer to the description of the method embodiments above. For brevity, here No longer.

Embodiments of the present disclosure also provide a computer-readable storage medium, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor. Computer readable storage media may be volatile or nonvolatile computer readable storage media.

An embodiment of the present disclosure also proposes an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

An embodiment of the present disclosure also provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.

Electronic devices may be provided as terminals, servers, or other forms of devices.

FIG. 7 shows a block diagram of an electronic device 700 according to an embodiment of the present disclosure. For example, the electronic device 700 may be a terminal such as a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, or a personal digital assistant.

7, electronic device 700 may include one or more of the following components: processing component 702, memory 704, power supply component 706, multimedia component 708, audio component 710, input/output (I/O) interface 712, sensor component 714 , and the communication component 716.

The processing component 702 generally controls the overall operations of the electronic device 700, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 702 may include one or more modules that facilitate interaction between processing component 702 and other components. For example, processing component 702 may include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702 .

The memory 704 is configured to store various types of data to support operations at the electronic device 700 . Examples of such data include instructions for any application or method operating on the electronic device 700, contact data, phonebook data, messages, pictures, videos, and the like. The memory 704 can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

The power supply component 706 provides power to various components of the electronic device 700 . Power components 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 700 .

The multimedia component 708 includes a screen providing an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 708 includes a front camera and/or a rear camera. When the electronic device 700 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.

The audio component 710 is configured to output and/or input audio signals. For example, the audio component 710 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 700 is in operation modes, such as call mode, recording mode and voice recognition mode. Received audio signals may be further stored in memory 704 or sent via communication component 716 . In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

The I/O interface 712 provides an interface between the processing component 702 and a peripheral interface module, which may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.

Sensor assembly 714 includes one or more sensors for providing status assessments of various aspects of electronic device 700 . For example, the sensor component 714 can detect the open/closed state of the electronic device 700, the relative positioning of components, such as the display and the keypad of the electronic device 700, the sensor component 714 can also detect the electronic device 700 or one of the electronic device 700 The position of components changes, the presence or absence of user contact with the electronic device 700 , the orientation or acceleration/deceleration of the electronic device 700 and the temperature change of the electronic device 700 . Sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 714 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor component 714 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 716 is configured to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 716 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, electronic device 700 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmable gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation for performing the methods described above.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as the memory 704 including computer program instructions, which can be executed by the processor 720 of the electronic device 700 to implement the above method.

FIG. 8 shows a block diagram of an electronic device 800 according to an embodiment of the present disclosure. For example, the electronic device 800 may be provided as a server. Referring to FIG. 8 , electronic device 800 includes processing component 822 , which further includes one or more processors, and a memory resource represented by memory 832 for storing instructions executable by processing component 822 , such as application programs. The application program stored in memory 832 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 822 is configured to execute instructions to perform the above method.

Electronic device 800 may also include a power supply component 826 configured to perform power management of electronic device 800, a wired or wireless network interface 850 configured to connect electronic device 800 to a network, and an input-output (I/O) interface 858 . The electronic device 800 can operate based on the operating system stored in the memory 832, such as the Microsoft server operating system (Windows ServerTM), the graphical user interface-based operating system (Mac OS ^XTM ) introduced by Apple Inc., the multi-user and multi-process computer operating system ( Unix ^™ ), a free and open-source Unix-like operating system (Linux ^™ ), an open-source Unix-like operating system (FreeBSD ^™ ), or the like.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as a memory 832 including computer program instructions, which can be executed by the processing component 822 of the electronic device 800 to implement the above method.

The present disclosure can be a system, method and/or computer program product. A computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to implement various aspects of the present disclosure.

A computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device, such as a printer with instructions stored thereon A hole card or a raised structure in a groove, and any suitable combination of the above. As used herein, computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.

Computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or Source or object code written in any combination, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as via the Internet using an Internet service provider). connect). In some embodiments, an electronic circuit, such as a programmable logic circuit, field programmable gate array (FPGA), or programmable logic array (PLA), can be customized by utilizing state information of computer-readable program instructions, which can Various aspects of the present disclosure are implemented by executing computer readable program instructions.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It should be understood that each block of the flowcharts and/or block diagrams, and combinations of blocks in the flowcharts and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices and/or other devices to work in a specific way, so that the computer-readable medium storing instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in flowcharts and/or block diagrams.

It is also possible to load computer-readable program instructions into a computer, other programmable data processing device, or other equipment, so that a series of operational steps are performed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process , so that instructions executed on computers, other programmable data processing devices, or other devices implement the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.

The computer program product can be specifically realized by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) etc. wait.

Having described various embodiments of the present disclosure above, the foregoing description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principle of each embodiment, practical application or improvement of technology in the market, or to enable other ordinary skilled in the art to understand each embodiment disclosed herein.

Claims

A heart rate detection method, characterized in that the method comprises:

Determining a sequence of thermal imaging images, the sequence of thermal imaging images includes a plurality of thermal imaging images sequentially collected within a preset time interval, and the thermal imaging images are used to characterize the temperature distribution of the target area of the human body;

determining at least one eigenvalue corresponding to the thermal imaging image, the eigenvalue being used to characterize the blood vessel temperature in the target area of the human body;

Determining a temperature change signal according to a time-series position of the thermal imaging image corresponding to at least one of the characteristic values, the temperature change signal being used to characterize the temperature change of blood vessels in the target area of the human body within a preset time interval;

A target heart rate is determined according to the temperature change signal, and the target heart rate represents the heart rate of the human body corresponding to the sequence of thermal imaging images.
The method according to claim 1, wherein said determining at least one characteristic value corresponding to said thermal imaging image comprises:

determining at least one region-of-interest image corresponding to the thermal imaging image, where the region-of-interest image is used to characterize the temperature distribution of blood vessels in the target region of the human body;

A feature value of a corresponding thermal imaging image is determined according to pixel values of at least one image of the region of interest.
The method according to claim 2, wherein said determining at least one ROI image corresponding to said thermal imaging image comprises:

Determining at least one blood vessel region image in the thermal imaging image, where the blood vessel region image is an image of a region where blood vessels are located in the thermal imaging image;

Image processing is performed on the image of the blood vessel region according to a preset image processing strategy to obtain an image of the region of interest.
The method according to claim 3, wherein the image processing of the blood vessel region image according to a preset image processing strategy to obtain the region of interest image comprises:

Performing image processing on the image of the blood vessel region at least including two operations of top-hat operation and adaptive binarization to obtain an image of the region of interest.
The method according to claim 3 or 4, wherein the image processing of the blood vessel region image according to a preset image processing strategy to obtain the region of interest image comprises:

Performing at least one anisotropic filter, top-hat operation, adaptive binarization and skeleton extraction on the image of the blood vessel region in sequence to obtain the image of the region of interest.
The method according to any one of claims 1-5, wherein the determining the temperature change signal according to the time series position of at least one of the characteristic values corresponding to the thermal imaging image comprises;

Draw a candidate change curve according to the position of at least one of the characteristic values corresponding to the thermal imaging image in time series;

The candidate change curve is filtered according to the heart rate frequency band, which is predetermined according to the heart rate range of the human body, to obtain the temperature change signal.
The method according to any one of claims 1-6, wherein the determining the target heart rate according to the temperature change signal comprises:

Perform Fourier transform on the temperature change signal, and determine the frequency corresponding to the highest amplitude in the transform result as the target heart rate.
A heart rate detection device, characterized in that the device comprises:

A sequence determination module, configured to determine a sequence of thermal imaging images, the sequence of thermal imaging images includes multiple thermal imaging images sequentially collected within a preset time interval, and the thermal imaging images are used to characterize the temperature distribution of the target area of the human body;

A temperature extraction module, configured to determine at least one feature value corresponding to the thermal imaging image, where the feature value is used to characterize the blood vessel temperature in the target area of the human body;

A signal determination module, configured to determine a temperature change signal according to the time series position of at least one of the characteristic values corresponding to the thermal imaging image, and the temperature change signal is used to characterize the blood vessel in the target area of the human body within a preset time interval Preset time interval for temperature changes;

The heart rate determination module is configured to determine a target heart rate according to the temperature change signal, and the target heart rate represents the heart rate of the human body corresponding to the thermal imaging image sequence.
An electronic device, characterized in that it comprises:

processor;

memory for storing processor-executable instructions;

Wherein, the processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1-7.
A computer-readable storage medium, on which computer program instructions are stored, wherein, when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.
A computer program product, comprising computer readable codes, or a non-volatile computer readable storage medium bearing computer readable codes, when the computer readable codes are run in a processor of an electronic device, the electronic The processor in the device executes the method for realizing any one of claims 1-7.