JP2010264095A - Heart rate measuring apparatus and heart rate measuring method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a heart rate measuring apparatus which can measure the heart rate of a living body without coming into contact with the living body. <P>SOLUTION: This heart rate measuring apparatus includes: acquiring means 10 to 13 which acquire temperature information at one or two or more areas of the living body; an extracting means which extracts frequency data corresponding to the frequency component of a frequency zone being equivalent to the heart rate of the living body; and a measuring means which measures the heart rate of the living body conforming to the frequency data which has been extracted by the extracting means. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a heart rate measuring device and a heart rate measuring method.

  Conventionally, a technique for measuring the number of breaths of a person to be measured, the intensity of breathing, etc. without touching the person to be measured by measuring the time change of the temperature of the nasal cavity of the person to be measured without using an infrared camera is known. (Patent Document 1).

Japanese Patent Laid-Open No. 9-253071

  However, the conventional technique is intended to measure the respiration of the measurement subject and cannot measure the heart rate of the measurement subject.

  The problem to be solved by the present invention is to provide a heart rate measuring device capable of measuring the heart rate of a living body without contacting the living body.

  The present invention extracts the frequency component of the frequency band corresponding to the heartbeat of the living body from the temperature information of one or more parts of the living body, and measures the heart rate of the living body based on the frequency component. To solve.

  According to the present invention, since the frequency component can be acquired from the temperature information of one or more parts of the living body, the heart rate of the living body can be measured without contacting the living body.

It is a figure which shows the structure of the heart rate measuring system which concerns on 1st Embodiment. It is a flowchart which shows the heart rate measurement process which concerns on 1st implementation person. It is a figure which shows an example of the thermal image imaged with the infrared camera. It is the enlarged view which expanded and showed the user's nasal cavity part periphery among the thermal images shown in FIG. It is a figure which shows an example of the temperature information of a nose protrusion. It is a figure which shows the frequency component converted by the Fourier transform from the temperature information of a nose protrusion shown in FIG. It is a figure which shows an example of the temperature information which extracted the frequency component of the frequency band corresponded to a heartbeat from the frequency component shown in FIG. 6 by the band pass filter, and was reconverted from the extracted frequency component. It is a figure for demonstrating the method of measuring a user's heart rate by wavelet transformation. It is a figure which shows an example of the temperature information of an earlobe part. It is a figure which shows the frequency component converted by the Fourier transform from the temperature information of the earlobe part shown in FIG. It is a flowchart which shows the heart rate measurement process which concerns on 2nd Embodiment. It is a flowchart which shows the heart rate measurement process which concerns on 3rd Embodiment. It is a flowchart (the 1) which shows the heart rate measurement process which concerns on 4th Embodiment. It is a flowchart (the 2) which shows the heart rate measurement process which concerns on 4th Embodiment. It is a flowchart which shows the heart rate measurement process which concerns on 5th Embodiment. It is a figure which shows the user's face divided | segmented into the several cell. It is a flowchart which shows the heart rate measurement process which concerns on 6th Embodiment. It is a figure which shows the example of installation of the infrared camera in 7th Embodiment. It is a flowchart which shows the heart rate measurement process which concerns on 7th Embodiment. It is a figure which shows an example of the thermal image imaged with the infrared camera installed in the user's left side. It is a block diagram which shows the heart rate measurement system which concerns on 8th Embodiment. It is a flowchart which shows the heart rate measurement process which concerns on 8th Embodiment. It is a figure for demonstrating the method of specifying a user's site | part using a visible light camera, Comprising: It is a figure which shows the brightness | luminance image imaged with the visible light camera. It is a figure for demonstrating the method of specifying a user's site | part using a visible light camera, Comprising: It is a figure which shows the thermal image imaged with the infrared camera.

<< First Embodiment >>
Hereinafter, embodiments of the present invention will be described with reference to the drawings. The heart rate measurement system of this embodiment is mounted on a vehicle, for example, and measures a user's heart rate. The user's heart rate measured by the heart rate measurement system is used, for example, when estimating the user's driving state.

  FIG. 1 is a diagram showing a configuration of a heart rate measurement system according to the present embodiment. As shown in FIG. 1, the heart rate measurement system includes an infrared camera 10 for detecting the surface (skin) temperature of the user and a controller 20 for measuring the heart rate of the user.

  As shown in FIG. 1, the infrared camera 10 is installed at a front position of the user, for example, a position 60 cm ahead of the user, and detects temperature data in a range centered on the user's face. The infrared camera 10 includes an image sensor and can detect temperature data in the imaging range of the infrared camera 10. In addition, a plurality of pixels (pixels) are provided on the imaging device of the infrared camera 10. For example, an imaging device having a pixel number of 320 × 240 is used as the imaging device of the infrared camera. Each pixel on the image sensor can detect temperature data in a range captured by each pixel. The infrared camera 10 is set to, for example, a frame rate of 1/30 fs and an emissivity of 1.0, and periodically detects temperature data of the user's surface (skin). The infrared camera 10 sequentially transmits a plurality of temperature data detected in each pixel to the controller 20.

  The controller 20 functions as a ROM (Read Only Memory) storing a program for measuring a user's heart rate, a CPU (Central Processing Unit) executing the program stored in the ROM, and an accessible storage device RAM (Random Access Memory).

  The controller 20 executes a program stored in the ROM by the CPU, so that each function of a temperature data acquisition function, a specific function, an average temperature calculation function, a storage function, a temperature information acquisition function, a conversion function, an extraction function, and a measurement function To realize. Below, each function with which the controller 20 is provided is demonstrated.

  The temperature data acquisition function of the controller 20 acquires temperature data detected in each pixel of the infrared camera 10 from the infrared camera 10. As described above, since the infrared camera 10 has a plurality of pixels and detects temperature data in each pixel, the temperature data acquisition function acquires a plurality of temperature data from the infrared camera 10.

  The specifying function of the controller 20 specifies a predetermined part of the user based on the temperature data acquired by the temperature data acquiring function, and specifies which part of the user the acquired temperature data is based on. A method for specifying which part of the user the temperature data detected by the infrared camera 10 by the specifying function will be described later.

  The storage function of the controller 20 stores the average temperature data of a predetermined part of the user in the RAM of the controller 20. The average temperature data of the predetermined part of the user stored by the storage function is the average temperature data of the predetermined part of the user calculated by the average temperature calculation function of the controller 20. That is, the average temperature calculation function calculates average temperature data from a plurality of temperature data detected at a predetermined part of the user specified by the specific function among the temperature data detected by all the pixels of the infrared camera 10, The storage function stores the average temperature data calculated by the average temperature calculation function along a time series. The time series data of the average temperature data of the predetermined part stored by the storage function is acquired as the temperature information of the predetermined part by the temperature information acquisition function of the controller 20.

  The conversion function of the controller 20 converts the temperature information of the user's part acquired by the temperature information acquisition function into a frequency component, and the extraction function of the controller 20 calculates the frequency corresponding to the heartbeat from the frequency component converted from the temperature information. Extract frequency components of the band. Here, the surface (skin) temperature of a person or the like is affected by fluctuations in blood circulating in the body. That is, the surface (skin) temperature changes according to the blood vessel contraction fluctuation. The extraction function extracts a blood vessel contraction component by extracting a frequency component in a frequency band corresponding to a heartbeat from frequency components converted from temperature information of a predetermined part of the user. A method of converting the temperature information into frequency components by the conversion function and the extraction function and extracting the frequency components in the frequency band corresponding to the heartbeat from the converted frequency components will be described later.

  The measurement function of the controller 20 measures the user's heart rate based on the frequency component extracted by the extraction function. A method for measuring the user's heart rate by the measurement function will be described later.

  Next, a heart rate measurement process in the present embodiment will be described with reference to FIG. FIG. 2 is a flowchart showing a heart rate measurement process according to the first embodiment. In the following, a method for measuring the user's heart rate based on the temperature data of the nasal protrusion near the center of the nose of the user's face will be described. 2 is executed by the controller 20 at regular time intervals.

In step S101, temperature data detected by the infrared camera 10 is acquired by the temperature data acquisition function. FIG. 3 is a diagram illustrating an example of a thermal image captured by the infrared camera 10. In FIG. 3, a pixel in which low temperature data is detected is a light color, a pixel in which high temperature data is detected is a dark color, and the temperature level of the detected temperature data is expressed in black and white. . That is, that in FIG. 3, the temperature of the face of the user, because it is displayed in a dark color compared to the background temperature T _b, the temperature of the face of the user is at a temperature higher than the background temperature T _b Show. The temperature data acquisition function includes all temperature data detected by the infrared camera 10, for example, in the example shown in FIG. 3, in addition to the temperature data detected in the pixel that images the user's face, Temperature data detected by a pixel that images the portion is also acquired.

  In step S102, based on the acquired temperature data, among the temperature data captured by a plurality of pixels, each temperature data based on the nasal protrusion near the center of the nose of the user's face is specified. . Below, an example of the method of specifying the temperature data based on a user's nose protrusion is demonstrated.

First, the specifying function specifies a pair of left and right nasal cavity portions of the user in order to specify temperature data of the user's nasal protrusion. For example, as shown in FIG. 3, the specific function obtains the central portion of the user's face from the thermal image captured by the infrared camera 10, while the background temperature and the portion other than the user's face detecting a background temperature T _b can be determined. Then, in the thermal image captured by the infrared camera 10, located in the nearest position from the central portion of the face portion of the user, and, by detecting the pixels detected temperature data close to the background temperature T _b 2 places, the user Identify the nasal cavity. As shown in FIG. 3, the nasal cavity is located near the center of the face, and the temperature data of the nasal cavity is detected as a lower temperature than other facial parts by the user's breathing. . In order to improve the detection accuracy of the nasal cavity of the user, in identifying nasal portion of the user is at a position closer to the center portion of the face portion of the user, and detects the temperature data close to the background temperature T _b It is preferable to specify a portion where a plurality of pixels are continuous as a nasal cavity portion.

  Next, the user's nasal projection is specified based on the specified pair of left and right nasal cavity portions. FIG. 4 is an enlarged view showing an enlargement of the periphery of the user's nasal cavity in the thermal image shown in FIG. 3. In FIG. 4, as in FIG. 3, the temperature level is represented by black and white shading. As shown in FIG. 4, the nose protrusion refers to a portion around the center of the nose. First, in order to identify the nasal protrusion, the specifying function obtains a center line passing through the center position between the specified pair of left and right nasal cavity parts in the Y-axis direction, while the upper limit of the pair of left and right nasal cavity parts ( The upper limit line in the X-axis direction passing through (upper end) is obtained. Then, as shown in FIG. 4, the specific function is a region that extends a predetermined range in the X-axis direction around the center line of the pair of left and right nasal cavity portions, and is located above the upper limit line of the pair of left and right nasal cavity portions. Then, an area that extends a predetermined range in the Y-axis direction is specified as a nose protrusion. Thereby, the specific function can specify each temperature data detected in the pixel which images the specified nose protrusion as temperature data based on the nose protrusion.

  In step S103, the average temperature calculation function calculates average temperature data of the user's nasal protrusion from each temperature data based on the user's nasal protrusion identified in step S102. The infrared camera 10 includes a plurality of pixels on the image sensor, and in step S102, a plurality of temperature data of the user's nasal protrusion is detected by a plurality of pixels that capture the nasal protrusion of the user. In step S103, average temperature data of the user's nasal projection is calculated from each temperature data detected in a plurality of pixels that image the nasal projection of the user.

  In step S104, the average temperature data of the nose protrusion calculated in step S103 is stored in the RAM of the controller 20 by the storage function. Here, FIG. 5 is a diagram illustrating an example of temperature information of the user's nose protrusion. The infrared camera 10 detects the photographing data at the set frame rate, and sequentially transmits the detected temperature data to the controller 20. And according to transmission of the temperature data from the infrared camera 10, the process to step S101-104 is performed. Therefore, in step S104, the average temperature data of the nose protrusion is stored in time series. In step S105, the temperature information acquisition function acquires the average temperature data of the nasal protrusion stored along the time series as the temperature information of the user's nasal protrusion.

  In step S106, the temperature information of the user's nasal protrusion is converted into a frequency component by the conversion function. In subsequent step S107, a frequency component in a frequency band corresponding to the heartbeat is extracted from the frequency components converted from the temperature information of the user's nose protrusion. In step S108, the heart rate of the user is measured by the measurement function based on the extracted frequency component. The processing from step S106 to step S108 is performed using a technique such as Fourier transform, band pass filter processing, or wavelet transform, for example. Hereinafter, a method for measuring a user's heart rate using each method will be described in detail.

  First, a method for measuring a user's heart rate using Fourier transform will be described.

  FIG. 6 is a diagram illustrating frequency components converted from the temperature information of the nose protrusion illustrated in FIG. 5 by Fourier transform. The conversion function performs Fourier transform on the temperature information of the nasal protrusion shown in FIG. 5, thereby converting the temperature information of the nasal protrusion into a frequency component as shown in FIG.

  The extraction function has a power spectrum intensity in a frequency band corresponding to a heartbeat, for example, a frequency band of 0.8 to 1.5 Hz, among frequency components converted from temperature information of the user's nose protrusion by the conversion function. Extract the maximum frequency. For example, regarding the example illustrated in FIG. 6, the extraction function extracts a frequency f <b> 1 that is a frequency at which the power spectrum intensity is maximum in a frequency band of frequency 0.8 Hz to 1.5 Hz that is a frequency band corresponding to a heartbeat.

Next, the measurement function measures the user's heart rate based on the frequency extracted by the extraction function. For example, regarding the example shown in FIG. 6, the measurement function measures the heart rate (beats / minute) by the following formula (1) based on the detected frequency f1.
Heart rate (beats / minute) = f1 × 60 (1)
That is, since the frequency f1 is assumed to be a frequency corresponding to a heartbeat, the heart rate of the user can be measured by calculating the frequency f1 per minute.

When the heart rate is measured by the measuring means, as shown in FIG. 6, the heart rate is not measured using the frequency f1 at which the power spectrum intensity is maximum, but a frequency band corresponding to the heart rate. Even if the frequency f2 that is the center of gravity is detected in the frequency band from 0.8 Hz to 1.5 Hz, and the heart rate (beats / minute) is measured based on the following equation (2) using the detected frequency f2. Good.
Heart rate (beats / minute) = f2 x 60 (2)

  Next, a method of measuring the user's heart rate by bandpass filter processing will be described.

  First, the temperature information of the nasal protrusion is Fourier-transformed by the conversion function, and the temperature information of the nasal protrusion is converted into a frequency component as shown in FIG. Next, the extraction function extracts a frequency component in a frequency band of a frequency band of 0.8 Hz to 1.5 Hz corresponding to a heartbeat from the converted frequency component by a band pass filter. Then, the extraction function performs inverse Fourier transform on the extracted frequency component, and reconverts the extracted frequency component into temperature information of the nasal protrusion.

  FIG. 7 shows an example of temperature information of the nasal protrusion obtained by extracting a frequency component of the frequency band corresponding to the heartbeat from the frequency component shown in FIG. 6 using a bandpass filter, and reconverting the extracted frequency component by inverse Fourier transform. FIG. In FIG. 7, the temperature information of the nasal protrusion after extraction by bandpass filter processing is indicated by a solid line, and a pulse waveform attached to the fingertip is used for comparison with the temperature information of the nasal protrusion. The finger plethysmogram measured in this way is indicated by a one-dot chain line.

The measurement function detects the peak number n of the temperature information of the nasal protrusion indicated by a solid line for the measurement time t seconds, and measures the user's heart rate by the following equation (3).
Heart rate (beats / minute) = (60 / t) × n (3)
Note that the criterion for determining whether the temperature information is a peak can be appropriately determined. For example, after the temperature increase rate at a predetermined time exceeds a certain value, the temperature decrease rate at the predetermined time becomes a certain value or more. It can be determined that it is a peak.

  Further, in FIG. 7, the finger plethysmogram is displayed by a one-dot chain line. As shown in FIG. 7, the finger plethysmogram and the temperature information of the nasal protrusion after the band-pass filter processing are almost the same, and the user's heart rate is appropriate based on the temperature information of the nasal protrusion. It can be seen that it can be measured.

  Next, a method for measuring a user's heart rate using wavelet transform will be described.

  FIG. 8 is a diagram for explaining a method of measuring a user's heart rate by wavelet transform. Note that the temperature information S on the left side in FIG. 8 shows an example of temperature information of the nose protrusion.

  First, the conversion function performs multi-resolution decomposition on the temperature information S of the nose protrusion shown in the left diagram of FIG. 8 by wavelet transform, and obtains frequency components from the layers d1 to d5 shown in the center of FIG. The frequency components of the layers d1 to d5 have the relationship of temperature information S = d1 + d2 + d3 + d4 + d5, the layer d1 indicates the frequency component corresponding to the highest frequency band of the temperature information S, and the layer d5 is the highest of the temperature information S. The frequency component of the low frequency band is shown. As described above, the hierarchy d5 to the hierarchy d1 shown in FIG. 8 extract the frequency component corresponding to the high frequency band from the frequency information of the low frequency band extracted from the temperature information S from the hierarchy d5 to the hierarchy d1. It has been extracted.

  Here, out of the frequency components of the layers d1 to d5 obtained by multi-resolution decomposition of the temperature information S of the nasal projection, the frequency components of the layers d2 and d3 correspond to the heart rate from the temperature information S of the nasal projection. The frequency components of the frequency band are extracted. Therefore, the extraction function extracts the frequency components of the layers d2 and d3 from the frequency components of the layers d1 to d5. Then, the extraction function performs inverse wavelet transform on the frequency components of the layers d2 and d3, which are frequency bands corresponding to heartbeats, and reconverts the temperature information shown in the right diagram of FIG.

  The temperature information S ′ shown in the right diagram of FIG. 8 is temperature information obtained by extracting the frequency component of the frequency band corresponding to the heartbeat from the temperature information S of the nose protrusion shown in the left diagram of FIG. 8. By counting the number of peaks n in the temperature information S ′ for the measurement time t seconds by the measurement function, it is possible to appropriately measure the user's heart rate (beats / minute) based on the above equation (3).

  As described above, the frequency component of the frequency band corresponding to the heartbeat is extracted from the temperature information of the nasal projection using Fourier transform, bandpass filter processing, wavelet transform, and the user's heartbeat is based on the extracted frequency component. The number can be measured.

  As described above, in the present embodiment, the temperature data of the user's surface (skin) is detected by the infrared camera 10, and the heart rate of the user is measured based on the temperature information that is time-series data of the detected temperature data. Is done. Here, the surface (skin) temperature of a living body is affected by fluctuations in blood circulating in the body. That is, the surface (skin) temperature of the living body changes minutely due to the blood vessel contraction fluctuation. Therefore, by detecting the temperature data of the user's surface (skin) with the infrared camera 10 and extracting the contraction component of the blood vessel based on the temperature information obtained from the detected temperature data, without touching the user, The user's heart rate can be measured.

  In particular, in this embodiment, the frequency component of the frequency band corresponding to the heartbeat is extracted from the temperature information using Fourier transform, bandpass filter processing, or wavelet transform, and the user's heartbeat is based on the extracted frequency component. Measure the number. Thereby, the temperature fluctuation component (noise) which does not originate in a user's heartbeat can be removed from the temperature information based on the temperature data detected by the infrared camera 10, and the measurement accuracy of the user's heart rate can be improved.

  Furthermore, in this embodiment, a user's heart rate is measured based on the temperature data of a user's face part, especially a nose protrusion part. A nasal protrusion is a blood vessel anastomosis part, and is a site | part where a capillary vessel concentrates on the skin surface. Therefore, by detecting the temperature data of the user's nasal protrusion, it is possible to appropriately detect the temperature change due to the capillary blood vessel contraction fluctuation, and as a result, it is possible to appropriately measure the user's heart rate.

  In addition, in 1st Embodiment, although a user's heart rate is measured based on the temperature information of a user's nose protrusion part, for example, a pair of left and right earlobe parts or a pair of left and right cheek parts of a user The user's heart rate may be measured based on temperature information obtained from the temperature data of these parts. FIG. 9 is a diagram illustrating an example of temperature information acquired in the user's earlobe. FIG. 10 is a diagram showing frequency components obtained by transforming the temperature information of the user's earlobe shown in FIG. 9 by Fourier transform. For example, the temperature information of the user's earlobe can be acquired by detecting the temperature data of the user's earlobe for a predetermined time by the infrared camera 10 as shown in FIG. In the example shown in FIG. 10, the power vector intensity is maximum at a frequency f3 in a frequency band corresponding to a heartbeat, for example, a frequency band of 0.8 Hz to 1.5 Hz. Therefore, the measurement function measures the user's heart rate as a frequency f3 × 60 (beats / minute). For example, when the frequency f3 is 1.15 Hz, the user's heart rate is measured as 1.15 × 60 = 69 (beats / minute).

  The earlobe is a peripheral site where the capillaries are concentrated, and the contraction of the capillaries in the peripheral site gives a minute change to the body temperature change. Therefore, by detecting the temperature data of the earlobe, the contraction component of the blood vessel can be appropriately extracted as in the case of the nasal protrusion, and the heart rate of the user can be measured without touching the user. The cheek is also a part of the face where blood vessels are concentrated, and by detecting changes in the temperature of the cheek, it is possible to extract the blood vessel contraction component in the same way as the nasal protrusion or earlobe. The user's heart rate can be measured without touching the user.

<< Second Embodiment >>
Subsequently, a second embodiment of the present invention will be described. FIG. 11 is a flowchart showing a heart rate measurement process according to the second embodiment. In addition to the function of the controller 20 of the first embodiment, the controller 20 of the second embodiment uses the difference between the temperature data detected at the user's nose and the temperature data detected at the user's forehead as difference data. A temperature data calculation function for calculating is provided, and the configuration is the same as that of the first embodiment except that the user's heart rate is measured according to the flow chat shown in FIG. In the following, the heart rate measurement process according to the second embodiment will be described with reference to the flowchart shown in FIG. The heart rate measurement process shown in FIG. 11 is also executed by the controller 20 at regular time intervals.

  In step S201, temperature data detected in all the pixels of the infrared camera 10 is acquired as in step S101 of the first embodiment. And in step S202, each temperature data based on a user's forehead part is specified with each temperature data based on a user's nose protrusion part from the acquired temperature data. In addition, it does not specifically limit as a method of specifying a user's nose protrusion and forehead part, For example, similarly to step S102 of 1st Embodiment, a user's nasal cavity part is specified and based on a user's nasal cavity part. The user's nose and forehead may be specified.

In step S203, the average temperature calculation function, from the temperature data detected in nasal projection and forehead of the user, the average temperature data T _n and average temperature data T _h forehead portion of the nose projection is calculated.

In step S204, the temperature data calculation, the difference between the average temperature data T _n of the average temperature data T _h and the user's nose projection forehead portion is calculated as the difference data T _h -T _n. Next, in step S205, the difference data calculated in step S204 is stored in the RAM of the controller 20.

  In step S206 to step S209, processing similar to that in steps S105 to S108 of the first embodiment is performed. That is, the difference data stored along the time series is acquired as temperature information (step S206), and the acquired temperature information is converted into frequency components (step S207). Then, the frequency component in the frequency band corresponding to the heartbeat is extracted from the frequency components converted from the temperature information (step S208), and the heart rate of the user is measured based on the extracted frequency component (step S209). .

  As described above, in the second embodiment, difference data is calculated from the difference between the average temperature data of the nose and the average temperature data of the forehead, and the heart rate of the user is measured based on the difference data. The While the body temperature of a person may vary due to individual differences, illness, or rhythm of the sun, since the forehead is located near the brain, temperature fluctuations in the forehead are relatively stable. Therefore, in the second embodiment, by obtaining difference data between the temperature data of the forehead and the temperature data of the nose protrusion, in addition to the effects of the first embodiment, the temperature data detected in the nose protrusion of the user The influence of individual differences and state differences can be reduced, and the measurement accuracy of the user's heart rate can be improved.

  In the second embodiment, difference data is obtained from the difference between the temperature data of the user's nose and the temperature data of the forehead of the user, and the heart rate of the user is measured based on the difference data. However, in order to measure the heart rate of the user, for example, difference data may be obtained from the difference between the temperature data of the user's earlobe and the temperature data of the user's forehead, or the temperature data of the user's cheek You may obtain | require difference data from the difference with the temperature data of a user's forehead part.

<< Third Embodiment >>
Subsequently, a third embodiment of the present invention will be described. FIG. 12 is a flowchart showing a heart rate measurement process according to the third embodiment. In addition to the function of the controller 20 of the first embodiment, the controller 20 of the third embodiment includes average temperature data obtained at a plurality of parts of the user's nose projection, a pair of left and right earlobe, and a pair of left and right cheeks. A temperature data calculation function for calculating the sum as addition data is provided, and the configuration is the same as that of the first embodiment except that the user's heart rate is measured according to the flow chat shown in FIG. In the following, the heart rate measurement process according to the third embodiment will be described with reference to the flowchart shown in FIG. The heart rate measurement process shown in FIG. 12 is also executed by the controller 20 at regular time intervals.

  In step S301, temperature data detected in all pixels of the infrared camera 10 is acquired as in step S101 of the first embodiment. Next, in step S302, in addition to the temperature data based on the user's nasal protrusion, the temperature data based on the pair of left and right earlobes and the pair of left and right cheeks is specified among the acquired temperature data. In addition, the identification method of a user's nose protrusion part, a pair of left and right earlobe parts, and a pair of left and right cheek parts may use arbitrary methods, and is not specifically limited.

  In step S303, average temperature data in each part is calculated from each temperature data based on the user's nasal protrusion, a pair of left and right earlobe, and a pair of left and right cheeks. In step S <b> 304, addition data is calculated from the sum of the calculated average temperature data of the user's nasal protrusion, the pair of left and right earlobe, and the pair of left and right cheeks. In step S305, the calculated addition data is stored in the RAM of the controller 20.

  In steps S306 to S309, processing similar to that in steps S105 to S108 of the first embodiment is performed. That is, the addition data stored along the time series is acquired as temperature information (step S306), and the acquired temperature information is converted into frequency components (step S307). The frequency component in the frequency band corresponding to the heartbeat is extracted from the frequency components converted from the temperature information (step S308), and the heart rate of the user is measured based on the extracted frequency component (step S309). .

  As described above, in the third embodiment, the average temperature data of the user's nasal protrusion, the pair of left and right earlobes, and the pair of left and right cheeks is calculated and calculated from the sum of the calculated average temperature data of a plurality of parts. The heart rate of the user is measured based on the added data. As described above, by measuring the user's heart rate based on the addition data of the average temperature data of the user's nasal protrusion, the pair of left and right earlobes, and the pair of left and right cheeks, the effect of the first embodiment is achieved. In addition, it is possible to reduce the influence of frequency components (noise) that are not caused by heartbeats included in the temperature data detected in the respective parts of the user's nasal protrusion, the pair of left and right earlobes, and the pair of left and right cheeks. The frequency component corresponding to the user's heartbeat can be emphasized. As a result, the measurement accuracy of the user's heart rate can be improved.

  In addition, when adding the average temperature data of each part of the user by the temperature data calculation function, normalize the average temperature data of each part to be added, from the sum of the normalized average temperature data of each part, Additional data may be calculated.

<< Fourth Embodiment >>
Subsequently, a fourth embodiment of the present invention will be described. 13A and 13B are flowcharts showing a heart rate measurement process according to the fourth embodiment. The controller 20 of the fourth embodiment estimates the orientation of the user's face in addition to the function of the controller 20 of the first embodiment, and according to the estimated orientation of the user's face, Temperature for calculating addition data from the sum of the selection function for selecting a part for extracting frequency components from a pair of earlobe parts and a pair of left and right cheek parts, and the average temperature data for the part selected by the selection function A data calculation function is provided, and the configuration is the same as that of the first embodiment except that the user's heart rate is measured according to the flowcharts of FIGS. 13A and 13B. Hereinafter, the heart rate measurement process according to the fourth embodiment will be described with reference to the flowcharts shown in FIGS. 13A and 13B. The heart rate measurement process shown in FIGS. 13A and 13B is also executed by the controller 20 at regular time intervals.

  In step S401, temperature data detected in all pixels of the infrared camera 10 is acquired as in step S101 of the first embodiment. Next, in step S402, among the acquired temperature data, in addition to the temperature data based on the user's nasal protrusion, the temperature data based on the pair of left and right earlobes and the pair of left and right cheeks are specified. In step S403, average temperature data of each part is calculated from each temperature data based on the user's nose projection, a pair of left and right earlobe, and a pair of left and right cheeks. In the subsequent step S404, the calculated average temperature data of the user's nose protrusion, the pair of left and right earlobes, and the pair of left and right cheeks are stored in the RAM of the controller 20, respectively.

  In step S405, the orientation of the user's face is estimated by the selection function. The method for estimating the orientation of the user's face is not particularly limited. For example, the optical direction is calculated based on the difference between the image stored in the RAM when the user's face is facing the front and the current user's face image. By obtaining the flow, the orientation of the user's face can be estimated. Note that the determination of whether the user's face is facing the front may be, for example, estimated from the image of the user's face captured over a predetermined time as a front direction in which the user tends to be facing, The template may be prepared in advance, and the direction approximating this template may be estimated as the front.

  In step S406, it is determined by the selection function whether the estimated orientation of the user's face is downward. When it is determined that the orientation of the user's face is downward, the process proceeds to step S407, and the frequency component is selected from the user's nose protrusion, the pair of left and right earlobes, and the pair of left and right cheeks by the selection function. A pair of left and right earlobe parts is selected as a part for extracting. Then, the average temperature data of the selected pair of left and right earlobe parts is added by the temperature data calculation function. On the other hand, if it is determined in step S406 that the user's face is not facing downward, the process proceeds to step S408.

  In step S408, the selection function determines whether the estimated orientation of the user's face is rightward. If it is determined that the orientation of the user's face is rightward, the process proceeds to step S409, and the frequency component is selected from the user's nasal protrusion, the pair of left and right earlobes, and the pair of left and right cheeks by the selection function. The left cheek portion and the left ear buttock portion are selected as the parts for extracting. Then, the temperature data calculation function adds the average temperature data of the selected left cheek and left earlobe. On the other hand, if it is determined in step S408 that the user's face is not facing right, the process proceeds to step S410.

  In step S410, it is determined by the selection function whether the estimated orientation of the user's face is leftward. If it is determined that the orientation of the user's face is leftward, the process proceeds to step S411, and the frequency component is selected from the user's nasal protrusion, the pair of left and right earlobes, and the pair of left and right cheeks by the selection function. The right cheek and right ear buttock are selected as the parts for extracting Then, the temperature data calculation function adds the average temperature data of the selected right cheek and right earlobe. On the other hand, if it is determined in step S410 that the user's face is not facing left, the process proceeds to step S412.

  In step S <b> 412, since it is determined that the user's face direction is neither downward, rightward, or leftward, it is estimated that the user's face direction is front. And it progresses to step S413 and all the site | parts of a user's nose projection part, a pair of left and right earlobe parts, and a pair of left and right cheek parts are selected as a part for extracting a frequency component by a selection function. Then, the temperature data calculation function adds the average temperature data of all parts of the selected user's nose projection, left and right pair of earlobe, and left and right pair of cheeks.

  Note that the process of determining the user's face orientation from step S406 to step S413 may not be performed in this order. For example, it may be first determined whether the user's face orientation in step S412 is the front. .

  In step S414, the addition data calculated from the sum of the average temperature data of the parts selected according to the orientation of the user's face is stored in the RAM of the controller 20. In steps S415 to S418, processing similar to that in steps S105 to S108 of the first embodiment is performed. That is, the time series data of the addition data calculated from the sum of the temperature data of the parts selected according to the orientation of the user's face is acquired as temperature information (step S415), and the acquired temperature information is converted into frequency components. (Step S416). Then, the frequency component in the frequency band corresponding to the heartbeat is extracted from the frequency components converted from the temperature information (step S417), and the heart rate of the user is measured based on the extracted frequency component (step S418). .

  As described above, in the fourth embodiment, the orientation of the user's face is estimated, and according to the estimated orientation of the user's face, the user's nose projection, the pair of left and right earlobes, and the pair of left and right cheeks The part for extracting the frequency component is selected from among the above, and the addition data is calculated from the sum of the average temperature data of the selected part. Thus, in the fourth embodiment, by estimating the orientation of the user's face, in addition to the effects of the first embodiment, even if the user tilts his / her face, the surface of the user (skin) A site where temperature data can be appropriately detected is selected, and the heart rate of the user can be appropriately measured based on the temperature data of the selected site. For example, when the user turns the face to the right, the left earlobe part and the left cheek part are selected as the parts for extracting the frequency component, and the average temperature data of the selected left earlobe part and left cheek part is selected. By calculating the sum data from the sum, the temperature of the user's nose, the right earlobe, and the right cheek is detected by the infrared camera 10 installed in front of the user because the user faces the right direction. Even when data cannot be detected properly, the user's heart rate can be measured appropriately.

«Fifth embodiment»
Subsequently, a fifth embodiment of the present invention will be described. FIG. 14 is a flowchart showing a heart rate measurement process according to the fifth embodiment. In addition to the function of the controller 20 of the first embodiment, the controller 20 of the fifth embodiment divides the face into a plurality of cells and calculates a cell temperature for obtaining average temperature data of each cell. And a selection function for selecting a cell used for extraction of frequency components, and has the same configuration as that of the first embodiment except that the user's heart rate is measured according to the flowchart shown in FIG. Hereinafter, a heart rate measurement process according to the fifth embodiment will be described with reference to a flowchart shown in FIG. The heart rate measurement process shown in FIG. 14 is also executed by the controller 20 at regular time intervals.

  In step S501, temperature data detected by the infrared camera 10 is acquired as in step S101 of the first embodiment.

  In step S502, the user's face is identified from the thermal image captured by the infrared camera 10, and the identified user's face is divided into a plurality of cells by the cell temperature calculation function. Here, FIG. 15 is a diagram showing a user's face divided into a plurality of cells. In FIG. 15, similarly to FIG. 3, the temperature level is expressed by black and white shading. In the example illustrated in FIG. 15, the cell temperature calculation function divides the user's face by 6 × 7 cells. Each cell is divided into regions having the same size as other cells. The division of the user's face shown in FIG. 15 is an example, and the number of cells that divide the user's face is not particularly limited, and the cell size (number of pixels of each cell) is also particularly limited. Not.

  In step S503, a pixel that detects temperature data that is lower than a predetermined reference temperature by a predetermined temperature or higher is specified by the cell temperature calculation function. Thereby, the pixel which detected the temperature data which is not judged as the temperature of a person's surface (skin) can be specified. As shown in FIG. 15, when temperature data of a human face is detected by the infrared camera 10, for example, 36.5 degrees that is generally considered to be normal or close to human heat is set as the reference temperature. The cell temperature calculation function determines whether the temperature data detected in each pixel is lower than the reference temperature of 36.5 degrees by a predetermined temperature or more, and if the temperature is lower than the reference temperature of 36.5 degrees by a predetermined temperature or more. The temperature data detected by this pixel is determined not to be the human surface (skin) temperature. In step S504, average temperature data in each cell is calculated by removing the temperature data determined not to be the temperature of the human skin by the cell temperature calculation function. For example, as shown in FIG. 15, when the user wears spectacles, the temperature data detected in the pixel that images the spectacles is detected as a temperature that is lower than the reference temperature 36.5 degrees by a predetermined temperature or more, The average temperature data of each cell is calculated excluding the temperature data detected by the pixels that image the glasses.

  In step S505, the calculated average temperature data of each cell is stored in the RAM of the controller 20. In step S506, time series data of average temperature data of each cell is acquired as temperature information, and in step S507, the acquired temperature information of each cell is converted into a frequency component. Next, in step S508, the frequency at which the power spectrum intensity is maximized at a frequency of 0.8 to 1.5 Hz, which is a frequency band corresponding to the heartbeat, among the frequency components converted from the cell temperature information by the selection function. Is determined as the maximum intensity.

  In step S509, the top n cells having the largest maximum intensity are selected from all the cells by the selection function. Next, in step S510, frequency components in the frequency band corresponding to the heartbeat are extracted from the frequency components of the selected top n cells, and in step S511, the extracted frequency components of the top n cells are temperature information. Reconverted to

  In step S512, by adding the temperature data at each time point in the temperature information of the re-converted top n cells, time series data consisting of the sum of the temperature data of the top n cells is calculated as temperature information. Is done. In step S513, the user's heart rate is measured based on the calculated time-series data.

  As described above, in the fifth embodiment, the user's face is divided into a plurality of cells, and among the divided cells, the top n power spectrum intensity values in the frequency band corresponding to the heartbeat are large. The user's heart rate is measured based on the frequency component of the selected cell. Thus, in the fifth embodiment, since the cell in which the frequency component corresponding to the heartbeat is well detected among the frequency components of the plurality of cells can be selected and the heart rate of the user can be measured, In addition to the effects of the embodiment, the measurement accuracy of the user's heart rate can be improved.

  Furthermore, in the fifth embodiment, average temperature data of a cell is calculated by excluding temperature data that cannot be determined as a user's surface (skin) temperature from temperature data detected in a plurality of pixels in the cell. Thus, by calculating the average temperature data of the cell from only the temperature data determined to be the temperature of the user's surface (skin), excluding temperature data that cannot be determined as the user's surface (skin) temperature, for example, FIG. As shown in FIG. 15, even when the user wears an artificial object such as glasses on the user's face, the user's heart rate is appropriately measured based on the temperature data of the user's surface (skin). be able to.

<< Sixth Embodiment >>
Subsequently, a sixth embodiment of the present invention will be described. FIG. 16 is a flowchart showing a heart rate measurement process according to the sixth embodiment. In addition to the function of the controller 20 of the first embodiment, the controller 20 of the sixth embodiment divides the face into a plurality of cells and calculates the cell temperature and obtains the average temperature data of each cell, and the temperature of each cell. It has a selection function for judging the correlation of information and selecting a cell to be used for extraction of frequency components, and a temperature data calculation function for calculating addition data from the sum of the temperature data of the cells. According to the flowchart shown in FIG. Except for measuring the heart rate, the configuration is the same as that of the first embodiment. In the following, the heart rate measurement process according to the sixth embodiment will be described with reference to the flowchart shown in FIG. The heart rate measurement process shown in FIG. 16 is executed by the controller 20 at regular time intervals.

  In step S601, temperature data detected by the infrared camera 10 is acquired as in step S101 of the first embodiment.

  In step S602, the user's face is specified, and the user's face is divided into a plurality of cells as shown in FIG. 15 by the cell temperature calculation function. In step S603, the cell temperature calculation function detects temperature data that is lower than a predetermined reference temperature by a temperature that is lower than a predetermined reference temperature in order to identify a pixel that has detected temperature data that is not determined to be a human surface (skin) temperature. A pixel is identified. In the next step S604, the cell temperature calculation function calculates cell average temperature data excluding temperature data determined not to be a human surface (skin) temperature.

  In step S605, the average temperature data calculated in each cell is stored in the RAM of the controller 20. In step S606, time series data of average temperature data calculated in each cell is acquired as temperature information.

  In step S607, the correlation of the temperature information of each cell is determined by the selection function by an independent component analysis or a method in which independent component analysis is combined with principal component analysis. Then, the temperature information of the cell determined to have a low correlation is removed. In subsequent step S608, cell temperature information excluding cells determined to have low correlation is selected by the selection function. Then, the temperature data calculation function adds the temperature data at each time point in the selected temperature information to calculate the time series data of the added data.

  In steps S609 to S612, processing similar to that in steps S105 to S108 of the first embodiment is performed. That is, the time series data of the addition data is acquired as temperature information (step S609), and the temperature information is converted into a frequency component (step S610). The frequency component in the frequency band corresponding to the heartbeat is extracted from the frequency components converted from the temperature information (step S611), and the user's heart rate is measured based on the extracted frequency component (step S612). .

  As described above, in the sixth embodiment, a cell obtained by removing temperature information of a cell having low correlation from temperature information of a plurality of cells by independent component analysis or a method in which principal component analysis is combined with independent component analysis. Temperature information is selected, and the heart rate of the user is measured based on the temperature information of the selected cell. As a result, in the sixth embodiment, in addition to the effects of the first embodiment, it is possible to measure the user's heart rate by removing the temperature information of the cell having low correlation, and improve the measurement accuracy of the user's heart rate. Can be made.

<< Seventh Embodiment >>
Subsequently, a seventh embodiment of the present invention will be described. FIG. 17 is a diagram illustrating an installation example of the infrared camera according to the seventh embodiment, and FIG. 18 is a flowchart illustrating a heart rate measurement process according to the seventh embodiment. In the seventh embodiment, as shown in FIG. 17, infrared cameras 11 to 13 for detecting temperature data of the back of the user's hand are provided at positions on the right side, the left side, and the upper side of the user. 18 has the same configuration as that of the first embodiment except that the user's heart rate is measured according to the flowchart shown in FIG.

  As shown in FIG. 17, the infrared camera 11 installed at the right side of the user has the back of the user's left hand from the right side of the user, and the infrared camera 12 installed on the left of the user has the back of the user's right hand from the left side of the user. The infrared camera 13 installed on the upper side of the user images the back of the user's right hand and left hand from different positions from the position above the user's head. FIG. 19 is a diagram illustrating an example of a thermal image captured by the infrared camera 12 installed on the left side of the user illustrated in FIG. In FIG. 19, similarly to FIG. 3, the temperature level is expressed by black and white shading. As shown in FIG. 19, the infrared camera 12 installed on the left side of the user can detect the temperature data of the back of the user's left hand holding the handle.

  Next, a heart rate measurement process according to the seventh embodiment will be described with reference to the flowchart shown in FIG. The heart rate measurement process shown in FIG. 18 is executed by the controller 20 at regular time intervals.

  In step S701, for example, as shown in FIG. 19, temperature data detected in all the pixels of the respective infrared cameras 11 to 13 is acquired. In step S702, among the temperature data acquired from each of the infrared cameras 11 to 13, each temperature data based on the back of the user's hand is specified. In subsequent step S703, average temperature data of the back of the user's hand is calculated from each temperature data based on the back of the user's hand, and in step S704, the calculated average temperature data of the back of the user's hand is stored in the RAM of the controller 20. .

  In step S705, the average temperature data of the back of the user's hand stored along the time series is acquired as temperature information of the back of the user's hand. Here, since the user may move the user's hand to operate the handle, when acquiring the temperature information, for example, the ratio of the pixels that image the back of the user's hand according to the movement of the user's hand The temperature information of the back of the user's hand detected by the infrared camera having the highest value may be acquired, and addition data from the sum of the average temperature data of the back of the user's hand detected in each of the plurality of infrared cameras 11 to 13 And temperature information may be acquired based on the calculated addition data.

  In steps S706 to S708, processing similar to that in steps S106 to S108 of the first embodiment is performed. That is, the temperature information is converted into frequency components (step S706), and the frequency components in the frequency band corresponding to the heartbeat are extracted from the frequency components converted from the temperature information (step S707). Based on the extracted frequency components The user's heart rate is measured (step S708).

  As described above, in the seventh embodiment, the temperature data of the back of the user's hand is detected, and the heart rate of the user is measured based on the temperature information of the back of the user's hand obtained from the temperature data of the back of the user's hand. The back of the hand is a part where blood vessels concentrate, and the contraction component of the blood vessel can be extracted from the temperature information on the back of the user's hand. Therefore, by detecting the temperature of the back of the user's hand, the user's heart rate can be measured without touching the user.

  Furthermore, the back of the hand is easier to identify than a nose projection, a pair of earlobe, or a pair of cheeks in a human face. Therefore, the temperature data of the back of the user's hand can be easily compared with the case where the user's heart rate is measured based on the temperature data of the nose protrusion, the pair of earlobe, or the pair of cheeks in the human face. Based on the user's heart rate.

  In addition, the installation method of the infrared cameras 11-13 in 7th Embodiment is an example, You may install suitably. Also, the number of cameras to be installed is not limited. For example, infrared cameras may be installed only at the left side position of the user and the right side position of the user, or more than three infrared cameras may be installed. .

<< Eighth Embodiment >>
Next, an eighth embodiment of the present invention will be described. FIG. 20 is a configuration diagram showing a heart rate measurement system according to the eighth embodiment. FIG. 21 is a flowchart showing a heart rate measurement process according to the eighth embodiment. In the eighth embodiment, as shown in FIG. 20, the visible light camera 30 is provided at the front position of the user and images the face of the user. And in 8th Embodiment, in addition to the function of the controller 20 of 1st Embodiment, the position coordinate detection function which detects the position coordinate of the pixel of the visible light camera 30 which images the predetermined site | part of a user's face part, and detection With reference to the position coordinates of the pixels, a matching function for specifying which part of the user the temperature data detected in each pixel of the infrared camera 10 is based on, and calculation in a plurality of parts of the user A temperature data calculation function for calculating the sum of the average temperature data as addition data, and has the same configuration as that of the first embodiment except that the user's heart rate is measured according to the flowchart shown in FIG.

  The visible light camera 30 includes a plurality of pixels on an image pickup device and picks up a luminance image. Examples thereof include a charge coupled device (CCD) and a complementary metal oxide semiconductor (CMOS). The angle of view, the number of pixels, and the frame rate of the visible light camera 30 and the infrared camera 10 are set to be the same, and the imaging ranges and imaging times of the visible light camera 30 and the infrared camera 10 are substantially the same. Image information captured by each pixel of the visible light camera 30 is transmitted to the controller 20.

  Next, a heart rate measurement process according to the eighth embodiment will be described with reference to a flowchart shown in FIG. The heart rate measurement process shown in FIG. 21 is also executed by the controller 20 at regular time intervals.

  In step S801, image information captured by each pixel of the visible light camera 30 is acquired by the position coordinate detection function. In step S <b> 802, the position coordinate detection function identifies the user's nasal protrusion, the pair of left and right earlobes, and the pair of left and right cheeks based on the luminance image captured by the visible light camera 30. The method for specifying the user's part based on the luminance image captured by the visible light camera 30 is not particularly limited. For example, a paper (TF Cootes, G, J, Edwards, and CJ Taylor, Active Appearance Model , Proc. European Conf. On Computer Version, Vol.2, p484-498, (1998)) and FSE (Face Sensing Engine / http: //www.oki.com/jp/fse/: Oki Electric Industry And a method using a method developed by Co., Ltd.).

  22 and 23 are diagrams for explaining a method of specifying each part of the user using the luminance image captured by the visible light camera 30. FIG. FIG. 22 is a diagram showing a luminance image captured by the visible light camera 30, and the user's face feature points (portions enclosed by squares in the figure) and the user's nose bumps identified from the feature points , A pair of left and right earlobe portions, and a pair of left and right cheeks (portions indicated by oblique lines in the figure). FIG. 23 is a diagram illustrating a thermal image captured by the infrared camera 10, in which a user's nose projection, a pair of left and right earlobe portions, and a pair of left and right cheeks (portions indicated by diagonal lines in the drawing). It is shown. In FIG. 23, as in FIG. 3, the temperature level is represented by black and white shading. The position coordinate detection function detects facial feature points such as the user's nose, eyes, ears, and mouth as shown in FIG. 22 by processing the luminance image captured by the visible light camera 30 by the above-described method. be able to. The position coordinate detection function identifies the user's nose protrusion, the pair of left and right earlobes, and the pair of left and right cheeks based on the detected facial feature points such as the user's nose, eyes, ears, and mouth. To do. For example, the position coordinate detection function specifies the periphery of the center of the position specified as the user's nose as a nose protrusion, specifies the position between the eyes, nose and ears as the cheek, and One-third of the identified area is identified as the earlobe area. Then, the position coordinate detection function detects the position coordinates of the pixels of the visible light camera 30 that images the region identified as the user's nose projection, the pair of left and right earlobes, and the pair of left and right cheeks.

  In step S803, temperature data detected by the infrared camera 10 is acquired as in step S101 of the first embodiment. Note that step S801, step S802, and step S803 may be performed simultaneously, and step S803 may be performed prior to step S801 and step S802.

  In step S804, the position coordinates detected in step S802 are referred to by the matching function, and among the temperature data detected in the infrared camera 10, as shown in FIG. , And each temperature data based on a pair of left and right cheeks.

  In step S805, average temperature data of these parts is calculated from the temperature data of the user's nose protrusion, the pair of left and right earlobes, and the pair of left and right cheeks identified in step S804. In step S806, the temperature data calculation function calculates addition data from the sum of the calculated average temperature data of the nasal protrusion, earlobe, and cheek. Next, in step S807, the calculated addition data is stored in the RAM of the controller 20.

  In steps S808 to S811, processes similar to those in steps S105 to S108 of the first embodiment are performed. That is, the addition data stored along the time series is acquired as temperature information (step S808), and the acquired temperature information is converted into a frequency component (step S809). The frequency component in the frequency band corresponding to the heartbeat is extracted from the frequency components converted from the temperature information (step S810), and the user's heart rate is measured based on the extracted frequency component (step S811). .

  As described above, in the eighth embodiment, the range and time taken by the visible light camera 30 and the infrared camera 10 are substantially matched, and the user's nasal projection and earlobe from the luminance image captured by the visible light camera 30. By identifying the cheeks and referring to the position coordinates of the visible light camera 30 that images these parts, the temperature data detected by each pixel of the infrared camera 10 is the temperature data based on which part of the user Determine if it exists. Compared with the infrared camera 10, the visible light camera 30 is excellent in performance of specifying the user's part by following the movement of the user's body even when the user's body moves. Therefore, the temperature data detected by each pixel of the infrared camera 10 is obtained by referring to the position coordinates of the pixels that image the user's nose, earlobe, and cheeks identified from the luminance image of the visible light camera 30. By specifying which part of the user the temperature data is based on, in addition to the effect of the first embodiment, even if the user's body moves, the user's nose protrusion, a pair of left and right Temperature data based on the earlobe and the pair of left and right cheeks can be detected appropriately, and the heart rate of the user can be measured appropriately.

  The embodiment described above is described for facilitating the understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

  Further, the present invention is not limited to the above-described embodiment, and may be implemented by combining the above-described embodiments. When combining the embodiments, for example, it is preferable to select and combine the optimal embodiments according to the driving situation of the user.

  For example, if the temperature data cannot be detected in pixels of a certain percentage or more of the user's part due to the user's face tilting, moving hands, etc., the temperature data detected in that part is excluded and other parts Additional data may be calculated from the sum of detected temperature data, and the user's heart rate may be measured based on the calculated additional data. Thereby, even when the user's face is tilted or the hand moves, the user's heart rate can be appropriately measured based on the temperature information of the part where the temperature data is appropriately detected.

  In the above-described embodiment, the frequency band corresponding to the heartbeat is 0.8 to 1.5 Hz. However, the frequency band corresponding to the heartbeat is limited to 0.8 to 1.5 Hz. It is not a thing and can be set suitably.

  In the fifth embodiment and the sixth embodiment, the user's face is divided into a plurality of cells, and the user's heart rate is measured based on the temperature data detected in each cell. In the seventh embodiment, when detecting the temperature data of the back of the user's hand, the back of the user's hand is divided into a plurality of cells, the temperature data of each cell is detected, and based on the detected temperature data, the user You may measure your heart rate.

  The infrared cameras 10 to 13 of the above-described embodiments are the acquisition means of the present invention, the conversion function and the extraction function are the extraction means of the present invention, the measurement function is the measurement means of the present invention, and the temperature data calculation function is the present invention. The selection function corresponds to the selection means of the present invention, and the specific function corresponds to the specification means of the present invention.

10-13 ... Infrared camera 20 ... Controller 30 ... Visible light camera

Claims (13)

Obtaining means for obtaining temperature information of one or more parts of the living body;
Extraction means for extracting frequency data corresponding to frequency components in a frequency band corresponding to the heartbeat of the living body from the temperature information;
A heart rate measuring apparatus comprising: a measuring unit that measures the heart rate of the living body based on the frequency data extracted by the extracting unit. The heart rate measuring device according to claim 1,
The said acquisition means acquires the temperature information of the 1 or 2 or more site | part on the face part of the said biological body, The heart rate measuring apparatus characterized by the above-mentioned. The heart rate measuring device according to claim 1 or 2,
The apparatus further comprises calculation means for calculating addition data from a sum of temperature data of the one or more parts, and calculating temperature information used for extraction of the frequency data by the extraction means based on the addition data. Heart rate measuring device. The heart rate measuring device according to claim 3,
The said acquisition means acquires the temperature information of at least 1 site | part among the nose projection part of the said biological body, a pair of left and right earlobe parts, or a pair of left and right cheek parts, The heart rate measuring apparatus characterized by the above-mentioned. The heart rate measuring device according to claim 4,
The acquisition means further acquires temperature information of the forehead portion of the living body,
Based on temperature information of at least one portion of the living body's nasal protrusion, earlobe, or cheek and temperature information of the forehead of the living body, at least of the nasal protrusion, earlobe, or cheek Calculation means for calculating difference data from a difference between temperature data of one part and temperature data of the forehead of the living body, and calculating temperature information used for extraction of the frequency data by the extraction means based on the difference data The heart rate measuring device further comprising: The heart rate measuring device according to any one of claims 1 to 5,
The heart rate measuring apparatus according to claim 1, wherein the acquisition means acquires temperature information of the back of the living body. The heart rate measuring device according to any one of claims 1 to 6,
The heart rate measuring apparatus, wherein the extraction means extracts the frequency data from the temperature information by Fourier transform, band pass filter processing, or wavelet transform. The heart rate measuring device according to claim 1,
A heart rate measuring apparatus, further comprising a selection unit that selects temperature information used for extraction of the frequency data by the extraction unit from among temperature information of one or more parts acquired by the acquisition unit. . The heart rate measuring device according to claim 8,
The selection means uses temperature information for extracting the frequency data by the extraction means from temperature information of each part of the living body's nasal protrusion, left and right earlobe, or left and right cheeks according to the orientation of the face. A heart rate measuring device characterized by selecting. The heart rate measuring device according to claim 8 or 9,
When the selection means selects temperature information used for extraction of the frequency data by the extraction means from the temperature information of one or more parts acquired by the acquisition means, the correlation with the heart rate of the living body is A heart rate measuring apparatus that performs the selection by excluding temperature information of a part determined to be low. The heart rate measuring device according to any one of claims 1 to 10,
The heart rate measuring apparatus further comprising a specifying unit for specifying the part. The heart rate measuring device according to claim 11,
The heart rate measuring apparatus characterized in that the specifying means specifies the part based on image information captured by a visible light camera.   Extracting a frequency component of a frequency band corresponding to the heartbeat of the living body from temperature information of one or more parts of the living body, and measuring the heart rate of the living body based on the frequency component Measuring method.

Images