JP2018051288A - Pulse wave measurement device and pulse wave measurement method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a further improved pulse wave measurement device.SOLUTION: A pulse wave measurement device 10 is provided with a processor and a memory. The processor outputs a command to an externally provided lighting device so that an amplitude of first hue waveforms obtained from a plurality of first visible light images belongs to a prescribed hue range, calculates a degree of correlation between a first visible light waveform obtained from a plurality of visible light images and a first infrared light waveform obtained from a plurality of first infrared light images, outputs an infrared light control signal controlling the light volume of the infrared light of an infrared light source 123 in accordance with the calculated degree of correlation, outputs a visible light control signal controlling the light volume of the lighting device 30, extracts a second visible light waveform and a second infrared light waveform each from a plurality of acquired third visible light images and a plurality of acquired second infrared light images, calculates first biological information from at least one of the feature amount of the second visible light waveform and the feature amount of the second infrared light waveform, and outputs the calculated first biological information.SELECTED DRAWING: Figure 7

Description

  The present disclosure relates to a pulse wave measurement device and a pulse wave measurement method for measuring a person's pulse wave in a non-contact manner.

  Patent Document 1 discloses a technique for measuring heartbeat and sleep depth in a non-contact state using millimeter waves, visible light, infrared light, and the like.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for satisfactorily switching from an infrared photographing mode in which infrared light is irradiated to a subject to a normal photographing mode in a photographing apparatus.

JP2013-192620A JP 2004-146873 A JP 2007-130182 A

  However, the techniques disclosed in Patent Document 1 and Patent Document 2 require further improvement.

  A pulse wave measurement device according to an aspect of the present disclosure is a pulse wave measurement device including a processor, and the processor defines a first correspondence relationship from an illumination device provided outside the pulse wave measurement device. A first control pattern is acquired, and the first correspondence relationship indicates a color temperature of visible light output from the illumination device corresponding to each of a plurality of instructions, and indicates a first color temperature held by the pulse wave measurement device. A first instruction corresponding to the information is determined with reference to the first control pattern, the first instruction is output to the illumination device, and the visible light having a color temperature corresponding to the first instruction is output by the illumination device. The first irradiated light is obtained by imaging the user irradiated with the first visible light image, obtaining a plurality of first visible light images, calculating a first plurality of hues from the plurality of first visible light images, and the first plurality of hues. A first hue waveform is extracted from When the amplitude of the hue waveform does not belong to a predetermined hue range, a second instruction corresponding to a second color temperature different from the first color temperature is determined with reference to the first control pattern, and the second instruction is determined. Outputting to the illumination device, and capturing the user irradiated with visible light having a color temperature corresponding to the second instruction by the illumination device in the visible light region to obtain a plurality of second visible light images; A second plurality of hues are calculated from the plurality of second visible light images, a second hue waveform is extracted from the second plurality of hues, and an amplitude of the second hue waveform belongs to the predetermined hue range In this case, a first process is performed, and the first process acquires a plurality of first infrared light images obtained by imaging the user irradiated with infrared light from an infrared light source in an infrared light region. From the acquired second visible light images, the previous A first visible light waveform which is a waveform indicating a user's pulse wave is extracted, and a first infrared light waveform which is a waveform indicating the user's pulse wave is extracted from the acquired first infrared light images. The degree of correlation between the extracted first visible light waveform and the extracted first infrared light waveform is calculated, and the amount of infrared light emitted from the infrared light source is calculated according to the degree of correlation. An infrared light control signal to be controlled is output to the infrared light source, and a visible light control signal for controlling the amount of visible light emitted from the lighting device is output to the lighting device according to the degree of correlation. A plurality of third visible light images obtained by imaging in a visible light region a user irradiated with visible light based on the visible light control signal by an apparatus, and the infrared light control signal by an infrared light source The user who has been irradiated with infrared light based on Acquiring a plurality of second infrared light images obtained by imaging, extracting a second visible light waveform, which is a waveform indicating the pulse wave of the user, from the acquired plurality of third visible light images; A second infrared light waveform, which is a waveform indicating the pulse wave of the user, is extracted from the acquired second infrared light images, and the feature amount of the second visible light waveform and the second infrared light waveform are extracted. Calculating the first biological information from at least one of the feature amounts, and outputting the calculated first biological information.

  This comprehensive or specific aspect may be realized by a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium, and any of an apparatus, a system, a method, an integrated circuit, a computer program, and a recording medium. It may be realized by various combinations. The computer-readable recording medium includes a non-volatile recording medium such as a CD-ROM (Compact Disc-Read Only Memory).

  According to the present disclosure, further improvements can be realized. Additional benefits and advantages of one aspect of the present disclosure will become apparent from the specification and drawings. This benefit and / or advantage may be provided individually by the various aspects and features disclosed in the specification and drawings, and not all are required to obtain one or more thereof.

FIG. 1 is a schematic diagram showing how a pulse wave measurement system according to the present embodiment is used by a user. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the pulse wave measuring device 10. FIG. 3 is a block diagram illustrating an example of a hardware configuration of the illumination device 30 according to the embodiment. FIG. 4 is a block diagram illustrating an example of a hardware configuration of the mobile terminal according to the first embodiment. FIG. 5 is a diagram for explaining an example of use of the pulse wave measuring apparatus. FIG. 6 is a diagram for explaining an example of use of the pulse wave measurement device. FIG. 7 is a block diagram illustrating an example of a functional configuration of the pulse wave measurement device according to the present embodiment. FIG. 8 is a graph showing an example of the luminance change of the visible light image and the infrared light image in the present embodiment. FIG. 9 is a graph showing an example of calculation of pulse wave timing in the present embodiment. FIG. 10 is a graph showing an example of heartbeat interval times acquired in time series. FIG. 11 is a graph for explaining a method of extracting an inflection point from a pulse wave. FIG. 12 is a graph showing a visible light waveform for explaining a method of calculating the slope between the vertex and the bottom point in the visible light waveform. FIG. 13 is a graph showing an infrared light waveform when a human skin image is acquired by an infrared light camera for each level of the light amount of the infrared light source. FIG. 14 is a graph showing the first heartbeat interval time and the second heartbeat interval time plotted with data in time series order. FIG. 15 is a diagram for describing a specific example of determining whether or not the heartbeat interval time is appropriate. FIG. 16 is a diagram for explaining an example in the case where excessive acquisition of peak points is performed in the visible light waveform and no excessive acquisition of peak points is performed in the corresponding infrared light waveform. FIG. 17 is a diagram for explaining a case where the degree of correlation is calculated using inflection points. FIG. 18 is a diagram for explaining an example that does not apply to the condition that the number of peak points in the first predetermined period exceeds the first threshold even though the number of peak points is excessive. FIG. 19 is a diagram illustrating an example for explaining that the peak point acquired during the adjustment of the light amount of the light source is not used for calculating the degree of correlation between the visible light waveform and the infrared light waveform. FIG. 20 is a diagram showing an example of the simplest steps for decreasing the light amount of the visible light source to zero and increasing the light amount of the infrared light source to an appropriate light amount using the pulse wave measurement device. is there. FIG. 21 illustrates that the light source control is waited until two or more predetermined feature points continuous from the waveform are extracted within the second predetermined period in each of the visible light waveform and the infrared light waveform. FIG. FIG. 22 is a diagram for explaining a difference in how a user's face is seen in the visible light imaging unit 122 due to a change in color temperature. FIG. 23 is a diagram for explaining calculation processing for calculating a hue signal of hue H from RGB luminance signals. FIG. 24 is a diagram for explaining the hue circle. FIG. 25 is a diagram illustrating a hue waveform acquired when converted to a different hue range. FIG. 26 shows a light source that reduces the light amount of the visible light source when the illumination device is a device that is dimmed by the second control pattern until it becomes zero, and increases the light amount of the infrared light source to an appropriate light amount. It is a figure for demonstrating switching control of. FIG. 27 is a diagram for describing light source switching control when the illumination device is a device that is dimmed according to the third control pattern. FIG. 28 is a diagram for describing an example of light source switching control when the illumination device is a device that is dimmed according to the fourth control pattern. FIG. 29 is a diagram illustrating an example of switching control to turn off when the illuminance of the lighting device is a predetermined threshold. FIG. 30 is a diagram illustrating an example when switching control is performed in a shortened completion time. FIG. 31 is a diagram illustrating a display example on the presentation device. FIG. 32 is a flowchart showing the flow of processing of the pulse wave measuring apparatus in the present embodiment. FIG. 33 is a flowchart showing details of peak point excess acquisition determination processing in the present embodiment. FIG. 34 is a flowchart showing details of correlation degree calculation processing in the present embodiment. FIG. 35 is a flowchart showing details of light amount adjustment processing in the present embodiment. FIG. 36 is a flowchart of the control pattern recognition process in the modification.

(Knowledge that became the basis of this disclosure)
The inventor has found that the following problems occur with respect to the technique described in the “Background Art” column.

  Patent Document 1 does not disclose adjustment of the light amount of an infrared light source when acquiring a pulse wave in a dark room, so that there is a problem that it is difficult to measure heartbeats and pulse waves in a dark room without contact.

  Further, in Patent Document 2, mode switching is used by using the ratio between the luminance of visible light and the luminance of infrared light. However, when the mode switching is applied to pulse wave measurement in a dark room, the ratio of luminance is used. There is a problem that the pulse wave cannot be easily measured by the switching by.

  Therefore, the present disclosure provides a pulse wave measuring device and the like that can accurately measure a pulse wave in a dark room.

  A pulse wave measurement device according to an aspect of the present disclosure is a pulse wave measurement device including a processor, and the processor defines a first correspondence relationship from an illumination device provided outside the pulse wave measurement device. A first control pattern is acquired, and the first correspondence relationship indicates a color temperature of visible light output from the illumination device corresponding to each of a plurality of instructions, and indicates a first color temperature held by the pulse wave measurement device. A first instruction corresponding to the information is determined with reference to the first control pattern, the first instruction is output to the illumination device, and the visible light having a color temperature corresponding to the first instruction is output by the illumination device. The first irradiated light is obtained by imaging the user irradiated with the first visible light image, obtaining a plurality of first visible light images, calculating a first plurality of hues from the plurality of first visible light images, and the first plurality of hues. A first hue waveform is extracted from When the amplitude of the hue waveform does not belong to a predetermined hue range, a second instruction corresponding to a second color temperature different from the first color temperature is determined with reference to the first control pattern, and the second instruction is determined. Outputting to the illumination device, and capturing the user irradiated with visible light having a color temperature corresponding to the second instruction by the illumination device in the visible light region to obtain a plurality of second visible light images; A second plurality of hues are calculated from the plurality of second visible light images, a second hue waveform is extracted from the second plurality of hues, and an amplitude of the second hue waveform belongs to the predetermined hue range In this case, a first process is performed, and the first process acquires a plurality of first infrared light images obtained by imaging the user irradiated with infrared light from an infrared light source in an infrared light region. From the acquired second visible light images, the previous A first visible light waveform which is a waveform indicating a user's pulse wave is extracted, and a first infrared light waveform which is a waveform indicating the user's pulse wave is extracted from the acquired first infrared light images. The degree of correlation between the extracted first visible light waveform and the extracted first infrared light waveform is calculated, and the amount of infrared light emitted from the infrared light source is calculated according to the degree of correlation. An infrared light control signal to be controlled is output to the infrared light source, and a visible light control signal for controlling the amount of visible light emitted from the lighting device is output to the lighting device according to the degree of correlation. A plurality of third visible light images obtained by imaging in a visible light region a user irradiated with visible light based on the visible light control signal by an apparatus, and the infrared light control signal by an infrared light source The user who has been irradiated with infrared light based on Acquiring a plurality of second infrared light images obtained by imaging, extracting a second visible light waveform, which is a waveform indicating the pulse wave of the user, from the acquired plurality of third visible light images; A second infrared light waveform, which is a waveform indicating the pulse wave of the user, is extracted from the acquired second infrared light images, and the feature amount of the second visible light waveform and the second infrared light waveform are extracted. Calculating the first biological information from at least one of the feature amounts, and outputting the calculated first biological information.

  According to this, the color temperature of the illumination device provided outside is adjusted and the color temperature of the illumination device is adjusted so that the amplitude of the hue waveform obtained from the plurality of first visible light images belongs to the predetermined hue range. After that, the user's pulse wave is extracted from the plurality of second visible light images and the plurality of first infrared light images obtained. Therefore, it is possible to obtain clear first hue waveform and second hue waveform that are hardly affected by noise due to luminance change.

  According to this, the degree of correlation between the first visible light waveform obtained from the plurality of second visible light images and the first infrared light waveform obtained from the plurality of first infrared light images is calculated. The light quantity of the illumination device and the light quantity of the infrared light emitted from the infrared light source are controlled according to the degree of correlation. Therefore, for example, even when a commercially available lighting device is used, adjustment of the amount of visible light and adjustment of the amount of infrared light can be appropriately performed, and biological information can be calculated with high accuracy.

  Further, the predetermined hue range may be a range where the hue is 0 degree or more and 60 degrees or less. The predetermined hue range may be a hue range based on a hue of 30 degrees.

  In this way, the color of the user's skin surface changes from white to reddish, and in particular, by changing the color temperature of the lighting device so that the value of the hue H is, for example, around 30 degrees, more body movement and environment The first hue waveform and the second hue waveform can be acquired robustly against noise. Therefore, it is possible to obtain clear first hue waveform and second hue waveform that are hardly affected by noise due to luminance change.

  In the calculation of the correlation, the processor extracts (1) a plurality of first peak points in a plurality of first unit times included in the plurality of first unit waveforms, and the plurality of first peak points are , A plurality of first maximum value points included in the plurality of first unit waveforms or a plurality of first minimum value points included in the plurality of first unit waveforms, and the first visible light waveform is the plurality of the plurality of first visible light waveforms. The plurality of first maximum value points correspond to the plurality of first unit waveforms, respectively, and the plurality of first minimum value points and the plurality of first unit waveforms correspond to each other. The plurality of first unit waveforms and the plurality of first unit times correspond to each other, and (2) a plurality of second peak points in a plurality of second unit times included in the plurality of second unit waveforms. And the plurality of second peak points are the plurality of second units. A plurality of second maximum value points included in the shape or a plurality of second minimum value points included in the plurality of second unit waveforms, and the first infrared light waveform includes the plurality of second unit waveforms. The plurality of second maximum value points and the plurality of second unit waveforms respectively correspond to each other, the plurality of second minimum value points and the plurality of second unit waveforms correspond to each other, and The second unit waveform corresponds to the plurality of second unit times, and (3) calculates a plurality of first heart beat interval times based on the plurality of first unit times, and the plurality of first heart beat intervals. The time is a time between a first time and a second time, the plurality of first unit times includes the first time and the second time, and the times included in the plurality of first unit times are It does not exist between the first time and the second time, and (4) based on the plurality of second unit times. A plurality of second heartbeat interval times, wherein the plurality of second heartbeat interval times is a time between a third time and a fourth time, and the plurality of second unit times are the third time and the second time The time included in the plurality of second unit times does not exist between the third time and the fourth time, and the correlation degree is calculated using the following (Equation 1). May be calculated.

  For this reason, the correlation between the first visible light waveform and the first infrared light waveform can be calculated.

  The processor calculates second biological information from at least one of the feature amount of the first visible light waveform and the feature amount of the first infrared light waveform, and outputs the calculated second biological information. Also good.

  Therefore, the second biological information is calculated from at least one of the feature amount of the first visible light waveform and the feature amount of the first infrared light waveform acquired before the adjustment of the light amount of visible light or infrared light, The calculated second biological information can be output.

  Further, in the case where the lighting device is a device that is further dimmed by a second control pattern that adjusts the light amount in one step of on / off, the processor is configured to output the infrared light in the output of the infrared light control signal. A control signal for increasing the amount of infrared light emitted from the light source by a predetermined first change amount is output as the infrared light control signal to the infrared light source, and in the output of the visible light control signal, A control signal for turning off the lighting device may be output to the lighting device as the visible light control signal.

  For this reason, even if the illuminating device is an illuminating device that adjusts the amount of light in one step, it is possible to appropriately adjust the amount of visible light and the amount of infrared light.

  Further, the processor is configured such that the lighting device further performs dimming with a third control pattern in which the light amount is adjusted in two steps, that is, a first visible light amount and a second visible light amount that is smaller than the first visible light amount. When the infrared light control signal is output, the amount of infrared light emitted from the infrared light source is increased by a predetermined second change amount from the first infrared light amount. A control signal for controlling the amount of infrared light is output to the infrared light source as the infrared light control signal, and a control signal for changing the illumination device from the first visible light amount to the second visible light amount is provided. , Output to the illumination device as the visible light control signal, the luminance change of the infrared light obtained from the first and second infrared light images, and the visible light obtained from the second and third visible light images Depending on the brightness change, the amount of infrared light A control signal for controlling the third infrared light amount, which is increased from the second infrared light amount by the determined third change amount, as the infrared light control signal. A second-stage control signal that is output to an external light source and turns off the lighting device may be output to the lighting device as the visible light control signal.

  According to this, when the illuminating device is an illuminating device that adjusts the amount of light in two stages, the pulse wave measuring device acquires the amount of decrease in the luminance of visible light in the first step dimming, Accordingly, the infrared light waveform can be acquired more effectively by increasing the light amount of the infrared light source.

  Further, in the case where the lighting device is a device that is further dimmed by a fourth control pattern that adjusts the amount of light in a stepless manner, and the calculated degree of correlation is equal to or greater than a predetermined threshold, the processor In the output of the external light control signal, a control signal for increasing the amount of infrared light in the infrared light source is output to the infrared light source as the infrared light control signal, and in the output of the visible light control signal, A control signal for reducing the amount of visible light in the illumination device is output to the illumination device as the visible light control signal, the third visible light image is acquired, the second visible light waveform is extracted, and the second infrared light is output. After the acquisition of the optical image and the extraction of the second infrared light waveform, the calculation of the degree of correlation is further repeated so that the amount of light of the illumination device is equal to or less than a second threshold value and the phase is repeatedly performed. Whenever the result of the operation of, when the correlation is not less than the predetermined threshold value, a control signal to turn off the lighting device, may be output to the illuminating device as the visible light control signal.

  According to this, compared with the case where it reduces linearly until it turns off visible light, it can be turned off earlier and can introduce into comfortable sleep.

  In addition, in the case where the lighting device is a device that is dimmed by a fourth control pattern that adjusts the light amount in a stepless manner, (i) when the calculated degree of correlation is equal to or greater than a predetermined threshold, In the output of the infrared light control signal, a control signal for increasing the amount of infrared light in the infrared light source at a first speed is output to the infrared light source as the infrared light control signal, and the visible light control is performed. In the output of the signal, a control signal for reducing the amount of visible light in the illumination device at a second speed is output to the illumination device as the visible light control signal, the acquisition of the third visible light image, and the second visible light After the extraction of the waveform, the acquisition of the second infrared light image, and the extraction of the second infrared light waveform, a normal process for repeatedly calculating the correlation degree; and (ii) the calculated correlation degree More than a predetermined threshold In this case, in the output of the infrared light control signal, a control signal for increasing the amount of infrared light in the infrared light source at a third speed that is twice or more faster than the first speed is used as the infrared light control signal. A control signal that is output to the infrared light source and that reduces the amount of visible light in the illumination device at a fourth speed that is twice or more faster than the second speed in the output of the visible light control signal. Output to the illumination device as a signal, after acquisition of the third visible light image, extraction of the second visible light waveform, acquisition of the second infrared light image, and extraction of the second infrared light waveform Further, any one of the short-time processing for repeatedly calculating the correlation degree may be executed.

  For this reason, the time required for switching control can be shortened.

  These general or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM. The system, method, integrated circuit, computer program And any combination of recording media.

(Embodiment)
In the present embodiment, a pulse wave measurement that acquires a user's pulse wave from each of the user's visible light image and infrared light image and controls the light source based on the degree of correlation between the acquired two pulse wave feature quantities The apparatus will be described.

[1-1. Constitution]
[1-1-1. Pulse wave measurement system]
The configuration of the pulse wave measurement system according to the present embodiment will be described.

  FIG. 1 is a schematic diagram illustrating a state in which the pulse wave measurement system 1 according to the present embodiment is used by a user U. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the pulse wave measuring device 10.

  The pulse wave measurement system 1 includes a pulse wave measurement device 10 and an illumination device 30. The pulse wave measurement system 1 may be further configured by a mobile terminal 200. The pulse wave measuring device 10, the illumination device 30, and the mobile terminal 200 are connected to be communicable with each other.

  The pulse wave measurement device 10 includes a visible light camera 22, an infrared light LED 23, an infrared light camera 24, and a pulse wave calculation device 100. The pulse wave measurement device 10 may be configured to include the pulse wave calculation device 100.

  As shown in FIG. 2, the pulse wave measuring device 10 has a housing 20, and each component shown in FIG. 2 is arranged on a surface (for example, a lower surface) on the light irradiation side of the housing 20. . Specifically, in the pulse wave measuring device 10, for example, a visible light camera 22, an infrared light LED (Light Emitting Diode) 23, and an infrared light camera 24 are arranged side by side on the upper side surface of the housing 20. ing. The pulse wave measuring device 10 acquires a user's pulse wave using images captured by the visible light camera 22 and the infrared light camera 24, and performs illumination based on the degree of correlation between the two acquired pulse waves. A pulse wave arithmetic device 100 that controls the light amount of the device 30 and the infrared LED 23 is provided.

The visible light camera 22 is a camera that captures visible light. The visible light camera 22 is, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor Imager).
This is a camera provided with an image sensor such as Sensor. The visible light camera 22 applies RGB color filters to the image sensor, so that visible light, that is, light existing in a wavelength region of 400 to 800 nm is applied to the image sensor by RGB (Red, Green, Blue). These three types of signals are acquired.

The infrared light LED 23 is a light source that emits infrared light. Infrared light is light in the wavelength region of the infrared light region (for example, 800 to 2500 nm). The infrared LED 23 may be composed of a plurality of bullet-type LEDs, or a plurality of surface mount types (SMD: Surface Mount).
(Device) LED may be used, or COB (Chip On Board) type LED may be used. The infrared LED 23 may be composed of a plurality of LEDs.

  The infrared light camera 24 is a camera that captures infrared light. The infrared light camera 24 may be a camera that captures electromagnetic waves in a wavelength region (for example, 700 nm to 900 nm) including a part of the visible light region. The infrared camera 24 is disposed at a position adjacent to the infrared LED 23. The infrared light camera 24 includes a filter different from that of the visible light camera 22 so that infrared light, that is, light existing in a wavelength region of 800 nm or more is acquired by the image sensor as one type of monochrome signal. .

  The pulse wave calculation device 100 is disposed inside the housing 20. The pulse wave computing device 100 includes a CPU (Central Processing Unit) 101, a main memory 102, a storage 103, and a communication IF (Interface) 104.

  The CPU 101 is a processor that executes a control program stored in the storage 103 or the like.

  The main memory 102 is a volatile storage area (main storage device) used as a work area used when the CPU 101 executes a control program.

  The storage 103 is a non-volatile storage area (auxiliary storage device) that holds control programs, various data, and the like.

  The communication IF 104 is a communication interface that transmits and receives data to and from other devices via a network. Specifically, the communication IF 104 outputs a control signal for controlling these devices to the illumination device 30, the visible light camera 22, the infrared light LED 23, and the infrared light camera 24. Further, the communication IF 104 acquires image data captured by each of the visible light camera 22 and the infrared light camera 24.

  The communication IF 104 may be a communication interface that transmits a control signal to the illumination device 30. Specifically, the communication IF 104 may be a communication interface that transmits a control signal to the lighting device 30 by infrared rays.

  The communication IF 104 may be a communication interface that can be connected to the mobile terminal 200 for communication. Specifically, the communication IF 104 may be a wireless LAN (Local Area Network) interface that conforms to the IEEE 802.11a, b, g, and n standards, or a wireless communication interface that conforms to the Bluetooth (registered trademark) standards. There may be.

[1-1-2. Lighting device]
A hardware configuration of the lighting device 30 will be described with reference to FIG.

  FIG. 3 is a block diagram illustrating an example of a hardware configuration of the illumination device 30 according to the embodiment.

  The illumination device 30 is a light source that emits visible light, and includes a visible light LED 31 and a controller 32. The illumination device 30 is a device that receives a predetermined control signal transmitted from a remote controller or the like and irradiates light of a light amount corresponding to the predetermined control signal. The lighting device 30 may be, for example, a commercially available ceiling light, pendant light, bracket light, stand light, foot light, spot light, down light, or the like, and receives a control signal from a remote controller. It may be a device such as an LED bulb, a straight tube LED lamp, or a round (ring-shaped) LED lamp configured so as to be able to perform the above.

  The visible light LED 31 is, for example, a white LED. Visible light is light in the wavelength region of the visible light region (for example, 400 to 800 nm). The visible light LED 31 is arranged in an annular shape on the lower surface of the housing, for example. The visible light LED 31 may be composed of a plurality of bullet-type LEDs, a plurality of surface-mount type (SMD) LEDs, or a COB (Chip On Board) type. The LED may be configured. Further, the visible light LED 31 may not be arranged in an annular shape. Note that the lighting device may be a device having a fluorescent light, a bulb-type fluorescent light, a light bulb, or the like as a light source instead of the visible light LED 31.

  The controller 32 receives a control signal transmitted from a predetermined remote controller, the pulse wave measuring device 10 or the portable terminal 200, and adjusts the light amount of the visible light LED 31 according to the received control signal. The controller 32 is realized by, for example, a microcontroller and a communication module. The communication module may receive an infrared control signal, a wireless LAN control signal, or a Bluetooth (registered trademark) control signal.

[1-1-3. Mobile device]
A hardware configuration of the mobile terminal 200 will be described with reference to FIG.

  FIG. 4 is a block diagram illustrating an example of a hardware configuration of the mobile terminal according to the first embodiment.

  As illustrated in FIG. 4, the mobile terminal 200 includes a CPU 201, a main memory 202, a storage 203, a display 204, a communication IF 205, and an input IF 206. The mobile terminal 200 is a communicable information terminal such as a smartphone or a tablet terminal.

  The CPU 201 is a processor that executes a control program stored in the storage 203 or the like.

  The main memory 202 is a volatile storage area (main storage device) used as a work area used when the CPU 201 executes a control program.

  The storage 203 is a non-volatile storage area (auxiliary storage device) that holds control programs, various data, and the like.

  A display 204 is a display device that displays a processing result in the CPU 201. The display 204 is, for example, a liquid crystal display or an organic EL display.

  The communication IF 205 is a communication interface that communicates with the pulse wave measurement device 10. The communication IF 205 may be, for example, a wireless LAN (Local Area Network) interface that conforms to the IEEE 802.11a, b, g, or n standard, or a wireless communication interface that conforms to the Bluetooth (registered trademark) standard. Good. In addition, the communication IF 205 is a radio that conforms to a communication standard used in a mobile communication system such as the third generation mobile communication system (3G), the fourth generation mobile communication system (4G), or LTE (registered trademark). It may be a communication interface.

  The input IF 206 is, for example, a touch panel that is arranged on the surface of the display 204 and receives input from the user to a UI (User Interface) displayed on the display 204. The input IF 206 may be an input device such as a numeric keypad or a keyboard.

  5 and 6 are diagrams for explaining an example of use of the pulse wave measuring device 10.

  For example, as illustrated in FIG. 5, the mobile terminal 200 may display a UI for operating the pulse wave measurement device 10 on the display 204. Further, the mobile terminal 200 may transmit a control signal to the pulse wave measuring device 10 in response to an input to the UI.

  In the pulse wave measurement system 1, the mobile terminal 200 can be used as a means for the user to switch on / off the illumination device 30 and the infrared LED 23. For example, the mobile terminal 200 can be used as a remote controller for the pulse wave measurement device 10 and the illumination device 30 by starting a remote control application for controlling the pulse wave measurement device 10 on the mobile terminal 200. As shown in FIG. 5A, the user can turn on the lighting device 30 by selecting “lighting ON”.

  FIG. 6A is an image example when the illumination device 30 is ON. When the user selects “infrared ON”, the infrared light source can be turned on regardless of whether the illumination device 30 is on or off. For example, FIG. 6B shows a state where the illumination device 30 is OFF but the infrared LED 23 is ON. In the case of infrared illumination, since the user is not dazzled, the user can sleep as usual. Further, when the user selects “OFF”, both the illumination device 30 and the infrared LED 23 are turned off, and the user is not irradiated with any light.

  Then, in the UI shown in FIG. 5B, when the user selects the “normal mode”, the light quantity of the lighting device 30 is gradually increased from the state where the lighting device 30 is ON and the infrared LED 23 is OFF. The infrared light LED 23 is turned on and the light amount is gradually increased to determine the optimum light amount of the infrared light LED 23, and even if the user is sleeping, the pulse wave Will be able to get.

  In addition, when the user selects the “short-time mode”, the speed at which the light amount of the illumination device 30 is decreased is more than twice as high as the difference in which the user selects the “normal mode”, and the light amount of the infrared LED 23 is increased. Increase the speed to increase more than twice. Thereby, the period when the illuminating device 30 is ON can be shortened rather than the case of normal mode. Detailed description of the time reduction mode will be described later.

[1-2. Functional configuration]
Next, the functional configuration of the pulse wave measuring device 10 will be described with reference to FIG.

  FIG. 7 is a block diagram illustrating an example of a functional configuration of the pulse wave measurement device according to the present embodiment.

  As shown in FIG. 7, the pulse wave measurement device 10 includes a visible light imaging unit 122, an infrared light source 123, an infrared light imaging unit 124, and a pulse wave calculation device 100.

  The visible light imaging unit 122 captures an irradiation target irradiated with visible light from the illumination device 30 in the visible light region. Specifically, the visible light imaging unit 122 captures a visible light image obtained by imaging a user's skin as an irradiation target in a visible light region (for example, color), and a visible light waveform computing unit of the pulse wave computing device 100. To 111. The visible light imaging unit 122 outputs, for example, a skin image obtained by imaging skin including a human face or hand as a visible light image. The visible light imaging unit 122 outputs, for example, a plurality of visible light images captured at a plurality of different timings to the visible light waveform calculation unit 111. The skin image is an image obtained by capturing the same portion of the skin including a human face or hand at a plurality of timings that are temporally continuous, and includes, for example, a moving image or a plurality of still images. The visible light imaging unit 122 is realized by the visible light camera 22, for example.

  The infrared light source 123 irradiates the user with infrared light, and the amount of light irradiated is adjusted by the light source control unit 115 of the pulse wave calculation device 100. The infrared light source 123 is realized by, for example, an infrared LED 23.

  The infrared light imaging unit 124 images an irradiation target irradiated with infrared light from the infrared light source 123 in the infrared light region. Specifically, the infrared light imaging unit 124 captures an infrared light image obtained by imaging the user's skin as an irradiation target in an infrared light region (for example, monochrome), in the infrared of the pulse wave calculation device 100. The result is output to the optical waveform calculator 112. For example, the infrared light imaging unit 124 outputs a plurality of infrared light images captured at a plurality of different timings to the infrared light waveform calculation unit 112. The infrared light imaging unit 124 images the same part as the part imaged by the visible light imaging unit 122. For example, the infrared light imaging unit 124 outputs a skin image obtained by imaging skin including a human face or hand as an infrared light image. This is because the infrared light imaging unit 124 can acquire the same pulse wave in the visible light region and the infrared light region by capturing the same region as the region captured by the visible light imaging unit 122. This is because it is easy to compare feature quantities.

  Note that, as an imaging method of the same part, a region of interest (ROI) having the same size is set in the visible light imaging unit 122 and the infrared light imaging unit 124. And it is judged whether the same site | part is imaged by comparing using the pattern recognition about the image in the said ROI imaged with the visible light imaging part 122 and the infrared light imaging part 124, for example. May be. Further, face recognition is performed on each of the visible light image obtained by the visible light imaging unit 122 and the infrared light image obtained by the infrared light imaging unit 124, and the coordinates of the feature points in the eyes, nose, mouth, and the like are performed. The same part is identified by calculating the coordinates (relative position) from the feature points of the eyes, nose, mouth, etc., considering the size ratio of the eyes, nose, mouth, etc. May be.

  Similar to the skin image obtained by the visible light imaging unit 122, the skin image obtained by the infrared light imaging unit 124 has a plurality of timings at which the same portion of the skin including a human face or hand is temporally continuous. For example, it is composed of a moving image or a plurality of still images. The infrared light imaging unit 124 is realized by the infrared light camera 24, for example.

  The pulse wave calculation device 100 includes a visible light waveform calculation unit 111, an infrared light waveform calculation unit 112, a correlation degree calculation unit 113, a control pattern acquisition unit 114, a light source control unit 115, and a biological information calculation unit 116. Is provided. Hereinafter, each component of the pulse wave calculation device 100 will be described in order.

(Visible light waveform calculation unit)
The visible light waveform calculation unit 111 acquires a visible light image from the visible light imaging unit 122, and extracts a visible light waveform that is a waveform indicating a user's pulse wave from the acquired visible light image. The visible light waveform calculation unit 111 extracts the first visible light waveform from the first visible light image acquired before the light amount of the lighting device 30 is controlled. In addition, the visible light waveform calculation unit 111 extracts a second visible light waveform from the second visible light image acquired after the light amount of the lighting device 30 is controlled. The control of the amount of light of the illumination device 30 means that the visible light control signal for reducing the amount of visible light in the illumination device 30 or the amount of visible light in the illumination device 30 by the light source control unit 115 described later. This means that a visible light control signal to be increased is output to the lighting device 30. As described above, in the plurality of visible light images acquired from the visible light imaging unit 122, the first visible light image acquired before the light amount control of the illumination device 30 is performed and the light amount control of the illumination device 30 are controlled. And a second visible light image acquired after is performed. The visible light waveforms extracted from the plurality of visible light images include a first visible light waveform extracted from the first visible light image and a second visible light waveform extracted from the second visible light image. It is.

  The visible light waveform calculation unit 111 may extract a plurality of first feature points that are predetermined feature points in the extracted first visible light waveform. Specifically, when the visible light waveform calculation unit 111 divides the first visible light waveform into a plurality of first unit waveforms in a pulse wave cycle unit, which is a pulse wave cycle, for each of the plurality of first unit waveforms. The first visible light waveform is extracted by extracting the first peak point which is one of the first vertex that is the maximum value in the first unit waveform and the first bottom point that is the minimum value in the first unit waveform. To extract a plurality of first peak points. The first peak point is an example of a first feature point.

  The visible light waveform calculation unit 111 acquires a pulse wave timing as a feature point of the visible light waveform, and calculates a heartbeat interval time from the adjacent pulse wave timings. That is, the visible light waveform calculation unit 111 calculates the time between each of the extracted first feature points and another first feature point adjacent to the first feature point as the first heartbeat interval time. . For example, the visible light waveform calculation unit 111 may, for each of the plurality of extracted first peak points, the first time at the first peak point and another first peak adjacent to the first peak point in time series. A plurality of the first heart beat interval times are calculated by calculating a first heart beat interval time which is a time interval between the second time at the point.

  Specifically, the visible light waveform calculation unit 111 extracts a visible light waveform based on temporal changes in luminance extracted from a plurality of visible light images each associated with an imaged timing. In other words, each of the plurality of visible light images acquired from the visible light imaging unit 122 is associated with the time (time point) when the visible light image is captured by the visible light imaging unit 122. The visible light waveform calculation unit 111 acquires the pulse wave timing of the user (hereinafter also referred to as pulse wave timing) by acquiring the interval between predetermined feature points of the visible light waveform. Then, the visible light waveform calculation unit 111 calculates, for each of the obtained plurality of pulse wave timings, an interval between the pulse wave timing and the next pulse wave timing as a heartbeat interval time.

  In addition, the visible light waveform calculation unit 111 may extract a plurality of third feature points that are predetermined feature points in the extracted second visible light waveform. Specifically, when the visible light waveform calculation unit 111 divides the second visible light waveform into a plurality of third unit waveforms in units of pulse wave periods, the third unit waveform for each of the plurality of third unit waveforms. Are extracted from the second visible light waveform by extracting a third peak point that is one of the third vertex and the third bottom point that is the minimum value in the third unit waveform. A peak point may be extracted. The third peak point is an example of a third feature point.

  For each of the extracted third peak points, the visible light waveform calculation unit 111 uses the fifth time at the third peak point and another third peak point adjacent to the third peak point in time series. A plurality of the third heart beat interval times may be calculated by calculating a third heart beat interval time that is a time interval from the sixth time.

  For example, the visible light waveform calculation unit 111 uses the extracted visible light waveform to specify the timing with the largest luminance change, and specifies the specified timing as the pulse wave timing. Alternatively, the visible light waveform calculation unit 111 identifies the positions of the faces or hands in the plurality of visible light images using the face or hand pattern held in advance, and changes the luminance with time at the identified positions. To identify the visible light waveform. The visible light waveform calculation unit 111 calculates the pulse wave timing using the identified visible light waveform. Here, the pulse wave timing is a time at a predetermined feature point in the time waveform of the luminance, that is, the time waveform of the pulse wave. The predetermined feature point is, for example, the peak position (the time at the apex or bottom point) in the luminance time waveform. The peak position can be specified by using a known local search method including, for example, a hill-climbing method, an autocorrelation method, and a method using a differential function. The visible light waveform calculation unit 111 is realized by, for example, the CPU 101, the main memory 102, the storage 103, and the like.

  In general, a pulse wave is a change in blood pressure or volume in the peripheral vasculature as the heart beats. That is, the pulse wave is a change in the volume of the blood vessel when blood is pumped out from the heart due to contraction of the heart and reaches the face or hand. Thus, when the volume of the blood vessel in the face or hand changes, the amount of blood passing through the blood vessel changes, and the skin color changes depending on the amount of components in the blood such as hemoglobin. For this reason, the brightness of the face or hand in the captured image changes according to the pulse wave. That is, information on blood movement can be acquired by using temporal changes in luminance of the face or hand obtained from images obtained by capturing the face or hand at a plurality of timings. As described above, the visible light waveform calculation unit 111 acquires pulse wave timing by calculating information related to blood movement from a plurality of images captured in time series.

  In obtaining the pulse wave timing in the visible light region, an image obtained by capturing the luminance in the green wavelength region in the visible light image may be used. This is because, in an image captured in the visible light region, a change due to the pulse wave appears greatly in the luminance in the wavelength region near the green. In a visible light image including a plurality of pixels, the luminance in the green wavelength region of a pixel corresponding to a face or hand in a state where a lot of blood is flowing is equivalent to a face or hand in a state where a small amount of blood is flowing It is smaller than the luminance in the green wavelength region of the pixel to be used.

  FIG. 8A is a graph showing an example of a change in luminance of a visible light image, particularly in green, in the present embodiment. Specifically, FIG. 8A shows the luminance change of the green component (G) in the cheek region of the user in the visible light image captured by the visible light imaging unit 122. In the graph of FIG. 8A, the horizontal axis represents time, and the vertical axis represents the luminance of the green component (G). It can be seen that the luminance change shown in FIG. 8A periodically changes in luminance due to the pulse wave.

  When skin is imaged in a daily environment, that is, in a visible light region, a visible light image includes noise due to scattered light from lighting or various factors. Therefore, the visible light waveform calculation unit 111 may perform signal processing using a filter or the like on the visible light image acquired from the visible light imaging unit 122, and obtain a visible light image including many skin luminance changes due to pulse waves. . An example of a filter used for signal processing is a low-pass filter. That is, in this embodiment, the visible light waveform calculation unit 111 performs a visible light waveform extraction process using the luminance change of the green component (G) that has passed through the low-pass filter.

  (A) of FIG. 9 is a graph which shows an example of calculation of the pulse wave timing in this Embodiment. In the graph of FIG. 9A, the horizontal axis indicates time and the vertical axis indicates luminance. In the time waveform of the graph of FIG. 9A, each point at times t1 to t5 is an inflection point or a vertex. Each point in the time waveform of the graph includes an inflection point and a peak point (vertex and bottom point) as feature points. The vertex is a point at the maximum value that is convex upward in the time waveform, and the base point is the point at the minimum value that is convex downward in the time waveform. At each of the above points included in the time waveform, the time at the point (vertex) where the luminance is higher than any of the previous or subsequent time points, or the point (bottom point) where the luminance is lower than any of the previous or subsequent time points. Time is pulse wave timing.

  A method for specifying the position of the vertex, that is, a peak searching method will be described using the luminance time waveform of the graph shown in FIG. The visible light waveform calculation unit 111 sets the current reference point as the point at time t2 in the luminance time waveform. The visible light waveform calculation unit 111 compares the point at time t2 with the point at the previous time t1, and compares the point at time t2 with the point at the next time t3. The visible light waveform calculation unit 111 determines that the reference point is positive when the luminance of the reference point is higher than the luminance of the point at the previous time and the point at the next time. That is, in this case, the visible light waveform calculation unit 111 determines that the reference point is a peak point (vertex) and the time at the reference point is the pulse wave timing.

  On the other hand, the visible light waveform calculation unit 111 determines NO when the luminance of the reference point is smaller than the luminance of at least one of the point at the previous time and the point at the next time. That is, in this case, the visible light waveform calculation unit 111 determines that the reference point is not a peak point (vertex) and that the time at the reference point is not a pulse wave timing.

  In FIG. 9A, the luminance at the point at time t2 is larger than the luminance at the point at time t1, but the luminance at the point at time t2 is smaller than the luminance at the point at time t3. The point at time t2 is determined as NO. Next, the visible light waveform calculation unit 111 increments the reference point by one and sets the next point at time t3 as the reference point. Since the luminance at the point at time t3 is higher than the luminance at the point at time t2 immediately before time t3 and the point at time t4 immediately after time t3, the visible light waveform calculation unit 111 performs the calculation at time t3. The point is determined to be positive. The visible light waveform calculator 111 outputs the time at the point determined to be positive to the correlation calculator 113 as the pulse wave timing. Thereby, as shown in (b) of Drawing 9, the time of a white circle mark is specified as pulse wave timing.

  In addition, the visible light waveform calculation unit 111 considers that the heartbeat interval time is between 333 ms and 1000 ms, for example, based on knowledge of a general heart rate (eg, 60 bpm to 180 bpm) in specifying the pulse wave timing. Thus, the pulse wave timing may be specified. The visible light waveform calculation unit 111 does not need to perform the above-described luminance comparison at all points by considering a general heartbeat interval time. If the luminance comparison is performed at some points, an appropriate pulse can be obtained. Wave timing can be specified. That is, the above-described luminance comparison may be performed using each point in the range from 333 ms to 1000 ms from the recently acquired pulse wave timing as a reference point. In this case, the next pulse wave timing can be specified without performing luminance comparison using points before that range as reference points. Therefore, it is possible to acquire a pulse wave timing that is robust in a daily environment.

  The visible light waveform calculation unit 111 further calculates a heartbeat interval time by calculating a time difference between the obtained adjacent pulse wave timings. The heartbeat interval time varies in time series. For this reason, it can be used to calculate the degree of correlation at a predetermined feature point between the visible light waveform and the infrared light waveform by comparing with the heartbeat interval time of the pulse wave specified from the infrared light waveform acquired in the same period. Can do.

  FIG. 10 is a graph showing an example of heartbeat interval times acquired in time series. In the graph of FIG. 10, the horizontal axis indicates the data number associated with the heartbeat interval time acquired in time series, and the vertical axis indicates the heartbeat interval time. As shown in FIG. 10, it can be seen that the heartbeat interval time varies with time. The data number indicates the order in which data (here, heartbeat interval time) is stored in the memory. That is, the data number corresponding to the heartbeat interval time recorded in the nth (n is a natural number) is “n”.

  The visible light waveform calculation unit 111 may further extract the time of the inflection point immediately after the pulse wave timing in the visible light waveform. Specifically, the visible light waveform calculation unit 111 obtains the minimum point of the visible light differential luminance by calculating the first derivative of the luminance value of the visible light waveform, and sets the time that becomes the minimum point as the time of the inflection point. (Hereinafter referred to as inflection point timing) is calculated. That is, the visible light waveform calculation unit 111 may extract a plurality of inflection points from the top to the bottom as predetermined feature points.

  The visible light waveform calculation unit 111 also calculates the inflection point timing in consideration of the fact that the heartbeat interval time is between 333 ms and 1000 ms, for example, based on general heart rate knowledge. Point timing may be calculated. As a result, even if the visible light waveform includes an inflection point that is completely unrelated to the heartbeat, the inflection point is not specified, so that the inflection point timing can be calculated more accurately. .

  FIG. 11 is a graph for explaining a method of extracting an inflection point from a pulse wave. Specifically, (a) of FIG. 11 is a graph showing a visible light waveform obtained from a visible light image, and (b) of FIG. 11 plots the first derivative values of (a) of FIG. It is a graph. In FIG. 11A, a circle represents a peak among peak points, and an X represents an inflection point. In (b) of FIG. 11, a circle indicates a point corresponding to the vertex in (a) of FIG. 11, and an X indicates a point corresponding to the inflection point in (a) of FIG. In the graph of FIG. 11A, the horizontal axis represents time, and the vertical axis represents the luminance value. In the graph of FIG. 11B, the horizontal axis indicates time, and the vertical axis indicates the derivative of the luminance value.

  In the extraction of the visible light waveform, a visible light image in which green light is captured as described above is used. The principle of this visible light waveform extraction will be described. When the blood volume in blood vessels such as the face or hand increases or decreases according to the pulse wave, the amount of hemoglobin in the blood increases or decreases according to the blood volume. That is, the amount of hemoglobin that absorbs light in the green wavelength region increases or decreases according to the increase or decrease of the blood volume in the blood vessel. For this reason, in the visible light image captured by the visible light imaging unit 122, the color of the skin near the blood vessel changes according to the increase or decrease of the blood volume, and the luminance value of the green component in particular of the visible light varies. . Specifically, since hemoglobin absorbs green light, the luminance value in the visible light image decreases by the amount absorbed by hemoglobin.

  Further, the visible light waveform has a characteristic that the gradient from the top point to the next bottom point is steeper than the gradient from the bottom point to the top point. Therefore, it is relatively susceptible to noise between the bottom point and the apex. On the other hand, since the slope is steep from the top to the next bottom point, it is not easily affected by noise. For this reason, the inflection point timing existing between the vertex and the bottom point is also less susceptible to noise and has a feature that it can be obtained relatively stably. From the above, the visible light waveform calculation unit 111 may calculate the time difference between the inflection points existing from the top to the bottom as the heartbeat interval time.

  Moreover, the peak point of the visible light waveform described above is a portion where the differential coefficient becomes 0 immediately before the inflection point. Specifically, as shown in FIG. 11 (b), the time at which the differential coefficient immediately before the mark X, which is the inflection point, becomes 0, is the time indicated by the circle indicating the apex in FIG. 11 (a). You can see that Using this feature, the visible light waveform calculation unit 111 may limit the vertex acquired from the visible light waveform to only the vertex immediately before the inflection point.

  The visible light waveform calculation unit 111 further calculates the inclination from the top to the bottom of the visible light waveform. The visible light waveform calculation unit 111 includes one first vertex of one of the plurality of first vertices and one first bottom immediately after the one first vertex among the plurality of first bottom points. The first slope of the first straight line connecting the points is calculated. The inclination in the visible light waveform should be as large as possible by adjusting the luminance of the illumination device 30. This is because the greater the inclination, the greater the kurtosis of the vertex in the visible light waveform, and the smaller the time lag of the pulse wave timing due to filter processing or the like.

  FIG. 12 is a graph showing a visible light waveform for explaining a method of calculating the slope of the visible light waveform. In the graph of FIG. 12, the horizontal axis indicates time, the vertical axis indicates the luminance value, the circle indicates a vertex, and the triangle indicates a bottom point. The visible light waveform calculation unit 111 connects the vertex (circle) and the next base point (triangle) with a straight line, and calculates the slope of the straight line. The inclination calculated here differs depending on the amount of light emitted from the light source in the illumination device 30, the user's skin site acquired by the visible light imaging unit 122, and the like. Accordingly, the pulse wave can be acquired clearly, for example, the ROI corresponding to the light quantity of the illumination device 30 and the user's part in the visible light imaging unit 122 so that the heartbeat interval time can be continuously acquired from 333 ms to 1000 ms. Each can be set, tilt information can be recorded, and compared with the tilt information in the pulse wave of infrared light. Further, the visible light waveform calculation unit 111 is in an initial state, that is, after the lighting device 30 is turned on, the light source control unit 115 controls the amount of visible light of the lighting device 30 or the infrared light of the infrared light source 123. The inclination from the top to the bottom of the visible light waveform in the state until the amount of light is changed is recorded in the memory (for example, the storage 103) as the first inclination A. While comparing the feature points between the visible light waveform and the infrared light waveform, the pulse wave measuring device 10 gradually decreases the light amount of the illumination device 30 to increase the light amount of the infrared light source 123. It is characterized by going. In this way, in order to gradually reduce the amount of visible light, it is in the initial state that the gradient from the top to the bottom of the visible light waveform becomes the largest.

(Infrared light waveform calculation unit)
The infrared light waveform calculation unit 112 acquires an infrared light image from the infrared light imaging unit 124, and extracts an infrared light waveform that is a waveform indicating a user's pulse wave from the acquired infrared light image. The infrared light waveform calculation unit 112 extracts the first infrared light waveform from the first infrared light image acquired before the light amount of the infrared light source 123 is controlled. In addition, the infrared light waveform calculation unit 112 extracts a second infrared light waveform from the second infrared light image acquired after the light amount of the infrared light source 123 is controlled. The control of the light amount of the infrared light source 123 means that the light source control unit 115 described later increases the amount of infrared light in the infrared light source 123 or an infrared light control signal. This means that an infrared light control signal for reducing the amount of infrared light in the light source 123 is output to the infrared light source 123. Thus, the plurality of infrared light images acquired from the infrared light imaging unit 124 include the first infrared light image acquired before the control of the light amount of the infrared light source 123 and the infrared light. 2nd infrared light image acquired after control of the light quantity of the light source 123 is contained.

  The infrared light waveform calculation unit 112 may extract a plurality of second feature points that are predetermined feature points in the extracted first infrared light waveform. Specifically, when the infrared light waveform calculation unit 112 divides the first infrared light waveform into a plurality of second unit waveforms in units of pulse wave periods, the second waveform of each of the plurality of second unit waveforms is the second. By extracting a second peak point that is one of the second vertex that is the maximum value in the unit waveform and the second bottom point that is the minimum value in the second unit waveform, a plurality of peak values are extracted from the first infrared light waveform. A second peak point is extracted. The second peak point is an example of a second feature point.

  Similar to the visible light waveform calculation unit 111, the infrared light waveform calculation unit 112 acquires the pulse wave timing as a feature point of the infrared light waveform, and calculates the heartbeat interval time from the adjacent pulse wave timing. In other words, the infrared light waveform calculation unit 112 calculates the time between each of the extracted second feature points and the other second feature points adjacent to the second feature point as the second heartbeat interval time. To do. Specifically, the infrared light waveform calculation unit 112 extracts an infrared light waveform based on a temporal change in luminance extracted from a plurality of infrared light images. That is, each of the plurality of infrared light images acquired from the infrared light imaging unit 124 is associated with the time (time point) when the infrared light image is captured by the infrared light imaging unit 124. For example, the infrared light waveform calculation unit 112, for each of the plurality of extracted second peak points, the third time at the second peak point and another second adjacent to the second peak point in time series. A plurality of second heart beat interval times are calculated by calculating a second heart beat interval time that is a time interval from the fourth time at the peak point.

  The infrared light waveform calculation unit 112 may extract a plurality of fourth feature points that are predetermined feature points in the extracted second infrared light waveform. Specifically, when the infrared light waveform calculation unit 112 divides the second infrared light waveform into a plurality of fourth unit waveforms in units of pulse wave periods, the fourth waveform of each of the plurality of fourth unit waveforms is calculated. By extracting the fourth peak that is the maximum value in the unit waveform and the fourth peak point that is the minimum value in the fourth unit waveform, a plurality of peak values are extracted from the second infrared light waveform. A fourth peak point may be extracted. The fourth peak point is an example of a fourth feature point.

  For each of the plurality of extracted fourth peak points, the infrared light waveform calculation unit 112 has a seventh time at the fourth peak point and another fourth peak point adjacent to the fourth peak point in time series. A plurality of fourth heart beat interval times may be calculated by calculating the fourth heart beat interval time, which is the time interval between the eighth time and

  Here, similarly to the visible light waveform calculation unit 111, the infrared light waveform calculation unit 112 calculates a peak position as a predetermined feature point of the infrared light waveform, for example, a hill climbing method, an autocorrelation method, and a differential function. It can be identified using a known local search method including the method used. The infrared light waveform calculation unit 112 is realized by, for example, the CPU 101, the main memory 102, the storage 103, and the like, similarly to the visible light waveform calculation unit 111.

  In general, in an infrared light image, as in a visible light image, the luminance of a skin region, for example, a face or a hand in the image changes depending on the amount of a component in blood such as hemoglobin. That is, information on blood movement can be acquired by using temporal changes in luminance of the face or hand obtained from images obtained by capturing the face or hand at a plurality of timings. In this manner, the infrared light waveform calculation unit 112 acquires pulse wave timing by calculating information related to blood movement from a plurality of images taken in time series.

  In the acquisition of the pulse wave timing in the infrared light region, an image obtained by capturing the luminance in the wavelength region of 800 nm or more in the infrared light image may be used. This is because, in an image captured in the infrared light region, a change due to the pulse wave appears greatly in the luminance in the wavelength region near 800 to 950 nm.

  FIG. 8B is a graph showing an example of the luminance change of the infrared light image in the present embodiment. Specifically, FIG. 8B shows a change in luminance of the cheek region of the user in the infrared light image captured by the infrared light imaging unit 124. In the graph of FIG. 8B, the horizontal axis indicates time, and the vertical axis indicates luminance. It can be seen that the luminance change shown in (b) of FIG. 8 periodically changes in luminance due to the pulse wave.

  However, when skin is imaged in the infrared light region, the amount of absorption of infrared light by hemoglobin is less than when skin is imaged in the visible light region. That is, the infrared light image captured in the infrared light region is likely to contain noise due to various factors such as body movement. Therefore, by applying signal processing to the captured infrared light image using a filter or the like and irradiating the user's skin area with an appropriate amount of infrared light, the infrared light contains many skin luminance changes caused by pulse waves. An optical image may be obtained. An example of a filter used for signal processing is a low-pass filter. In other words, in this embodiment, the infrared light waveform calculation unit 112 performs infrared light waveform extraction processing using the luminance change of the infrared light that has passed through the ρ bus filter. The method for determining the amount of infrared light by the infrared light source 123 will be described in the correlation calculation unit 113 or the light source control unit 115.

  Next, a peak search method in the infrared light waveform calculation unit 112 will be described. The peak search in the infrared light waveform can use the same method as the peak search in the visible light waveform.

  As with the visible light waveform calculation unit 111, the infrared light waveform calculation unit 112 specifies the pulse wave timing based on knowledge of a general heart rate (for example, 60 bpm to 180 bpm), for example, 333 ms to 1000 ms. The pulse wave timing may be specified in consideration of the period up to. The infrared light waveform calculation unit 112 does not need to perform the above-described luminance comparison at all points by considering a general heartbeat interval time. The pulse wave timing can be specified. That is, the above-described luminance comparison may be performed using each point in the range from 333 ms to 1000 ms from the recently acquired pulse wave timing as a reference point. In this case, the next pulse wave timing can be specified without performing luminance comparison using points before that range as reference points.

  Similar to the visible light waveform calculation unit 111, the infrared light waveform calculation unit 112 calculates a heartbeat interval time by calculating a time difference between the obtained adjacent pulse wave timings. Further, the infrared light waveform calculation unit 112 may further extract the time of the inflection point immediately after the pulse wave timing in the infrared light waveform. Then, for example, the infrared light waveform calculation unit 112 obtains the minimum point of the infrared light differential luminance by calculating the first derivative of the luminance value of the infrared light waveform, and sets the time that becomes the minimum point as the inflection point. Is calculated (inflection point timing). That is, the infrared light waveform calculation unit 112 may extract a plurality of inflection points from the top to the bottom as predetermined feature points.

  Similarly to the visible light waveform calculation unit 111, the infrared light waveform calculation unit 112 calculates a tilt from the top to the bottom of the infrared light waveform. In other words, the infrared light waveform calculation unit 112 has the fourth vertex of one of the plurality of fourth vertices and the plurality of fourth bottom points in the second infrared light waveform. A second slope that is the slope of the second straight line connecting the first base point immediately after the first time series is calculated.

  As described above, the infrared light waveform calculation unit 112 performs a process similar to that of the visible light waveform calculation unit 111 to extract a plurality of predetermined feature points as second feature points. However, the infrared light waveform varies greatly depending on the amount of infrared light emitted from the light source as compared with the visible light waveform. That is, the infrared light waveform is more susceptible to the light amount of the light source than the visible light waveform.

  FIG. 13 is a graph showing an infrared light waveform when a human skin image is acquired by an infrared light camera for each level of the light amount of the infrared light source. In FIG. 13, the light amount level in the infrared light source is increased in order from (a) to (d). That is, the light source level indicates that the light source level 1 has the smallest light amount, the light amount increases as the light source level increases, and the light source level 4 has the largest light amount. The light source level indicates that the control voltage of the light source increases by about 0.5 V every time the level increases by 1. Further, a circle in each graph of FIG. 13 indicates the peak position (vertex) of the pulse wave. As shown in FIG. 13A, when the amount of light in the light source is small, the noise becomes larger than the infrared light from the infrared light source, and it is difficult to specify the pulse wave timing. On the other hand, as shown in FIGS. 13C and 13D, if the amount of light in the light source is large, a change in skin brightness corresponding to the pulse wave is buried in the light amount of the light source, and the shape of the pulse wave becomes small. It is difficult to specify the pulse wave timing.

  By the way, when acquiring a pulse wave using an image that is irradiated with visible light and captured in the visible light region, even if the visible light is irradiated with a light amount that is not too strong for the user's eyes, the pulse wave is sufficiently emitted with the irradiation amount. You can get it. However, when acquiring a pulse wave using an image that is irradiated with infrared light and captured in the infrared light region, even if the light amount of the infrared light is controlled, as described above, it contains noise or the infrared light There is too much light. For this reason, it is difficult to acquire a pulse wave only within a considerably narrow light amount range. In addition, even if the amount of light from the infrared light source is determined in advance to a predetermined value, it varies depending on the area of the skin to be acquired, the skin quality of the user, the color of the skin, and the like. Is difficult. Therefore, it is necessary to control the amount of infrared light to an appropriate value while reducing the amount of visible light so that the visible light waveform matches the infrared light waveform by the correlation degree calculation unit 113 described below. There is.

(Correlation degree calculation part)
The correlation degree calculation unit 113 calculates the degree of correlation between the visible light waveform obtained from the visible light waveform calculation unit 111 and the infrared light waveform obtained from the infrared light waveform calculation unit 112. Correlation degree calculation unit 113 then determines a command to adjust each light quantity in illumination device 30 and infrared light source 123 according to the calculated correlation degree, and sends the determined command to light source control unit 115.

  Correlation degree calculation unit 113 converts a plurality of first heartbeat interval times calculated from the first visible light waveform and a plurality of second heartbeat interval times calculated from the first infrared light waveform into visible light waveform calculation unit 111 and Obtained from the infrared light waveform calculation unit 112, respectively. Then, the correlation degree calculation unit 113 calculates a first correlation degree between the plurality of first heart beat interval times and the plurality of second heart beat interval times that correspond to each other in time series.

  Further, the correlation degree calculation unit 113 converts the plurality of third heart beat interval times calculated from the second visible light waveform and the plurality of fourth heart beat interval times calculated from the second infrared light waveform into a visible light waveform calculation unit. 111 and the infrared light waveform calculation unit 112, respectively. Then, the correlation degree calculation unit 113 may calculate the second correlation degree between the plurality of third heart beat interval times and the plurality of fourth heart beat interval times, which correspond to each other in time series.

  FIG. 14 is a graph showing the first heartbeat interval time and the second heartbeat interval time plotted with data in time series order. In the graph of FIG. 14, the horizontal axis indicates the data numbers in time series, and the vertical axis indicates the heartbeat interval time corresponding to each data number. Here, the data number indicates the order in which data of each heartbeat interval time is stored in the memory. That is, in the first heartbeat interval time, the data number corresponding to the heartbeat interval time recorded nth (n is a natural number) is “n”. In the second heartbeat interval time, the data number corresponding to the heartbeat interval time recorded nth (n is a natural number) is “n”. Furthermore, since the first heartbeat interval time and the second heartbeat interval time are the results of measuring the pulse wave at the same timing, as a general rule, as long as there is no measurement error, if the data numbers are the same, the timing is almost the same. It can be said that it is the result of measuring the pulse wave. That is, the plurality of first heart beat interval times and the plurality of second heart beat interval times include a pair of first heart beat interval times and second heart beat interval times corresponding to each other in time series.

  The correlation degree calculation unit 113 calculates the degree of correlation between the plurality of first heartbeat intervals and the plurality of second heartbeat intervals using the correlation method. Specifically, the correlation degree calculation unit 113 uses the following (Equation 1) to calculate the first time between a plurality of first heartbeat intervals and a plurality of second heartbeat intervals corresponding to each other in time series. One correlation coefficient is calculated as the first degree of correlation.

  Further, the correlation degree calculation unit 113 uses the following (Equation 2) to calculate the second phase relationship between the plurality of third heart beat interval times and the plurality of fourth heart beat interval times corresponding to each other in time series. The number is calculated as the second correlation degree.

  For example, if the first correlation coefficient is a second threshold (predetermined threshold), for example, 0.8 or more, the correlation degree calculation unit 113 may include a plurality of first heartbeat intervals and a plurality of second heartbeat intervals. For example, a signal of “TRUE” is transmitted to the light source control unit 115 as a signal indicating that the time is almost the same. On the other hand, if the correlation coefficient is a value smaller than a second threshold, for example, 0.8, the correlation degree calculation unit 113 has a plurality of first heartbeat intervals and a plurality of second heartbeat intervals. For example, a signal “FALSE” is transmitted to the light source control unit 115 as a signal indicating that they do not match. Correlation degree calculation unit 113 performs the above-described processing on the second correlation coefficient as well as the first correlation coefficient.

  In addition, the correlation degree calculation unit 113 determines whether each heartbeat interval time is appropriate as well as the correlation between the first heartbeat interval time and the second heartbeat interval time, and transmits the determination result to the light source control unit 115. May be. Specifically, the correlation calculation unit 113 calculates the time interval between the first heartbeat interval time and the second heartbeat interval time that correspond to each other in time series among the plurality of first heartbeat interval times and the plurality of second heartbeat interval times. It is determined whether or not the absolute error between the two exceeds a third threshold (for example, 200 ms). For example, the correlation calculation unit 113 calculates the absolute error of the first heartbeat interval time and the second heartbeat interval time having the same data number, and determines whether or not the absolute error exceeds the third threshold value. Then, for example, when it is determined that the absolute error exceeds the third threshold value, the correlation calculation unit 113 determines that the number of peak points of either the visible light waveform or the infrared light waveform is excessive. To do. Then, the correlation degree calculation unit transmits the waveform (visible light waveform or infrared light waveform) having the excessive number of peak points to the light source control unit 115. The absolute error calculation is obtained by the following equation 3.

e = RRI _RGB -RRI _IR (Formula 3)

In Equation 1, e indicates the absolute error between the corresponding first heartbeat interval time and the second heartbeat interval time, RRI _RGB indicates the first heartbeat interval time, and RRI _IR indicates the second heartbeat interval time.

  Further, if e is smaller than (−1) × third threshold (for example, −200 ms), correlation degree calculation unit 113 determines that the number of peak points in visible light is excessive, and e is the third If it is larger than the threshold (for example, 200 ms), it is determined that the number of peak points in the infrared light is excessive. Then, the correlation degree calculation unit 113 transmits information indicating whether the waveform having the excessive number of peak points is the visible light waveform or the infrared light waveform to the light source control unit 115 as a determination result. In this way, it is possible to specify that the peak point is excessively acquired in either waveform or the acquisition of the peak point is unsuccessful from the shift in the corresponding heartbeat interval time between the two waveforms.

  Correlation degree calculation unit 113, for example, has a corresponding absolute error of the first heartbeat interval time and the second heartbeat interval time that exceeds the third threshold, and the peak point is excessively acquired in the visible light waveform. If it is determined that the signal is present, a “False, RGB” signal indicating the result of the determination is transmitted to the light source controller 115. If the correlation degree calculation unit 113 determines that the absolute error exceeds the third threshold and that the peak point is excessively acquired in the infrared light waveform, the correlation calculation unit 113 determines that the light source control unit 115 performs the determination. A “False, IR” signal indicating the result is transmitted.

  FIG. 15 is a diagram for describing a specific example of determining whether or not the heartbeat interval time is appropriate. FIG. 15A is a graph showing a case where a plurality of acquired heartbeat intervals are not appropriate. FIG. 15B is a graph showing an example of a visible light waveform or an infrared light waveform corresponding to FIG. In the graph of FIG. 15A, the horizontal axis indicates the data numbers in time series, and the vertical axis indicates the heartbeat interval time corresponding to each data number. In the graph of FIG. 15B, the horizontal axis indicates time, and the vertical axis indicates luminance in the image.

  In FIG. 15A, the two heartbeat intervals surrounded by a dotted line are not appropriate. The heartbeat interval time generally fluctuates and fluctuates, but the value hardly fluctuates abruptly. For example, as shown in FIG. 15A, in the region other than the portion surrounded by the dotted line, the average value is about 950 ms, and the standard deviation is about 50 ms. However, the value of the heartbeat interval time at the two points surrounded by the dotted line changes abruptly to about 600 to 700 ms. This occurs because the portion with the broken line in FIG. 15B is acquired as the peak point. That is, it occurs when the peak points are acquired excessively in the visible light waveform calculation unit 111 or the infrared light waveform calculation unit 112.

  When either the visible light waveform calculation unit 111 or the infrared light waveform calculation unit 112 obtains a result as shown in FIG. 15, data of a plurality of first heart beat interval times and a plurality of second heart beat interval times When the numbers are compared, the data numbers do not match.

  This is shown in FIG. FIG. 16 is a diagram for explaining an example in the case where excessive acquisition of peak points is performed in the visible light waveform and no excessive acquisition of peak points is performed in the corresponding infrared light waveform.

  A plurality of data of the first or second heart beat interval time is stored in the storage 103 in the form of (data No, heart beat interval time), for example. Data indicating a plurality of first heartbeat intervals acquired in the visible light waveform is, for example, (x, t20-t11), (x + 1, t12-t20), (x + 2, t13-t12). Moreover, the data which shows the some 2nd heartbeat interval time acquired in an infrared-light waveform will be (x, t12-t11), (x + 1, t13-t12), for example. Thereby, when the data acquired in each of the visible light waveform and the infrared light waveform are compared, the number of data is shifted although it is the data acquired in the same time interval t11 to t13. As a result, the data correspondence between the subsequent first heartbeat interval time and the second heartbeat interval time is all shifted, and the degree of correlation of the time fluctuation of the heartbeat interval time is shifted.

  Therefore, the correlation degree calculation unit 113 has an absolute error of the heartbeat interval time in each data number of the third or fourth heartbeat interval time obtained by the visible light waveform calculation unit 111 and the infrared light waveform calculation unit 112. When the third threshold value is, for example, 200 ms or more, one pulse wave peak having the larger number of peak points is deleted. Then, the correlation degree calculation unit 113 performs a process of reducing the subsequent data numbers one by one from the data number corresponding to the deleted peak.

When it is determined that the peak points (that is, the predetermined feature points) are acquired excessively as described above, the correlation degree calculation unit 113 has a waveform having a larger number of the predetermined feature points (visible light waveform or infrared light). A predetermined feature point that is a reference for calculating the heartbeat interval time in the optical waveform may be excluded from the calculation target of the heartbeat interval time. In other words, if e is smaller than (−1) × the third threshold, the correlation degree calculation unit 113 sets the peak point that is the basis of the RRI _RGB calculation used to calculate the e as the first heartbeat interval time. Excluded from the operation target. If e is larger than the third threshold value, correlation degree calculation unit 113 excludes the peak point that is the basis for the RRI _IR calculation used to calculate e from the calculation target of the second heartbeat interval time.

  That is, the correlation calculation unit 113 calculates an absolute error between the third heart beat interval time and the fourth heart beat interval time corresponding to each other in time series among the plurality of third heart beat interval times and the plurality of fourth heart beat interval times. It is determined whether or not exceeds a third threshold. Then, when it is determined that the absolute error exceeds the third threshold, the correlation degree calculation unit 113 compares the number of the plurality of third peak points with the number of the plurality of fourth peak points. Correlation degree calculation unit 113 calculates the peak point of the third heartbeat interval time and the fourth heartbeat interval time determined to exceed the third threshold as a result of comparison and determined to be larger. Identify the beat interval time. The correlation degree calculation unit 113 excludes the peak point that is the reference for calculating the specified heartbeat interval time from the calculation target of the heartbeat interval time.

  Further, excessive acquisition of peak points occurs due to a lot of noise in the acquired waveform (visible light waveform or infrared light waveform). For this reason, it is ascertained whether the excessively acquired waveform is a visible light waveform or an infrared light waveform, and for example, a signal such as “FALSE, RGB” is generated as described above, and the generated signal Is transmitted to the light source control unit 115. That is, if the light source control unit 115 receives the “FALSE, RGB” signal, the heartbeat interval times between the visible light waveform and the infrared light waveform are not matched, and the cause is not matched. It can be grasped that it is a visible light waveform. As described above, since the data shift in the acquisition of the peak point between the visible light waveform and the infrared light waveform can be grasped and information indicating the grasped result can be transmitted to the light source control unit 115, the visible light waveform and the infrared light waveform can be transmitted. It becomes possible to acquire a user's pulse wave more accurately.

  In the correlation degree calculation unit 113, the determination of the degree of correlation between the first heartbeat interval time and the second heartbeat interval time is made with the second threshold being 0.8, but the present invention is not limited to this. Specifically, the second threshold value may be changed according to the accuracy of the biological information that the user wants to measure. For example, if the user wants to more accurately acquire biological information during sleep, for example, information such as heartbeat and blood pressure, by accurately extracting pulse waves with infrared light during sleep, the second criterion is used. May be increased to a value such as 0.9, for example.

  Further, when the second threshold value of the correlation coefficient as a reference is adjusted, the reliability may be displayed on the presentation device 40 as the reliability of the acquired data according to the adjusted second threshold value. For example, if the feature amount between the visible light waveform and the infrared light waveform does not match easily and the amount of light from the visible light source cannot be reduced during sleep or the like, the second correlation coefficient as the reference is used. The threshold value may be changed to a value smaller than 0.8, such as 0.6. At this time, since the accuracy related to the degree of correlation becomes small, it may be displayed on the presentation device 40 that the degree of reliability has become small.

  Correlation degree calculation unit 113, when the correlation coefficient of the first and second heartbeat interval times acquired in time series from the visible light waveform and the infrared light waveform is smaller than the second threshold, or visible light waveform calculation unit 111 When the peak point of the first predetermined period is excessively acquired in the infrared light waveform calculation unit 112, the visible light waveform and the infrared light waveform are obtained using the inflection points of the visible light waveform and the infrared light waveform, respectively. The degree of correlation may be determined. That is, a phase between a plurality of third heart beat interval times calculated using the first inflection points and a plurality of fourth heart beat interval times calculated using the second inflection points, which correspond to each other in time series. The number of relationships may be calculated as the second correlation coefficient by using (Equation 2).

  Specifically, as described above, when the correlation coefficient of the first and second heartbeat interval times in the visible light waveform and the infrared light waveform is smaller than a second threshold, for example, 0.8, or visible The number of peak points acquired by the optical waveform calculation unit 111 and the infrared light waveform calculation unit 112 do not match in the first predetermined section (for example, 5 seconds), and the number of peak points in at least one waveform is the first. When the threshold value (for example, 10) is exceeded, the inflection points in both the visible light waveform and the infrared light waveform are used, and the correlation of the time interval information between the inflection points is determined in each waveform. May be.

  That is, the correlation calculation unit 113 performs the tenth determination for determining whether the number of the plurality of third peak points or the number of the plurality of fourth peak points exceeds the first threshold in the first predetermined period. . If the degree of correlation calculation unit 113 determines that the number of third peak points or the number of fourth peak points exceeds the first threshold value in the first predetermined period, the correlation degree calculation unit 113 may perform the following processing.

  In other words, the correlation degree calculation unit 113 causes the visible light waveform calculation unit 111 to make a time series of the third vertex and the third vertex of the third vertex for each of the plurality of third vertices. A plurality of the first inflection points are extracted by extracting a first inflection point that is an inflection point between the immediately following third base point. In addition, the correlation calculation unit 113 causes the infrared light waveform calculation unit 112 to time-series the fourth vertex of the fourth vertex and the fourth bottom point for each of the fourth vertex. A plurality of second inflection points are extracted by extracting a second inflection point, which is an inflection point between the fourth base point immediately after in step. In addition, the correlation calculation unit 113 causes the visible light waveform calculation unit 111 to calculate the ninth time at the first inflection point and the first inflection point for each of the extracted first inflection points. A time interval from the tenth time at another adjacent first inflection point is calculated as a third heartbeat interval time. In addition, the correlation degree calculation unit 113 sends the infrared light waveform calculation unit 112 the seventh time at the second inflection point and the second inflection point for each of the extracted second inflection points. A time interval from the eighth time at another second inflection point adjacent to is calculated as the fourth heartbeat interval time. Then, the correlation degree calculation unit 113 corresponds to each other in the time series, and a plurality of third heart beat interval times calculated using the first inflection points and a plurality of fourth heart beats calculated using the second inflection points. The second correlation coefficient between the interval times is calculated as the second correlation degree using (Equation 2).

  Further, regardless of the result of the tenth determination, the correlation degree calculation unit 113, as described above, uses the plurality of third heartbeat interval times calculated using the first inflection point and the second variable as described above. The second correlation coefficient between the plurality of fourth heartbeat interval times calculated using the inflection points may be calculated as the second correlation degree using (Equation 2). In this case, the standard deviation of the heartbeat interval time calculated from the peak point that is determined to be smaller as a result of the comparison is less than or equal to the fourth threshold value.

  FIG. 17 is a diagram for explaining a case where the degree of correlation is calculated using inflection points. FIG. 17A is a graph showing peak points (vertices) acquired in the visible light waveform, and FIG. 17B is a graph showing peak points (vertices) acquired in the infrared light waveform. It is. In both (a) and (b) of FIG. 17, the horizontal axis indicates time, the vertical axis indicates luminance, the black circle indicates the acquired vertex, and the white circle indicates the acquired inflection point.

  In (a) of FIG. 17, the peak point is excessively acquired in the visible light waveform, and the peak point is equal to or higher than the first threshold value or exceeds the first threshold value in the first predetermined period (5 seconds). It can be seen that there are 10 or 11. On the other hand, in FIG. 17B, the peak point is acquired at a constant heartbeat interval in the infrared light waveform, and the standard deviation is 100 ms or less. At this time, the time-series data numbers indicating the first and second heartbeat interval times in the visible light waveform and the infrared light waveform are shifted.

  Therefore, the correlation degree calculation unit 113 uses the inflection points existing between the top and bottom points of each pulse wave, which are obtained from the visible light waveform calculation unit 111 and the infrared light waveform calculation unit 112, to display visible light. The degree of correlation between the waveform and the infrared light waveform may be calculated. For example, the correlation calculation unit 113 causes the visible light waveform calculation unit 111 and the infrared light waveform calculation unit 112 to calculate the first heartbeat interval time and the second heartbeat interval time calculated using the inflection point, and The degree of correlation between the first and second heartbeat intervals is calculated. As a specific calculation method, evaluation is performed based on the correlation or absolute error of the heartbeat interval time between the inflection points of the visible light waveform and the infrared light waveform.

  Note that in the correlation calculation unit 113, when the correlation coefficient of the heartbeat interval time in the visible light waveform or the infrared light waveform is smaller than the second threshold, or the number of peak points in the visible light waveform or the infrared light waveform is In the first predetermined period, when the number of peak points in at least one waveform is greater than the first threshold, the correlation degree between the visible light waveform and the infrared light waveform using the heartbeat interval time between the inflection points. However, the present invention is not limited to this. For example, the correlation degree calculation unit 113 may calculate the degree of correlation between the visible light waveform and the infrared light waveform using the heartbeat interval time between the inflection points from the beginning without using the peak point. As a result, even if peak points cannot be accurately obtained from the visible light waveform or infrared light waveform, the time similar to the heartbeat interval time is calculated by calculating the heartbeat interval time between the inflection points. it can. However, the heartbeat interval time between the inflection points is less susceptible to noise than the heartbeat interval time that can be acquired from the peak point, but the inflection point is likely to fluctuate between the vertex and the bottom point. That is, the peak-to-vertex heartbeat interval time is stable, for example, the standard deviation tends to be within 100 ms, and the time error tends to be smaller than the heartbeat interval time between the inflection points. Therefore, in the present disclosure, the heartbeat interval time calculated from the peak point is preferentially used unless otherwise specified.

  In addition to the above, when the following condition is satisfied, the correlation degree calculation unit 113 is used for calculating the correlation degree instead of the heartbeat interval time for calculating the heartbeat interval time between the inflection points from the peak point. Also good. The condition is, for example, the standard deviation of the heartbeat interval time corresponding to the waveform having the smaller number of peak points of the visible light waveform and the infrared light waveform among the plurality of heartbeat intervals and the plurality of heartbeat intervals. Is less than or equal to a fourth threshold value (for example, 100 ms). This is the number of peak points in the first predetermined period. When determining whether or not an excessive number of peak points have been acquired, the number of peak points in the first predetermined period is actually excessive. The condition that the number exceeds the first threshold value does not apply, and an excessively acquired peak point may be overlooked.

  For example, FIG. 18 is a diagram for describing an example in which the condition that the number of peak points in the first predetermined period exceeds the first threshold value although the number of peak points is excessive is not applicable. In both (a) and (b) of FIG. 18, the horizontal axis indicates time, the vertical axis indicates luminance, the black circle indicates the acquired vertex, and the white circle indicates the acquired inflection point.

  As shown in FIG. 18A, when the number of peak points acquired in 5 seconds in the visible light waveform is 8, the number of peak points in the first predetermined period exceeds the first threshold. However, the number of peak points different from the number of peak points acquired in the infrared light waveform shown in FIG. 18B is acquired. At this time, as described above, if even one peak point is acquired excessively, there is a problem that the data numbers in the first heartbeat interval time and the second heartbeat interval time are shifted one by one. Alternatively, if it can be shown that the heartbeat interval time of either one of the infrared light waveforms is substantially constant, it can be adjusted (deleted) according to the number of peak points of the waveform. Details of the adjustment of the peak point are as described with reference to FIG.

  It should be noted that when the standard deviation of the heartbeat interval time in the first predetermined period exceeds the fourth threshold value in both the visible light waveform and the infrared light waveform, the correlation degree calculation unit 113 selects an appropriate value from both waveforms. It is determined that the pulse wave timing cannot be acquired, and a “False, Both” signal indicating that appropriate pulse wave timing cannot be acquired from both waveforms is transmitted to the light source control unit 115.

  Correlation degree calculation unit 113, when the use of pulse wave measuring device 10 is started, and when peak points can be appropriately acquired in first predetermined period by visible light waveform calculation unit 111 (that is, the standard of heartbeat interval time) When the deviation is smaller than the fourth threshold), the visible light waveform calculation unit 111 is stored in the memory as a result of calculating the slope between the vertex and the bottom of the visible light waveform as the first slope A. Then, every time the light amount in the illumination device 30 or the infrared light source 123 is changed by the light source control unit 115, the correlation degree calculation unit 113 has the second slope between the vertex and the bottom of the infrared light waveform as the first slope. A command is sent to the light source control unit 115 so that the inclination A becomes. Further, the correlation calculation unit 113 does not need to use the peak point acquired during the adjustment of the light amount of the light source in the light source control unit 115 for calculation of the correlation between the visible light waveform and the infrared light waveform. Good.

  FIG. 19 is a diagram illustrating an example for explaining that the peak point acquired during the adjustment of the light amount of the light source is not used for calculating the degree of correlation between the visible light waveform and the infrared light waveform. In the graph of FIG. 19, the horizontal axis indicates time, the vertical axis indicates luminance, and the state in which the light amount of the light source is adjusted in the shaded area. White circles and black circles indicate the acquired peak points.

  As shown in FIG. 19, by adjusting the light amount of the light source, the luminance gain of the visible light waveform or infrared light waveform changes, and the kurtosis of the peak point also changes accordingly. When the peak point after the kurtosis is changed is filtered by the visible light waveform calculation unit 111 or the infrared light waveform calculation unit 112, the peak point is determined by the kurtosis of the peak of the raw waveform before the filter is applied. The position changes back and forth on the time axis. This error is not a problem as long as the heart rate is calculated as biometric information, but in the case of calculating blood pressure from the pulse wave propagation time, the influence of this error is large. Therefore, in the pulse wave measuring device 10 of the present disclosure, from the visible light waveform or the infrared light waveform acquired while the light amount of the illumination device 30 or the infrared light source 123 is controlled by the first to fourth control signals. It is not necessary to extract a predetermined feature point (that is, peak point).

  In other words, the visible light waveform calculation unit 111 includes a plurality of first visible light waveforms obtained in a period excluding a period in which the light amount of the illumination device 30 is controlled by the visible light control signal in the extraction of the plurality of first peak points. The first peak point is extracted. In addition, the visible light waveform calculation unit 111 includes a plurality of second visible light waveforms obtained in a period excluding a period in which the light amount of the illumination device 30 is controlled by the third control signal in the extraction of a plurality of third peak points. The third peak point is extracted.

  In addition, the infrared light waveform calculation unit 112 obtains the first red acquired in the period excluding the period in which the light amount of the infrared light source 123 is controlled by the infrared light control signal in the extraction of the plurality of second peak points. A plurality of second peak points are extracted from the external light waveform. In addition, the infrared light waveform calculation unit 112 extracts the second infrared light acquired in a period excluding the period in which the light amount of the infrared light source 123 is controlled by the fourth control signal in the extraction of the plurality of fourth peak points. A plurality of fourth peak points are extracted from the optical waveform.

  It should be noted that when the correlation coefficient of the heartbeat interval time in the visible light waveform and the infrared light waveform is smaller than the second threshold, the correlation degree calculation unit 113 has an excessive number of peak points in one or both waveforms. Assuming that the heartbeat interval time error and the standard deviation of each heartbeat interval time are calculated and the predetermined condition is satisfied, the heartbeat interval time between the inflection points between the top and bottom of the waveform is used. However, it is not limited to this. For example, even if the correlation coefficient between the first heartbeat interval time and the second heartbeat interval time is smaller than the second threshold, the correlation degree calculation unit 113 can appropriately acquire the peak points in both waveforms (for example, both When both the standard deviations in the heartbeat interval time of the waveform are equal to or smaller than the fourth threshold value), a “False” signal is transmitted to the light source control unit 115.

  In this way, the correlation degree calculation unit 113 outputs signals (for example, “True”, “False”, and the like corresponding to the calculated correlation degree and the extraction result of the predetermined feature points from the visible light waveform and the infrared light waveform. "False, RGB", "False, IR", or "False, Both") is transmitted to the light source control unit 115.

  As described above, the correlation degree calculation unit 113 performs the following determination based on the first heartbeat interval time and the second heartbeat interval time.

  That is, the correlation calculation unit 113 performs a second determination that determines whether the first standard deviation exceeds the fourth threshold and whether the second standard deviation exceeds the fourth threshold. . Further, when the correlation degree calculation unit 113 determines that the first standard deviation exceeds the fourth threshold and the second standard deviation exceeds the fourth threshold as a result of the second determination, One second heart beat interval time corresponding to the first heart beat interval time in time series among the first heart beat interval time of the first heart beat interval times and the plurality of second heart beat interval times. And a fourth determination as to whether or not the first time difference is greater than a sixth threshold value that is greater than the fifth threshold value.

  On the other hand, if the correlation calculation unit 113 determines that the first time difference is smaller than the fifth threshold as a result of the third determination and the fourth determination, whether or not the second standard deviation is less than or equal to the fourth threshold. A fifth determination is made to determine

  Further, the correlation calculation unit 113 may perform the following determination based on the third heartbeat interval time and the fourth heartbeat interval time.

  That is, the correlation calculation unit 113 performs a sixth determination that determines whether the third standard deviation exceeds the fourth threshold and whether the fourth standard deviation exceeds the fourth threshold. Further, as a result of the sixth determination, the correlation degree calculation unit 113 determines that the third standard deviation exceeds the fourth threshold and that the fourth standard deviation exceeds the fourth threshold. One fourth heart beat interval corresponding to the one third heart beat interval time among the plurality of third heart beat interval times and the one third heart beat interval time in time series. A seventh determination is made as to whether the second time difference from the time is less than a fifth threshold, and an eighth determination is made as to whether the second time difference is greater than a sixth threshold.

  On the other hand, when the correlation calculation unit 113 determines that the second time difference is smaller than the fifth threshold as a result of the seventh determination and the eighth determination, whether or not the fourth standard deviation is equal to or smaller than the fourth threshold. Ninth determination is performed.

(Control pattern acquisition unit)
The control pattern acquisition unit 114 is a control pattern for dimming the illumination device 30 outside the pulse wave measuring device 10, and acquires a control pattern predetermined for the illumination device 30. The control pattern acquisition unit 114 transmits the acquired control pattern to the light source control unit 115. Specifically, the control pattern acquisition unit 114 stores a plurality of control patterns of various types of lighting devices 30 for each model, and each time the lighting device 30 is recognized, Matching with the recognized lighting device 30 is performed, and a control pattern for controlling the recognized lighting device 30 is selected.

  For example, the control pattern acquisition unit 114 may store a product number of various manufacturers and a control pattern for controlling the lighting device corresponding to the product number. Thereby, for example, when the user uses the pulse wave measuring device 10 for the first time, the control pattern acquisition unit 114 receives the input of the product number of the illumination device 30 and selects the control pattern corresponding to the received product number. Thus, the control pattern may be acquired. The input from the user may be accepted by the pulse wave measuring device 10 if the pulse wave measuring device 10 has an input IF such as an input button, or via a remote control application activated by the mobile terminal 200. May be. In the latter case, the pulse wave measuring device 10 receives the product number input to the mobile terminal 200 from the mobile terminal 200. Thereby, the control pattern acquisition part 114 can recognize the control pattern according to each product number, and can select the control signal according to the product number.

  Note that the control pattern is not an ON / OFF signal, but can be determined, for example, depending on the type of lighting device, for example, a two-stage control pattern, a multi-stage illumination change pattern, and whether the color temperature can be changed. The device can automatically recognize the lighting device. That is, the control pattern includes a control pattern corresponding to the model of the lighting device 30, and may include at least one of the first control pattern, the second control pattern, the third control pattern, and the fourth control pattern. The first control pattern is a control pattern for adjusting the light amount and the color temperature. The second control pattern is, for example, a control pattern that adjusts the light amount in one stage of on / off. The third control pattern is a control pattern that adjusts the light amount in two stages, a first visible light amount and a second visible light amount that is smaller than the first visible light amount. The fourth control pattern is a control pattern for adjusting the amount of light in a stepless manner.

(Light source controller)
The light source controller 115 receives the amount of visible light from the illumination device 30 and the amount of infrared light from the infrared light source 123 according to the correlation degree and the signal corresponding to the extraction result received from the correlation degree calculator 113. Is determined to be increased, decreased, or maintained, and first to fourth control signals corresponding to the determination result are output to the illumination device 30 and the infrared light source 123.

  In addition, the light source control unit 115 acquires a control pattern used for dimming the lighting device 30 from the control pattern acquisition unit 114, and the light amount of the visible light LED 31 that is a light source of the lighting device 30 according to the acquired control pattern. And the amount of light is determined. Specifically, the light source control unit 115 uses the control pattern acquired by the control pattern acquisition unit 114 in accordance with the control of the amount of infrared light emitted from the infrared light source 123, and the amount of light emitted by the illumination device 30. A visible light control signal for controlling the light is output to the lighting device.

  When the light source control unit 115 receives the “False” signal, the correlation coefficient of the first and second heartbeat interval times in the visible light waveform and the infrared light waveform is smaller than the second threshold, but the heartbeat of each waveform It can be determined that the interval time has been acquired appropriately. At this time, the light source control unit 115 is in a state where the signal of the infrared light waveform is weak with respect to the visible light waveform and a predetermined feature point in each waveform can be acquired, but for example, the kurtosis of the peak point is small. It can be determined that the peak position is shifted each time by the filter processing or the like. Therefore, in this case, the light source control unit 115 determines the amount of light in the infrared light source 123 until the second inclination from the top to the bottom of the infrared light waveform becomes the first inclination A stored in the memory. increase.

  Further, when the light source control unit 115 receives the “TRUE” signal, the light source control unit 115 can determine that the predetermined feature points in the visible light waveform and the infrared light waveform match. For this reason, the light source control unit 115 reduces the amount of visible light in the illumination device 30, and stores the amount of infrared light in the infrared light source 123 as a second inclination from the top to the bottom of the infrared light waveform. Is increased until the first inclination A stored in FIG. In other words, the light source control unit 115 decreases the amount of visible light in the visible light source and increases the amount of infrared light in the infrared light source when the correlation degree is equal to or greater than the second threshold. Further, when the amount of infrared light is increased, the amount of infrared light is increased until the second slope in the infrared light waveform becomes the first slope A stored in the memory (storage 103).

  The acquisition of the third visible light image, the extraction of the second visible light waveform, the acquisition of the second infrared light image, the extraction of the second infrared light waveform, and the calculation of the second correlation coefficient are performed as pulse wave calculations. It is repeatedly performed in each processing unit of the apparatus 100. In the calculation of the second correlation coefficient that is repeatedly performed, the light source control unit 115 compares the second inclination with the first inclination stored in the memory, and the second inclination becomes the first inclination. Until it is, the infrared light control signal is output to the infrared light source 123.

  For example, when the light source control unit 115 receives a “False, IR” signal, the light source control unit 115 can determine that the infrared light waveform calculation unit 112 has not properly acquired a predetermined feature point in the infrared light waveform. That is, for example, the “False, IR” signal indicates that there is a lot of noise in the infrared light waveform. For this reason, the light quantity in the illumination device 30 is not adjusted, and the light quantity in the infrared light source 123 is increased.

  That is, if the light source control unit 115 determines that the absolute error e is greater than the sixth threshold value (200 [ms]) as a result of the third determination and the fourth determination, it is the infrared light. A control signal is output to the infrared light source 123. In addition, when the light source control unit 115 determines that the absolute error e is greater than the sixth threshold value (200 [ms]) as a result of the seventh determination and the eighth determination, it is infrared light. A control signal is output to the infrared light source 123. The light source control unit 115 increases the amount of light in the infrared light source 123 by outputting an infrared light control signal to the infrared light source 123.

  When the light source control unit 115 receives the “False, RGB” signal, the light source control unit 115 can determine that the visible light waveform calculation unit 111 has not properly acquired a predetermined feature point in the visible light waveform. In this case, the light source control unit 115 cannot determine whether or not the infrared light waveform calculation unit 112 can appropriately acquire a predetermined feature point in the infrared light waveform. Therefore, for example, if the standard deviation of the heartbeat interval time in the first predetermined period is equal to or smaller than the fourth threshold value in the infrared light waveform, the light source control unit 115 decreases the light amount of the light source in the illumination device 30 and The light amount of the light source in the light source 123 is increased until the slope between the top and bottom of the infrared light waveform becomes A. If the standard deviation in the infrared light waveform exceeds the fourth threshold, the light source control unit 115 determines that both signals cannot be acquired, and changes the signal to “False, Both”.

  That is, when the light source control unit 115 determines that the second standard deviation is equal to or smaller than the fourth threshold value as a result of the fifth determination, the light source control unit 115 outputs a visible light control signal to the illumination device 30 and the infrared light control signal. Is output to the infrared light source 123. If the light source control unit 115 determines that the second standard deviation is greater than the fourth threshold value, the light source control unit 115 outputs the third control signal to the illumination device 30 and outputs the fourth control signal to the infrared light source. As described above, the fifth determination is performed when it is determined that the first time difference is smaller than the fifth threshold as a result of the third determination and the fourth determination, and the second standard deviation is the fourth threshold. It is a determination of whether or not:

  In addition, when the light source control unit 115 determines that the fourth standard deviation is equal to or smaller than the fourth threshold value as a result of the ninth determination, the light source control unit 115 outputs a visible light control signal to the illumination device 30 and the infrared light control signal Is output to the infrared light source 123. If the light source control unit 115 determines that the fourth standard deviation is greater than the fourth threshold value as a result of the ninth determination, the light source control unit 115 outputs the third control signal to the illumination device 30 and outputs the fourth control signal to the infrared light. Output to the light source 123. As described above, the ninth determination is performed when it is determined that the second time difference is smaller than the fifth threshold as a result of the seventh determination and the eighth determination, and the fourth standard deviation is the fourth threshold. It is a determination of whether or not:

  Further, when the light source control unit 115 receives the “FALSE, Both” signal, the light source control unit 115 can determine that a predetermined feature point has not been acquired in both the visible light waveform and the infrared light waveform. In this case, the light source control unit 115 increases the amount of light of the illumination device 30 until the slope from the top to the bottom of the visible light waveform reaches the first slope A. Note that the light source control unit 115 may increase the light amount of the illumination device 30 until the initial light amount of the visible light waveform is stored in the memory. Further, the light source control unit 115 reduces the light amount of the infrared light source 123 to zero. That is, the light source control unit 115 is the state in which the light quantity of the illumination device 30 and the infrared light source 123 can be most reliably acquired when predetermined feature points cannot be acquired in both the visible light waveform and the infrared light waveform. The light quantity is set to the initial state, and the light quantity is adjusted again.

  That is, as a result of the third determination and the fourth determination, the light source control unit 115 determines that the absolute error e as the first time difference is not less than the fifth threshold and not more than the sixth threshold. The light is output to the illumination device 30 and the fourth control signal is output to the infrared light source 123. Further, when the light source control unit 115 determines that the absolute error e as the second time difference is not less than the fifth threshold and not more than the sixth threshold as a result of the seventh determination and the eighth determination, the light source control unit 115 outputs the third control signal. The light is output to the illumination device 30 and the fourth control signal is output to the infrared light source 123. The light source control unit 115 outputs the third control signal to the illumination device 30, thereby increasing the light amount of the illumination device 30, and outputs the fourth control signal to the infrared light source 123, thereby causing the infrared light source 123. Reduce the amount of light.

  That is, the light source control unit 115 has a case where the standard deviation of the plurality of first heart beat interval times exceeds the fourth threshold value and the standard deviation of the plurality of second heart beat interval times exceeds the fourth threshold value. If the difference between the first heartbeat interval time and the second heartbeat interval time corresponding to each other in the time series is smaller than the fifth threshold value ((−1) × third threshold value), When the amount of infrared light in the infrared light source 123 is increased and the amount of infrared light is increased, the second slope in the infrared light waveform is stored in the memory. The amount of infrared light is increased until the slope A is reached.

  In addition, the light source control unit 115 has a case where the standard deviation of the plurality of first heartbeat interval times exceeds the fourth threshold value, and the standard deviation of the plurality of second heartbeat interval times exceeds the fourth threshold value. When the difference between the first heartbeat interval time and the second heartbeat interval time corresponding to each other in the time series is larger than the sixth threshold (that is, the third threshold), the amount of infrared light in the infrared light source 123 When the amount of infrared light increases, the amount of infrared light is increased until the second slope in the infrared light waveform becomes the first slope A stored in the memory.

  In addition, the light source control unit 115 has a case where the standard deviation of the plurality of first heartbeat interval times exceeds the fourth threshold value, and the standard deviation of the plurality of second heartbeat interval times exceeds the fourth threshold value. When the difference between the first heartbeat interval time and the second heartbeat interval time corresponding to each other in the time series is a value between the fifth threshold value and the sixth threshold value, the amount of visible light in the illumination device 30 And the amount of infrared light in the infrared light source 123 is decreased.

  Note that the light source control unit 115 reduces the light amount of the infrared light source 123 to red unless a predetermined feature point cannot be acquired in both the visible light waveform and the infrared light waveform, such as “False, Both”. Although the second inclination of the external light waveform is increased until it reaches the first inclination A, the present invention is not limited to this. For example, when the average luminance value in the ROI exceeds a seventh threshold value, for example, 240, the light source control unit 115 embeds an image captured from the user's skin in the noise information because the light amount of the light source is too strong. End up. The average luminance “240” is “240” from 0 to 255 indicating the luminance, and the larger the value is, the higher the luminance is. For this reason, in this case, the light source control unit 115 is considered that the second inclination of the infrared light waveform exceeds the first inclination A. Therefore, the second inclination becomes the first inclination A. The amount of infrared light may be reduced.

  FIG. 20 is a diagram showing an example of the simplest steps for decreasing the light amount of the visible light source to zero and increasing the light amount of the infrared light source to an appropriate light amount using the pulse wave measurement device. is there. In all graphs (a) to (d) in FIG. 20, the horizontal axis represents time, and the vertical axis represents luminance. In FIG. 20, the visible light waveform is represented as RGB, and the infrared light waveform is represented as IR.

  (A) of FIG. 20 is a figure which shows the visible light waveform and infrared light waveform which were acquired in the initial state which the user turned on the illuminating device 30 by the pulse wave measuring device 10. FIG. The visible light waveform in (a) of FIG. 20 is a waveform having the largest inclination from the top to the bottom among the visible light waveforms in (a) to (d) of FIG. Therefore, the inclination from the top to the bottom of the visible light waveform at this time is stored in the memory as the first inclination A.

  At this time, the infrared light source 123 is OFF. For this reason, an infrared light waveform is hardly acquired. In this state, the correlation calculation unit 113 transmits, for example, a signal “False, IR” to the light source control unit 115. Therefore, the light source control unit 115 increases the light amount of the infrared light source 123 in the infrared light source 123. At this time, as the light amount of the infrared light source 123 is increased, the infrared light waveform calculation unit 112 can acquire a predetermined feature point of the infrared light waveform, and can acquire the second heartbeat interval time. In addition, the acquired standard deviation of the second heartbeat interval time falls within the fourth threshold. Then, as shown in FIG. 20 (b), the second deviation between the apex-bottom points of the infrared light waveform is maintained while maintaining the standard deviation of the second heartbeat interval time within the fourth threshold. The light amount of the infrared light source 123 is increased until the inclination becomes the first inclination A. When the second inclination becomes the first inclination A, the correlation calculation unit 113 transmits, for example, a signal of “TRUE, AMP = A” to the light source control unit 115. For this reason, the light source control unit 115 once stops the adjustment of the light source when the signal of “TRUE, AMP = A” is received.

  Next, from the state of FIG. 20B, the light source control unit 115 decreases the light amount of the visible light source in the illumination device 30. FIG. 20C shows a state in which the standard deviation of the heartbeat interval time is equal to or smaller than the fourth threshold in the infrared light waveform calculation unit 112, and the light source in the illumination device 30 is OFF. FIG. 20D further shows a state where the light source in the illumination device 30 is OFF and the second slope in the infrared light waveform is the first slope A, that is, the final state. It is a state to aim for.

  In the process of changing from the state of FIG. 20B to the state of FIG. 20C, the amount of visible light is decreased by a constant interval, for example, by 1W. Then, every time the amount of visible light is decreased, the infrared light waveform calculation unit 112 and the correlation degree calculation unit 113 confirm whether or not a predetermined feature point can be appropriately acquired in the infrared light waveform. Further, if the infrared light waveform calculation unit 112 and the correlation degree calculation unit 113 can confirm that predetermined feature points can be appropriately acquired in the infrared light waveform, as shown in FIG. The amount of light in the light source of the external light source 123 is increased until the second slope in the infrared light waveform becomes the first slope A.

  Therefore, in the process of changing from the state of FIG. 20B to the state of FIG. 20C, the correlation calculation unit 113 sends a “True” signal or “False,” to the light source control unit 115. IR signal is transmitted, and the light source control unit 115 adjusts the light amount of the infrared light source 123 until “True” is received every time the “False, IR” signal is received. When the light source control unit 115 receives “False, RGB” from the correlation calculation unit 113 by reducing the light amount of the illumination device 30, the light source control unit 115 ends this process.

  Alternatively, in the process of changing from the state of FIG. 20C to the state of FIG. 20D, the correlation calculation unit 113 transmits a “False, RGB” signal to the light source control unit 115, and the light source The control unit 115 continues to increase the light amount of the light source in the infrared light source 123 until the second slope in the infrared light waveform reaches the first slope A. For example, the visible light waveform cannot be obtained, and the first When the signal of “False, RGB, AMP = A” indicating that the inclination of 2 has become the first inclination A is received from the correlation calculation unit 113, the control of the light amount of the light source by the light source control unit 115 is finished. .

  In addition, the light source control unit 115 acquires, in the visible light waveform calculation unit 111 or the infrared light waveform calculation unit 112, two or more predetermined feature points that are continuous from the visible light waveform or the infrared light waveform, respectively. After completion, the light source is controlled. In other words, the light source control unit 115 outputs the infrared light control signal until two or more first peak points continuous from the first visible light waveform are extracted within the second predetermined period or the second visible light. The output of the infrared light control signal is waited until two or more third peak points continuous from the optical waveform are extracted within the second predetermined period. In addition, the light source control unit 115 outputs the infrared light control signal until two or more second peak points continuous from the first infrared light waveform are extracted within the second predetermined period, or the second The output of the infrared light control signal is waited until two or more continuous fourth peak points from the infrared light waveform are extracted within the second predetermined period.

  FIG. 21 illustrates that the light source control is waited until two or more predetermined feature points continuous from the waveform are extracted within the second predetermined period in each of the visible light waveform and the infrared light waveform. FIG. The graph in FIG. 21 shows a visible light waveform or an infrared light waveform. In the graph of FIG. 21, the horizontal axis represents time, and the vertical axis represents luminance.

  When the light source control unit 115 changes the light amount of the illumination device 30 or the infrared light source 123, the luminance gain of the visible light waveform or the infrared light waveform changes. When the luminance gain changes, the position of the pulse wave timing shifts, so that a large error occurs in the calculation of timing such as the heartbeat interval time. In the present disclosure, the heartbeat interval time is mainly used as a material for determining the degree of correlation between the visible light waveform and the infrared light waveform, and two consecutive peak points are necessary to calculate the heartbeat interval time. It is. Therefore, as shown in FIG. 21, the light source control unit 115 adjusts the amount of light source after confirming that two or more peak points are continuously taken in the visible light waveform or the infrared light waveform.

  So far, the control method in the light source control unit 115 when the lighting device 30 is completely dimmable and controllable (that is, when the lighting device is dimmable by the fourth control pattern) has been described. Next, the case where the lighting device 30 is a lighting device that can be adjusted by the second control pattern and the case that it is a lighting device that can be adjusted by the third control pattern will be described.

  The basic control method is as described in the illumination control pattern storage unit 110, but a characteristic case in the light source control method for the determination in the correlation degree calculation unit 113 will be described.

  The illumination device 30 has a second control pattern that adjusts the light amount in one step of on / off, or the first visible light amount and the second visible light amount, as compared with the case of the fourth control pattern that adjusts the light amount steplessly. In the case of a device that is dimmed by the third control pattern that adjusts the light amount in two stages, the light amount of visible light cannot be freely changed to an arbitrary light amount.

  Therefore, for example, when the light source control unit 115 receives a signal of “TRUE” from the correlation degree calculation unit 113, the second inclination in the infrared light waveform of the infrared light source becomes the first inclination A. And a control signal for increasing to a range where a predetermined feature point (that is, peak point) of the infrared light waveform can be detected is output to the infrared light source 123 as an infrared light control signal. To do.

  After outputting the infrared light control signal, the light source control unit 115 outputs a control signal that lowers the amount of illumination light in the illumination device 30 by one step as a visible light control signal.

  Specifically, when the lighting device 30 is a device that is dimmed according to the second control pattern, the light source control unit 115 uses, as a visible light control signal, a control signal that turns the lighting device 30 from an on state to an off state. Output.

  In addition, when the lighting device 30 is a device that is dimmed according to the third control pattern, the light source control unit 115 determines the first visible light amount to the first visible light amount if the light amount of the lighting device 30 is the first visible light amount. A control signal for changing to a second visible light amount smaller than the light amount is output as a visible light control signal. In addition, when the lighting device 30 is a device that is dimmed according to the third control pattern, the light source control unit 115 controls the lighting device 30 to be turned off if the light amount of the lighting device 30 is the second visible light amount. The signal is output as a visible light control signal.

<< First control pattern >>
Next, the case where the light control of the illumination device 30 is performed by adjusting the light amount and the color temperature will be described.

  When the illumination device 30 is a device that is dimmed by the fourth control pattern that adjusts the light amount and the color temperature, first, the color temperature of the visible light irradiated by the illumination device 30 is set to a predetermined color temperature or lower, for example, 2500K or lower. Then, the above-described light source switching control is performed.

  FIG. 22 is a diagram for explaining a difference in how a user's face is seen in the visible light imaging unit 122 due to a change in color temperature. FIG. 22A is a diagram showing an example of an image obtained by capturing the user's face in daily lighting, for example, day white (about 5000 K), and FIG. 22B shows the color temperature. It is a figure which shows an example of the image which imaged the user's face at the time of making low and irradiating a light bulb color (about 2500K). At this time, the pulse wave measuring apparatus 10 uses a hue signal of hue H calculated from the RGB luminance signal, instead of changing the algorithm to be used and acquiring the visible light waveform from the RGB luminance signal.

  FIG. 23 is a diagram for explaining calculation processing for calculating a hue signal of hue H from RGB luminance signals. (A)-(c) of FIG. 23 is a graph which shows each RGB signal (visible light waveform) which can be acquired by the visible light imaging part 122. In FIG. In each of (a) to (c) of FIG. 23, the horizontal axis indicates time, and the vertical axis indicates the luminance of each RGB. FIG. 23D is a graph showing a hue H signal (hue waveform) calculated from these three signals. In FIG. 23D, the horizontal axis represents time, and the vertical axis represents the angle in the hue circle. Note that 0 degrees in the hue circle is a state in which there is gain in the R signal, and the other G and B signals are zero. Further, the hue signal is calculated from the RGB luminance signal according to Equation 3.

  In Equation 3, “R” is the luminance value of the R signal (red signal), “B” is the luminance value of the B signal (blue signal), and “G” is the luminance value of the G signal (green signal).

  Expression 3 is an expression when the luminance signals in the colors are in the order of R> G> B, but the user's skin color basically satisfies this relationship, and the above expression 3 is used. be able to. In this way, when the RGB luminance signal is converted into a hue signal using Expression 2, as shown in FIG. 24, the user's skin color exists in the hue range of 0 degrees to 60 degrees in the hue ring. Expressed as a color. In other words, by using a hue signal of hue H instead of an RGB luminance signal, the luminance component included in the RGB luminance signal can be canceled, and a change in color component can be obtained. For this reason, it is possible to reduce the influence of noise due to luminance changes.

  That is, the light source control unit 115 outputs a control signal for adjusting the color temperature of the lighting device 30 to a predetermined temperature (for example, 2500K) to the lighting device 30 as a color temperature control signal. Then, the visible light waveform calculation unit 111 calculates the hue obtained from the third visible light image acquired after the output of the color temperature control signal using Expression 3, and uses the calculated hue to display the visible light waveform. To extract.

  Furthermore, by performing the control of setting the color temperature to 2500 K or lower first, reddish light such as a light bulb color is applied to the user's cheek. Thereby, as a result of imaging by the visible light imaging unit 122, a result that the color change of the user's skin surface vibrates around 30 degrees of the hue circle is obtained. At this time, the 30 degree axis is characterized by intersecting perpendicularly with the RGB G signal axes, and is therefore most susceptible to changes in the G signal in which changes in pulse waves are likely to occur. Therefore, by changing the color temperature of the lighting device 30 so that the color of the user's skin surface changes from white to reddish, in particular, the value of the hue H is around 30 degrees, more body movement and environmental noise are achieved. On the other hand, a visible light waveform can be acquired robustly.

  That is, the visible light waveform calculation unit 111 obtains the user who has been irradiated with visible light having a predetermined color temperature by the lighting device 30 after the output of the color temperature control signal, in the visible light region. The obtained third visible light image is acquired. And the visible light waveform calculating part 111 calculates the hue of the acquired 3rd visible light image, and extracts the hue waveform which is a waveform which shows a user's pulse wave from the calculated hue. The light source control unit 115 illuminates the lighting device 30 so that the extracted hue waveform falls within a hue range (for example, a range of 0 degrees to 60 degrees) based on a predetermined reference value (for example, 30 degrees in the hue circle). A control signal for adjusting the color temperature is output to the lighting device 30 as a color temperature control signal.

  Here, FIG. 25 is a diagram illustrating a hue waveform acquired when converted into different hue ranges. FIG. 25A shows the hue circle. (B) of FIG. 25 is acquired when the color temperature of the lighting device 30 is adjusted so that the extracted hue waveform falls within the hue range of 60 degrees to 120 degrees with respect to 90 degrees in the hue circle. The hue waveform is shown. (C) of FIG. 25 is acquired when the color temperature of the illuminating device 30 is adjusted so that the extracted hue waveform falls within the hue range of 0 degrees to 60 degrees with respect to 30 degrees in the hue circle. The hue waveform is shown. (D) of FIG. 25 is acquired when the color temperature of the lighting device 30 is adjusted so that the extracted hue waveform falls within a hue range of −60 degrees to 0 degrees with respect to 90 degrees in the hue circle. Shows the hue waveform.

  As shown in FIG. 25, when the color temperature of the lighting device 30 is adjusted so that the extracted hue waveform falls within the hue range of 0 degree to 60 degrees with reference to 30 degrees in the hue circle, Compared with the case where the color temperature is adjusted to the hue range, it can be seen that a clear waveform that is hardly affected by noise due to a luminance change can be obtained.

  Furthermore, since the light bulb color has the effect of giving the user a relaxing effect and making it easier to sleep, it is also advantageous for the user to change the color temperature from white (5000 K) to red (2500 K). .

  The adjustment of the color temperature of the visible light output from the lighting device 30 may be performed as follows.

  First, the control pattern acquisition unit 114 acquires a first control pattern that defines the first correspondence from the illumination device 30 provided outside the pulse wave measuring device 10. The first correspondence relationship indicates a plurality of instructions and a plurality of color temperatures of visible light output from the illumination device 30. The plurality of instructions and the plurality of color temperatures have a one-to-one relationship. The pulse wave calculation device 100 holds information indicating the first color temperature that is held in advance. Next, the light source control unit 115 determines a first instruction corresponding to the first color temperature with reference to the first correspondence relationship defined in the first control pattern. Next, the light source control unit 115 outputs a first instruction to the illumination device 30. Next, the illumination device 30 irradiates the user with visible light having a color temperature corresponding to the first instruction. Next, the visible light imaging unit 122 captures a user irradiated with visible light having a color temperature corresponding to the first instruction in the visible light region, and acquires a plurality of first visible light images. Next, the visible light waveform calculation unit 111 calculates a first plurality of hues from the plurality of first visible light images, and extracts a first hue waveform that is a waveform indicating a user's pulse wave from the first plurality of hues. To do. Details of extracting the hue waveform have already been described in detail with reference to FIG. The light source control unit 115 determines whether or not the amplitude of the first hue waveform belongs to a predetermined hue range. If the light source control unit 115 determines that the amplitude belongs, the process proceeds to the light source switching control described above. The light source control unit 115 determines whether or not the amplitude of the first hue waveform belongs to a predetermined hue range. If the light source control unit 115 determines that the amplitude does not belong, the process proceeds to the following process. The light source control unit 115 determines a second instruction corresponding to a second color temperature different from the first color temperature with reference to the first control pattern, and outputs the second instruction to the lighting device 30. Next, the illumination device 30 irradiates the user with visible light having a color temperature corresponding to the second instruction. Next, the visible light imaging unit 122 is irradiated with visible light having a color temperature corresponding to the second instruction, captures the user in the visible light region, and acquires a plurality of fourth visible light images. The visible light waveform calculation unit 111 calculates a second plurality of hues from the plurality of fourth visible light images, and extracts a second hue waveform that is a waveform indicating a user's pulse wave from the calculated second plurality of hues. . Details of the extraction of the hue waveform have already been described in detail with reference to FIG. Next, the light source control unit 115 determines whether the amplitude of the second hue waveform belongs to a predetermined hue range. When the light source control unit 115 determines that the amplitude of the second hue waveform belongs to the predetermined hue range, the processing shifts to the light source switching control described above. In the light source switching control, the correlation calculation unit 113 extracts a visible light waveform that is a waveform indicating the user's pulse wave using the plurality of fourth visible light images, and extracts the visible light waveform and the plurality of infrared light images. The degree of correlation may be obtained from an infrared light waveform that is a waveform indicating the user's pulse wave to be extracted.

<< Second control pattern >>
FIG. 26 shows a light source that decreases the light amount of the visible light source when the illumination device is a device that is dimmed by the second control pattern until it becomes zero and increases the light amount of the infrared light source to an appropriate light amount. It is a figure for demonstrating switching control of. (A) of FIG. 26 is a graph which shows the change of the voltage according to each light quantity in each of the illuminating device 30 and infrared light source 123 which are visible light sources. In the graph of FIG. 26A, the horizontal axis indicates time, and the vertical axis indicates voltage according to the amount of light. FIGS. 26B and 26C both show the visible light waveform and the infrared light waveform when the voltage applied to each light source is changed as shown in FIG. In the graphs of FIGS. 26B and 26C, the horizontal axis indicates time, and the vertical axis indicates luminance.

  In the light source switching control from the illuminating device 30 that is a light source that emits visible light to the infrared light source 123, T is a completion time for completing the switching of the light source. The completion time T is, for example, about 2 to 10 minutes after the start of switching. Thereby, a visible light waveform and an infrared light waveform can be acquired and compared more accurately.

  As shown in FIG. 26, when the lighting control stage of the lighting device 30 is one stage, the lighting device 30 is light-controlled by turning on or off the visible light irradiated. Therefore, the pulse wave measuring device 10 adjusts the light amount in the infrared light source 123 while the illumination device 30 is turned on to adjust the light amount so that the user's pulse wave can be acquired under infrared light. There is a need. Specifically, the light source controller 115 uses the infrared light source 123 as a control signal that increases the amount of infrared light emitted from the infrared light source 123 by a predetermined first change amount. Output to. When the infrared light source 123 receives the infrared light control signal, a predetermined voltage as shown in FIG. 26A is applied so that the amount of light increases by the first change amount. As shown in b) and (c), the amount of light increases by the first change amount.

  The infrared camera 24 that is hardware of the infrared imaging unit 124 is also affected by light in the wavelength band of the visible light region. For this reason, the pulse wave measuring device 10 predicts a decrease in luminance due to turning off the illumination device 30 in the infrared light imaging unit 124 in advance before turning off the illumination device 30, and the infrared light source 123. It is necessary to increase the amount of light.

  Then, the light source control unit 115 outputs a control signal for turning off the illumination device 30 to the illumination device 30 as a visible light control signal. When the illumination device 30 receives the visible light control signal, the illumination device 30 is turned off and is not irradiated with visible light. As described above, since the pulse wave measuring device 10 increases the amount of light in the infrared light source 123 in advance even when the lighting device 30 is turned off, the feature point (infrared light waveform) ( For example, the timing of the peak point etc. can be acquired.

  The light source control unit 115 may learn the adjustment of the light amount of the infrared light source 123 by overlapping the number. For example, as shown in FIG. 22 (c), the visible light in the illumination device 30 is insufficient because the amount of increase in the amount of light in the infrared light source 123 is insufficient when the above switching control is performed only once. When is turned off, the characteristic point (for example, peak point) of the infrared light waveform may not be acquired. In this case, the light source control unit 115 further increases the amount of light in the infrared light source 123 until a feature point of the infrared light waveform can be acquired. Then, the light source control unit 115 further increases after the illumination device 30 is turned off and the light amount of the infrared light immediately before the illumination device 30 is turned off when the feature amount of the infrared light waveform can be acquired. The light amount of the infrared light source 123 that has been made may be stored, and the sum of these light amounts may be set as the light amount of the infrared light source 123 that is set immediately before the illumination device 30 is turned off in the next switching control. Thereby, the pulse wave measuring device 10 can reduce the probability of failing to acquire the infrared light waveform each time, and can more effectively acquire the pulse wave during the user's sleep.

<< Third control pattern >>
Next, the case where the dimming stage of the illuminating device 30 has two stages will be described.

  FIG. 27 is a diagram for describing light source switching control when the illumination device is a device that is dimmed according to the third control pattern. (A) of FIG. 27 is a graph which shows the change of the voltage according to each light quantity in each of the illuminating device 30 and infrared light source 123 which are visible light sources. In FIG. 27A, the horizontal axis indicates time, and the vertical axis indicates a voltage corresponding to the amount of light. FIG. 27B shows a visible light waveform and an infrared light waveform when the voltage applied to each light source is changed as shown in FIG. In FIG. 27B, the horizontal axis represents time, and the vertical axis represents luminance. Similarly to FIG. 26, the completion time in the switching control is T.

  As shown in FIGS. 27A and 27B, when the dimming stage of the illumination device 30 is in two stages, even if the visible light falls from the first visible light quantity to the second visible light quantity by one stage. Since the second visible light amount is not 0, a visible light waveform can be acquired. Therefore, in the case of two stages, first, the first one stage of light control is performed. That is, the light source control unit 115 controls the light amount of the infrared light emitted from the infrared light source 123 to a second infrared light amount that is increased from the first infrared light amount by a predetermined second change amount. Is output to the infrared light source 123 as an infrared light control signal. Then, the light source control unit 115 outputs a control signal for changing the illumination device 30 from the first visible light amount to the second visible light amount to the illumination device 30 as a visible light control signal.

  When receiving the infrared light control signal, the infrared light source 123 applies a predetermined voltage as shown in the first stage change in FIG. 27A so that the light amount increases by the second change amount. Then, as shown in FIG. 27B, the amount of light increases by the second change amount. Further, when receiving the visible light control signal, the lighting device 30 changes the light amount from the first visible light amount to the second visible light amount.

  At this time, the pulse wave measuring device 10 can grasp the amount of decrease in the luminance of visible light due to the voltage drop in the lighting device 30 in the first-stage dimming. Thereby, in the next one-stage dimming, the luminance drop of visible light due to the voltage drop can be predicted.

  That is, the light source control unit 115 changes the brightness of the infrared light obtained from the first and second infrared light images before and after outputting the infrared light control signal, and the first and second before and after outputting the visible light control signal. The third change amount in the amount of infrared light of the infrared light source 123 is determined according to the luminance change of visible light obtained from the third visible light image.

  The first infrared light image is an infrared light image captured by the infrared light imaging unit 124 before the infrared light control signal is output, and the second infrared light image is the infrared light control. It is an infrared light image captured by the infrared light imaging unit 124 after the signal is output. The second visible light image is a visible light image captured by the visible light imaging unit 122 before the visible light control signal is output, and the third visible light image is after the visible light control signal is output. It is a visible light image imaged by the visible light imaging unit 122.

  The third change amount determined here is, for example, a value equal to or greater than the brightness change of infrared light that can affect the infrared light image when the lighting device 30 is turned off by the second-stage dimming. May be. Then, the light source control unit 115 sends a control signal for controlling the third infrared light amount increased by the determined third change amount from the second infrared light amount to the infrared light source 123 as an infrared light control signal. Output. Thereafter, the light source control unit 115 outputs a control signal for turning off the lighting device 30 to the lighting device 30 as a visible light control signal.

  When the infrared light source 123 receives the infrared light control signal, the infrared light source 123 applies a predetermined voltage as shown in the second stage change in FIG. 27A so that the amount of light increases by the third change amount. Then, as shown in FIG. 27B, the amount of light increases by the third change amount. When the illumination device 30 receives the visible light control signal, the illumination device 30 turns off and does not emit visible light.

  Thus, when the illuminating device 30 is a device that is dimmed according to the third control pattern, the pulse wave measuring device 10 obtains the amount of decrease in the luminance of visible light in the first-stage dimming, and obtains the obtained decrease. By increasing the amount of light from the infrared light source 123 according to the amount, the infrared light waveform can be acquired more effectively.

  In addition, as the lighting device 30 performs dimming from three stages to more stages, the infrared light waveform can be acquired more effectively by repeating the same principle as above for each stage. .

  When the light source control unit 115 receives the signal “FALSE, BOTH”, the light source control unit 115 returns the light amount of visible light in the illumination device 30 to the brightest light amount, and then changes the light amount of the infrared light source 123 to the infrared light again. The process of increasing is repeated until the second slope of the optical waveform becomes the first slope A.

<< 4th control pattern >>
Next, the case where the light control stage of the illuminating device 30 is stepless will be described.

  FIG. 28 is a diagram for describing an example of light source switching control when the illumination device is a device that is dimmed according to the fourth control pattern. (A) of FIG. 28 is a graph which shows the change of the voltage according to each light quantity in each of the illuminating device 30 and infrared light source 123 which are visible light sources. In FIG. 28A, the horizontal axis indicates time, and the vertical axis indicates voltage corresponding to the amount of light. FIG. 28B shows a visible light waveform and an infrared light waveform when the voltage applied to each light source is changed as shown in FIG. In FIG. 28B, the horizontal axis indicates time, and the vertical axis indicates luminance.

  As shown to (a) of FIG. 28, when the step of the light control of the illuminating device 30 is a non-step, the light quantity of visible light reduces linearly when the applied voltage is lowered linearly, The power is turned off at completion time T. On the other hand, it can be seen that the amount of infrared light in the infrared light source 123 increases linearly as the applied voltage increases linearly. At this time, the visible light waveform decreases as the voltage changes, and the infrared light waveform increases as the voltage changes, as shown in FIG. Therefore, when the illumination device 30 is a device that is dimmed according to the fourth control pattern, even if the switching from the visible light waveform acquisition to the infrared light waveform acquisition is not successful, the linear increase / decrease is performed. Therefore, it is possible to acquire a pulse wave in infrared light while performing fine adjustment. Further, unlike the illumination device in the case where there is a stage, fine adjustment is possible, so that the feature points of the infrared light waveform in the infrared light can be acquired while capturing the feature points of the visible light waveform in the visible light.

  In the case where the dimming stage of the illumination device 30 is stepless, the visible light is linearly decreased and the infrared light is linearly increased. However, the present invention is not limited to this. For example, as illustrated in FIG. 29, the light source control unit 115 has a predetermined threshold value, for example, 50 to 200 lux, for the illuminance by the lighting device 30, and the infrared light waveform calculation unit 112 performs feature points of the infrared light waveform Can be acquired, the illumination device 30 whose luminance is controlled so that the illuminance becomes a predetermined threshold may be turned off. By controlling in this way, compared with the case where it reduces linearly until it turns off visible light, it can turn off earlier and can introduce into comfortable sleep.

  Here, FIG. 29 is a diagram illustrating an example of switching control to be turned off when the illuminance of the lighting device is a predetermined threshold. (A) of FIG. 29 is a graph which shows the change of the voltage according to each light quantity in each of the illuminating device 30 and infrared light source 123 which are visible light sources. In FIG. 29A, the horizontal axis indicates time, and the vertical axis indicates voltage according to the amount of light. FIG. 29B shows a visible light waveform and an infrared light waveform when the voltage applied to each light source is changed as shown in FIG. In FIG. 29B, the horizontal axis indicates time, and the vertical axis indicates luminance.

  That is, in this case, the pulse wave measuring device 10 increases the amount of infrared light in the infrared light source 123 when the degree of correlation calculated by the degree-of-correlation calculating unit 113 is equal to or greater than a predetermined threshold (second threshold). The control signal to be output is output to the infrared light source 123 as an infrared light control signal, and the control signal for reducing the amount of visible light in the illumination device 30 is output to the illumination device 30 as a visible light control signal. After the acquisition of the second visible light waveform, the acquisition of the second infrared light image, and the extraction of the second infrared light waveform, the calculation of the degree of correlation is further repeated. The second visible light waveform is a waveform indicating the user's pulse wave extracted from the third visible light image. A 2nd infrared-light waveform is a waveform which shows a user's pulse wave extracted from the 2nd infrared-light image.

  Then, the light source control unit 115 illuminates the lighting device 30 when the light intensity of the lighting device 30 is equal to or smaller than the second threshold value and the correlation degree is equal to or larger than the predetermined threshold value as a result of repeated correlation degree calculation. A control signal for turning off the light may be output to the lighting device as a visible light control signal. Note that the second threshold value in this case is the amount of light by the illumination device 30 when the illuminance is a predetermined threshold value.

  The lighting device 30 sets the completion time for switching as T in order to perform switching control of the light source from the lighting device 30 that is a light source that radiates visible light more effectively to the infrared light source 123. However, it is not limited to this. In particular, when the lighting control stage of the lighting device 30 is stepless, switching control with an earlier adjustment time may be performed in accordance with an instruction from the user. Some users feel painful that the visible light is controlled during the completion time T (for example, between 2 to 10 minutes) for switching the light source every time they sleep. Therefore, as shown in FIG. 5B, for example, two types of switching control of “normal mode” and “time reduction mode” may be prepared. When the “normal mode” is selected by the user, the pulse wave measuring apparatus 10 performs switching control over the normally set completion time T. In addition, when the “time reduction mode” is selected by the user, priority is given to the speed rather than the accuracy of obtaining the visible light waveform and the infrared light waveform. For example, the completion time for switching is set to T / 3 ( For example, the switching control may be performed by using a visible light waveform and an infrared light waveform acquired during this period from about 30 seconds to about 3 minutes.

  In other words, the pulse wave measuring device 10 executes any one of normal processing in the normal mode and short-time processing in the time reduction mode. In the normal processing, when the calculated degree of correlation is equal to or greater than a predetermined threshold, a control signal for increasing the amount of infrared light at the infrared light source 123 at a first speed is output as an infrared light control signal, and the illumination device A control signal for reducing the amount of visible light at 30 at a second speed is output as a visible light control signal, acquisition of a third visible light image, extraction of a second visible light waveform, acquisition of a second infrared light image, and In addition, after the extraction of the second infrared light waveform, the correlation is further calculated repeatedly. In short-time processing, when the calculated degree of correlation is equal to or greater than a predetermined threshold, a control signal for increasing the amount of infrared light in the infrared light source 123 at a third speed that is twice or more faster than the first speed is red. Output as an external light control signal, and output as a visible light control signal a control signal for reducing the amount of visible light in the illumination device 30 at a fourth speed that is at least twice as fast as the second speed, and obtain a third visible light image , The second visible light waveform extraction, the acquisition of the second infrared light image, and the extraction of the second infrared light waveform.

  FIG. 30 is a diagram illustrating an example when switching control is performed in a shortened completion time. (A) of FIG. 30 is a graph which shows the change of the voltage according to each light quantity in each of the illuminating device 30 and infrared light source 123 which are visible light sources. In FIG. 30A, the horizontal axis indicates time, and the vertical axis indicates voltage according to the amount of light. FIG. 30B shows a visible light waveform and an infrared light waveform when the voltage applied to each light source is changed as shown in FIG. In FIG. 30B, the horizontal axis indicates time, and the vertical axis indicates luminance.

  As shown to (a) of FIG. 30, the light quantity irradiated from the illuminating device 30 is turned off by completion time T / 3 from the start of switching control. At this time, as shown in FIG. 30B, the number of peaks of the visible light waveform is smaller than the number of peaks of the visible light waveform acquired in the switching control in the normal mode with the completion time T. Therefore, in the time reduction mode, the number of data of the feature points of the visible light waveform to be compared in order to acquire the infrared light waveform in the switching control is reduced. For this reason, the accuracy of the switching control is reduced, but the time required for the switching control can be shortened. By performing the switching control in the short mode at this time, the user can quickly switch and go to sleep on a day or the like when he wants to sleep early.

(Biometric information calculation unit)
The biological information calculation unit 116 uses either one of the feature quantities of the visible light waveform acquired by the visible light waveform calculation unit 111 or the infrared light waveform acquired by the infrared light waveform calculation unit 112, and The biometric information is calculated. Specifically, when the illumination device 30 is ON and the visible light waveform calculation unit 111 can acquire a visible light waveform, the biological information calculation unit 116 calculates the first heartbeat interval time from the visible light waveform calculation unit 111. get. Then, the biological information calculation unit 116 calculates biological information such as a heart rate and a stress index using the first heartbeat interval time.

  On the other hand, the biological information calculation unit 116 is a case where the illumination device 30 is OFF or the visible light waveform calculation unit 111 cannot acquire a visible light waveform, and the infrared light waveform calculation unit 112 performs the infrared light waveform. Can be acquired from the infrared light waveform calculation unit 112, the second heartbeat interval time is acquired. Then, the biological information calculation unit 116 similarly calculates biological information such as a heart rate and a stress index using the second heartbeat interval time.

  The biological information calculation unit 116 can extract the feature amount (heartbeat interval time) of each waveform (visible light waveform and infrared light waveform) in both the visible light waveform calculation unit 111 and the infrared light waveform calculation unit 112. If it is, the biological information is calculated using the first heartbeat interval time from the visible light waveform calculation unit 111. This is because visible light is more robust to noise such as body movement and more reliable than infrared light.

  The biological information calculation unit 116 may calculate biological information using the acquired feature amount of the visible light waveform, or calculate biological information using the acquired feature amount of the infrared light waveform. Also good. In addition, the biological information calculation unit 116 may calculate the user's biological information using the feature amount of the second visible light waveform acquired after the second control information is output from the light source control unit 115, or the light source The biometric information of the user may be calculated using the feature amount of the first visible light waveform acquired before the second control information is output from the control unit 115. Similarly, the biological information calculation unit 116 may calculate the user's biological information using the feature amount of the second infrared light waveform acquired after the infrared light control signal is output from the light source control unit 115. Then, the biometric information of the user may be calculated using the feature amount of the first infrared light waveform acquired before the infrared light control signal is output from the light source control unit 115.

  The biological information calculation unit 116 may calculate biological information using the acquired feature amount of the visible light waveform, or calculate biological information using the acquired feature amount of the infrared light waveform. Also good. In addition, the biological information calculation unit 116 may calculate the user's biological information using the feature amount of the second visible light waveform acquired after the second control information is output from the light source control unit 115, or the light source The biometric information of the user may be calculated using the feature amount of the first visible light waveform acquired before the second control information is output from the control unit 115. Similarly, the biological information calculation unit 116 may calculate the user's biological information using the feature amount of the second infrared light waveform acquired after the infrared light control signal is output from the light source control unit 115. Then, the biometric information of the user may be calculated using the feature amount of the first infrared light waveform acquired before the infrared light control signal is output from the light source control unit 115.

  The biometric information to be calculated is a heart rate or a stress index, but is not limited to this. For example, an acceleration pulse wave may be calculated from the obtained pulse wave to calculate an arteriosclerosis index. Alternatively, the pulse wave timing may be accurately acquired from two different user sites, and the blood pressure may be estimated from the time difference (pulse wave propagation time). Alternatively, the sleep depth may be calculated by calculating the superiority of the sympathetic nerve and the parasympathetic nerve from the fluctuation of the heartbeat interval time.

  The biometric information calculation unit 116 may output information indicating “high stress”, “low stress”, or the like as the stress index according to the numerical value of LF / HF.

  In addition, the biological information calculation unit 116 can obtain the sleep depth as shown in Patent Document 3. Specifically, the sleep depth can be determined based on LF, HF, and the presence or absence of body movement. The sleep depth is an index indicating the degree of activity of the subject's brain. For example, it may be determined whether the sleep depth corresponds to non-REM sleep or REM sleep. Further, in non-REM sleep, it may be determined whether it corresponds to shallow sleep or deep sleep.

  In addition, the biometric information calculation part 116 may output the said numerical value as a sleep depth by providing the numerical value according to the step for every step of the determined sleep depth.

  Note that LF (Low Frequency) and HF (Hi Frequency) can be obtained by performing processing as shown in Patent Document 3. That is, the pulse interval data (heart interval time) is converted into a frequency spectrum distribution by, for example, FFT (Fast Fourier Transform). Next, LF and HF are obtained from the obtained frequency spectrum distribution. Specifically, LF and HF are obtained by taking the arithmetic average of the total value of three points including the peak value of a plurality of power spectra and one point at regular intervals around the peak value. As other examples of the FFT method as the frequency analysis method, an AR model, a maximum entropy method, a wavelet method, or the like can be used.

(Presentation device)
The presentation device 40 is a device that presents biological information received from the biological information calculation unit 116. Specifically, the presentation device 40 is a device that presents biological information such as a heart rate, a stress index, and a sleep depth obtained from the biological information calculation unit 116. For example, the presentation device 40 may be realized by the mobile terminal 200, and may display a graphic indicating biological information on the display 204 of the mobile terminal 200, or may output sound indicating biological information from a speaker (not shown) of the mobile terminal 200. May be.

  The presentation device 40 may be realized by the display when the pulse wave measurement device 10 has a built-in display, or may be realized by the speaker when the pulse wave measurement device 10 has a built-in speaker. May be.

  Although the presentation device 40 presents the biological information obtained from the biological information calculation unit 116, the present invention is not limited to this. The presentation device 40 may always present the light amount of the light source in the illumination device 30 and the light amount of the light source in the infrared light source 123, for example. Further, the presentation device 40 may present the degree of coincidence at the current time point from the correlation degree calculation unit 113, for example, in% display as the reliability. Specifically, the presentation device 40 may present a correlation coefficient between the visible light waveform and the infrared light waveform.

  FIG. 31 is a diagram illustrating a display example on the presentation device. As shown in FIG. 31, the presentation device 40 is a graphic showing a heart rate, a stress index, a sleep depth, and a current reliability (that is, a correlation coefficient between heartbeat intervals of a visible light waveform and an infrared light waveform). Is displayed. The presentation device 40 may also display the ratio of the light amounts of the visible light source and the infrared light source at the present time. In addition, the presentation device 40 refers to a table in which the sleep state is associated with the numerical values of the heart rate, the stress index, and the sleep depth based on these parameters as to what the user's sleep state is. The sleep state may be determined and the determined sleep state may be displayed. For example, the presentation device 40 displays “GOOD” when the heart rate is 65 or less, the stress index is 40 or less, and the sleep depth is 70 or more. Note that the presentation device 40 may not display the presentation content such as the biological information immediately after the calculation. That is, since the user basically sleeps, instead of immediately presenting the presentation contents such as biological information obtained by calculation, for example, recording (accumulating) For example, the presentation content may be presented when it gets up the next morning. Thereby, the user can confirm whether the good sleep was taken immediately after waking up.

[1-3. Operation]
Next, the operation of the pulse wave measuring apparatus 10 according to the present embodiment will be described. FIG. 32 is a flowchart showing the flow of processing of the pulse wave measuring apparatus 10 in the present embodiment.

  First, the lighting device 30 is activated when the user enters the room or the user himself / herself controls the control.

  First, the light source control unit 115 acquires a fourth control pattern from the illumination device 30 (S001).

  The light source control unit 115 outputs a visible light control signal to the illumination device 30 based on the acquired fourth control pattern, so that the extracted hue waveform has a predetermined reference value for the color temperature of the visible light of the illumination device 30. The color temperature of the illumination device 30 is adjusted so as to be within a hue range (for example, a range of 0 ° to 60 °) based on (for example, 30 ° in the hue circle) (S002).

  The visible light waveform calculation unit 111 acquires a second visible light image obtained by capturing an image of the user irradiated with visible light by the illumination device 30 in the visible light region (S003).

  The infrared light waveform calculation unit 112 acquires a first infrared light image obtained by imaging a user irradiated with infrared light from the infrared light source 123 in the infrared light region (S004).

  The visible light waveform calculation unit 111 extracts a first visible light waveform that is a waveform indicating the pulse wave of the user from the acquired second visible light image (S005). The visible light waveform calculation unit 111 extracts a plurality of first feature points that are predetermined feature points in the visible light waveform. And the visible light waveform calculating part 111 calculates 1st heartbeat interval time as a feature-value of a visible light waveform. Further, the visible light waveform calculation unit 111 stores the inclination from the top to the bottom of the visible light waveform at this time in the memory as the first inclination A.

  The infrared light waveform calculation unit 112 extracts a first infrared light waveform that is a waveform indicating the pulse wave of the user from the acquired first infrared light image (S006). The infrared light waveform calculation unit 112 extracts a plurality of second feature points that are predetermined feature points in the infrared light waveform. Then, the infrared light waveform calculation unit 112 calculates the second heartbeat interval time as the feature amount of the infrared light waveform.

  Then, the correlation calculation unit 113 determines a peak point (S007). Specifically, the correlation degree calculation unit 113 determines whether or not there is an excessively acquired peak point for the first feature point extracted in the visible light waveform. Further, the correlation calculation unit 113 determines whether or not there is an excessively acquired peak point for the second feature point extracted from the infrared light waveform. Details of the peak point determination processing by the correlation degree calculation unit 113 will be described later.

  Next, the correlation degree calculation unit 113 calculates the degree of correlation between the visible light waveform and the infrared light waveform (S008). The details of the correlation degree calculation processing by the correlation degree calculation unit 113 will be described later.

  Next, the light source control unit 115 adjusts the light amount of each light source (S009). The light source control unit 115 outputs a control signal for controlling the light amount of each light source according to the result of the light amount adjustment. Details of the light amount adjustment processing of the illumination device 30 and the infrared light source 123 by the light source control unit 115 will be described later.

  Next, after each light source is adjusted, the processes in steps S003 to S006 are repeated as steps S0010 to S013.

  Next, the biological information calculation unit 116 calculates biological information from at least one of the feature amount of the visible light waveform and the feature amount of the infrared light waveform (S014).

  Next, the biological information calculation unit 116 outputs the calculated biological information to the presentation device 40 (S015).

  FIG. 33 is a flowchart showing details of peak point excess acquisition determination processing in the present embodiment.

The correlation degree calculation unit 113 calculates the standard deviation SD _RGB of the first heartbeat interval time (S101).

Next, the correlation degree calculation unit 113 determines whether or not the standard deviation SD _RGB is equal to or smaller than a fourth threshold value (S102).

When it is determined that the standard deviation SD _RGB is equal to or smaller than the fourth threshold (Yes in S102), the correlation degree calculation unit 113 calculates the standard deviation SD _IR of the second heartbeat interval time (S103).

Then, the correlation degree calculation unit 113 determines whether or not the standard deviation _SDIR is less than or equal to the fourth threshold (S104).

As described above, the correlation calculation unit 113 performs at least one of step S102 and step S104, so that the calculated standard deviation SD _RGB exceeds the fourth threshold, and the calculated standard deviation _SDIR is A second determination is made to determine whether the fourth threshold is exceeded.

When it is determined that the standard deviation _SDIR is equal to or less than the fourth threshold (Yes in S104), the correlation calculation unit 113 transmits a “False” signal to the light source control unit 115 (S105).

On the other hand, the correlation calculation unit 113 determines that the standard deviation SD _RGB exceeds the fourth threshold (No in S102), or determines that the standard deviation SD _IR exceeds the fourth threshold (S104). No), the absolute error e between the corresponding first heartbeat interval time and the second heartbeat interval time is calculated (S106).

  The correlation calculation unit 113 determines whether or not the absolute error e is smaller than −200 [ms] (S107).

  When determining that the absolute error e is smaller than −200 [ms] (Yes in S107), the correlation degree calculation unit 113 transmits a “False, RGB” signal to the light source control unit 115 (S109).

  On the other hand, when it is determined that the absolute error e is −200 [ms] or more (No in S107), the correlation calculation unit 113 determines whether or not the absolute error e is greater than 200 [ms] (S108). ).

That is, if the correlation degree calculation unit 113 determines that the standard deviation SD _RGB exceeds the fourth threshold and the standard deviation SD _IR exceeds the fourth threshold as a result of the second determination, A third determination of whether or not an absolute error e (time difference) between the first heartbeat interval time and the second heartbeat interval time corresponding to each other in the series is less than a fifth threshold, and a sixth greater than the fifth threshold A fourth determination is made as to whether or not the time difference is larger than the threshold value.

  When it is determined that the absolute error e is greater than 200 [ms] (Yes in S108), the correlation degree calculation unit 113 transmits a “False, IR” signal to the light source control unit 115 (S110).

  When it is determined that the absolute error e is 200 [ms] or less (No in S108), the correlation degree calculation unit 113 transmits a “False, Both” signal to the light source control unit 115 (S111).

  FIG. 34 is a flowchart showing details of correlation degree calculation processing in the present embodiment.

  First, the correlation degree calculation unit 113 calculates the degree of correlation between the plurality of first heart beat interval times and the plurality of second heart beat interval times (S201).

  The correlation degree calculation unit 113 determines whether or not the correlation degree obtained by the calculation is larger than the second threshold (S202). That is, the correlation degree calculation unit 113 performs the first determination that determines whether or not the calculated correlation degree is equal to or greater than the second threshold value.

  When it is determined that the degree of correlation is greater than the second threshold (Yes in S202), the correlation degree calculation unit 113 transmits a “True” signal to the light source control unit 115 (S203).

  On the other hand, when the correlation degree calculation unit 113 determines that the correlation degree is equal to or lower than the second threshold (No in S202), the correlation degree calculation unit 113 transmits a “False” signal to the light source control unit 115 (S204).

  FIG. 35 is a flowchart showing details of light amount adjustment processing in the present embodiment.

  In the light source control unit 115, the signal received from the correlation degree calculation unit 113 is any one of the signals “True”, “False”, “False, IR”, “False, RGB”, and “False, Both”. Is determined (S301).

  When the received signal is a “True” signal, the light source control unit 115 decreases the amount of visible light and increases the amount of infrared light (S302).

  When the received signal is a “False” or “False, IR” signal, the light source control unit 115 increases the amount of infrared light (S303). That is, the light source control unit 115 receives the “False, IR” signal when the correlation degree calculation unit 113 determines that the absolute error e is larger than the sixth threshold value. A control signal for increasing the amount of light is output to the infrared light source 123 as an infrared light control signal.

  When the light source control unit 115 increases the amount of infrared light in step S302 or step S303, whether the second slope of the infrared light waveform is equal to the first slope A stored in the memory or not. Is determined (S304). Further, the light source control unit 115 may determine whether or not the amount of visible light is 0 when the amount of visible light is decreased in step S302.

  If the light source control unit 115 determines that the second inclination is equal to the first inclination (Yes in S304), the light amount adjustment processing ends. Further, if the light source control unit 115 further determines that the amount of visible light is 0, the light amount adjustment processing may be terminated.

When the received signal is “False, RGB”, the light source control unit 115 determines whether or not the standard deviation _SDIR is less than or equal to the fourth threshold value (S305). That is, the light source control unit 115 determines whether or not the standard deviation _SDIR is equal to or smaller than the fourth threshold when the correlation calculation unit 113 determines that the absolute error e is smaller than the fifth threshold. 5. Make a decision.

When the light source control unit 115 determines that the standard deviation _SDIR is equal to or less than the fourth threshold value (Yes in S305), the light source control unit 115 performs the process of step S302. That is, the light source control unit 115 outputs a visible light control signal for reducing the amount of visible light in the illumination device 30 when the correlation calculation unit 113 determines that the standard deviation SD _IR is equal to or less than the fourth threshold. 30 and outputs an infrared light control signal for increasing the amount of infrared light in the infrared light source 123 to the infrared light source 123.

When the received signal is “False, Both” or when the standard deviation _SDIR is determined to be larger than the fourth threshold (No in S305), the light source control unit 115 increases the amount of visible light. The light quantity is returned to the initial light quantity, and the light quantity of the infrared light is decreased to turn off the infrared light source 123 (S306). That is, the light source control unit 115 receives the “False, Both” signal when the correlation calculation unit 113 determines that the absolute error e is greater than or equal to the fifth threshold value and less than or equal to the sixth threshold value. A control signal for increasing the amount of visible light at 30 is output to the illumination device 30 as a visible light control signal, and a control signal for decreasing the amount of infrared light at the infrared light source 123 is red as an infrared light control signal. Output to the external light source 123. _{Alternatively,} when the correlation calculation unit 113 determines that the standard deviation _SDIR is greater than the fourth threshold, the light source control unit 115 uses a control signal for increasing the amount of visible light in the illumination device 30 as a visible light control signal. A control signal that is output to the illumination device 30 and that reduces the amount of infrared light in the infrared light source 123 is output to the infrared light source 123 as an infrared light control signal.

  If the light source control unit 115 determines in step S304 that the second inclination is different from the first inclination A (No in S304) or if step S306 ends, the process returns to step S001. That is, in this case, if the pulse wave measuring device 10 does not satisfy the determination condition in step S304 even if the amount of visible light in the illumination device 30 and the amount of infrared light in the infrared light source 123 are changed, step S001 is performed. Return to, repeat the acquisition of the visible light image, the acquisition of the infrared light image, the extraction of the visible light waveform, the extraction of the infrared light waveform, and the calculation of the correlation degree, depending on the result of the repeated calculation of the correlation degree The infrared light control signal and the visible light control signal are output. That is, acquisition of visible light image, acquisition of infrared light image, extraction of visible light waveform, extraction of infrared light waveform, calculation of correlation degree, output of infrared light control signal, and output of visible light control signal are This is repeated until the determination condition in step S304 is satisfied. Note that the visible light image repeatedly acquired in the second and subsequent processes is the third visible light image, and the infrared light image repeatedly acquired in the second and subsequent processes is the second infrared light image. The visible light waveform repeatedly extracted by the processing is the second visible light waveform, and the infrared light waveform repeatedly extracted by the second and subsequent processing is the second infrared light waveform.

  For example, the second visible light image is a visible light image captured by the visible light imaging unit 122 before the visible light control signal is output, and the third visible light image is after the visible light control signal is output. It is a visible light image imaged by the visible light imaging unit 122. The first infrared light image is an infrared light image captured by the infrared light imaging unit 124 before the infrared light control signal is output, and the second infrared light image is the infrared light control. It is an infrared light image captured by the infrared light imaging unit 124 after the signal is output.

[1-4. Effect etc.]
According to the pulse wave measuring device 10 according to the present embodiment, the lighting device 30 is controlled using a control pattern predetermined for the lighting device 30 in accordance with the control of the amount of infrared light emitted from the infrared light source 123. Controls the amount of light emitted. Therefore, for example, even when a commercially available lighting device is used, adjustment of the amount of visible light and adjustment of the amount of infrared light can be appropriately performed, and biological information can be calculated with high accuracy.

  Further, according to the pulse wave measuring apparatus 10, the second biological information is calculated from at least one of the feature amount of the first visible light waveform and the feature amount of the first infrared light waveform, and the calculated second biological information is output. To do.

  Therefore, the second biological information is calculated from at least one of the feature amount of the first visible light waveform and the feature amount of the first infrared light waveform acquired before the adjustment of the light amount of visible light or infrared light, The calculated second biological information can be output.

  Moreover, according to the pulse wave measuring device 10, when the illumination device 30 is a device that is dimmed by the first control pattern that adjusts the light amount in one step of on / off, the light amount of infrared light emitted from the infrared light source. Is output to the infrared light source 123 as an infrared light control signal, and the control signal for turning off the lighting device 30 is used as the visible light control signal. Output to.

  For this reason, even if the illuminating device 30 is an illuminating device which adjusts the light quantity in one step, it is possible to appropriately adjust the light quantity of visible light and the light quantity of infrared light.

  Moreover, according to the pulse wave measuring device 10, the illumination device 30 uses the second control pattern that adjusts the light amount in two steps, that is, the first visible light amount and the second visible light amount that is smaller than the first visible light amount. When the device is dimmed, control is performed to control the amount of infrared light emitted from the infrared light source 123 to a second infrared light amount that is increased from the first infrared light amount by a predetermined second change amount. A signal is output to the infrared light source 123 as an infrared light control signal, and a control signal for changing the illumination device 30 from the first visible light amount to the second visible light amount is output to the illumination device 30 as a visible light control signal. Then, according to the luminance change of the infrared light obtained from the first and second infrared light images and the luminance change of the visible light obtained from the first and third visible light images, the first in the amount of infrared light. 3 change amounts are determined, and the determined third change amount from the second infrared light quantity A control signal for controlling the increased amount of the third infrared light is output to the infrared light source as an infrared light control signal, and a second-stage control signal for turning off the illumination device 30 is a visible light control signal. To the lighting device 30.

  According to this, when the illuminating device 30 is a device that is dimmed according to the second control pattern, the pulse wave measuring device 10 obtains and acquires the amount of decrease in the luminance of visible light in the first-stage dimming. By increasing the light amount of the infrared light source according to the decrease amount, the infrared light waveform can be acquired more effectively.

  Further, according to the pulse wave measuring device 10, when the lighting device 30 is a device that is dimmed by the third control pattern that adjusts the light quantity in a stepless manner, the calculated correlation degree is equal to or greater than a predetermined threshold value. In the output of the infrared light control signal, the control signal for increasing the amount of infrared light in the infrared light source is output to the infrared light source as the infrared light control signal, and in the output of the visible light control signal, the illumination device A control signal for reducing the amount of visible light at 30 is output to the illuminating device 30 as a visible light control signal, acquisition of a third visible light image, extraction of a second visible light waveform, acquisition of a second infrared light image, and After the second infrared light waveform is extracted, the calculation of the degree of correlation is further repeated, the light quantity of the lighting device is equal to or less than the second threshold value, and the degree of correlation is determined as a result of the repeated calculation of the degree of correlation. If the threshold is exceeded, A control signal to turn off 30, and outputs to the illumination device 30 as a visible-light control signal.

  According to this, compared with the case where it reduces linearly until it turns off visible light, it can be turned off earlier and can introduce into comfortable sleep.

  Further, according to the pulse wave measuring device 10, in the case where the lighting device 30 is a device that is dimmed by the third control pattern that adjusts the amount of light in a stepless manner, (i) the calculated degree of correlation is equal to or greater than a predetermined threshold value In this case, in the output of the infrared light control signal, a control signal for increasing the amount of infrared light in the infrared light source 123 at the first speed is output to the infrared light source as an infrared light control signal, and visible light is emitted. In the output of the control signal, a control signal for reducing the amount of visible light in the lighting device at the second speed is output to the lighting device as a visible light control signal, acquisition of the third visible light image, extraction of the second visible light waveform, After the acquisition of the second infrared light image and the extraction of the second infrared light waveform, a normal process for repeatedly calculating the correlation degree, and (ii) when the calculated correlation degree is equal to or greater than a predetermined threshold value, In infrared light control signal output, the infrared light source A control signal for increasing the amount of infrared light at a third speed that is at least twice as fast as the first speed is output to the infrared light source as an infrared light control signal. A control signal for reducing the amount of light at a fourth speed that is twice or more faster than the second speed is output to the illumination device 30 as a visible light control signal, acquisition of a third visible light image, extraction of a second visible light waveform, After the acquisition of the second infrared light image and the extraction of the second infrared light waveform, any one of the short-time processing for repeatedly calculating the correlation degree is executed.

  For this reason, the time required for switching control can be shortened.

  Further, according to the pulse wave measuring device 10, when the lighting device 30 is a device that is dimmed by the fourth control pattern that adjusts the light amount and the color temperature, the color temperature of the lighting device 30 is set to a predetermined temperature. The control signal to be adjusted is output to the illumination device 30 as a visible light control signal, and the second visible light waveform is extracted using the hue obtained from the third visible light image acquired after the output of the visible light control signal. To do. Further, the pulse wave measuring device 10 is obtained by imaging in a visible light region a user who has been irradiated with visible light having a predetermined color temperature by the lighting device 30 after the output of the visible light control signal. The obtained third visible light image is acquired, a hue waveform that is a waveform indicating the user's pulse wave is extracted from the hue of the acquired third visible light image, and the extracted hue waveform is based on a predetermined reference value. A control signal for adjusting the color temperature of the lighting device so as to fall within the range is output to the lighting device 30 as a visible light control signal.

  According to this, the color of the user's skin surface changes from white to reddish, and in particular, by changing the color temperature of the lighting device so that the value of the hue H is, for example, around 30 degrees, A visible light waveform can be acquired robustly against environmental noise.

  Moreover, according to the pulse wave measuring device 10, the visible light waveform obtained from the visible light image obtained by imaging the user's pulse wave and the infrared light image obtained by imaging the same pulse wave are obtained from the infrared light waveform. The degree of correlation with the infrared light waveform is calculated, and the amount of infrared light emitted from the infrared light source is controlled according to the degree of correlation. For this reason, the amount of infrared light can be adjusted appropriately, and the user's biological information can be acquired even in a dark state such as during sleep. Thereby, the living body monitoring at the time of sleep without contact etc. becomes possible, without providing the living body sensor which contacts a person.

  Further, according to the pulse wave measuring device 10, the correlation degree calculation unit 113 compares the first heartbeat interval time calculated from the visible light waveform and the second heartbeat interval time calculated from the infrared light waveform, Calculate the degree of correlation. For this reason, it is possible to easily calculate the degree of correlation between the visible light waveform and the infrared light waveform.

  Further, according to the pulse wave measuring device 10, in order to compare the second inclination in the infrared light waveform after adjusting the light amount of the infrared light source and the first inclination A stored in the memory, It can be determined whether or not the amount of light from the external light source has reached an appropriate amount.

  Moreover, according to the pulse wave measuring device 10, when the absolute error e exceeds the third threshold value, the first heart beat interval time and the second heart beat interval time determined to exceed the third threshold value. The predetermined feature point that is the reference for calculating the heartbeat interval time in the waveform having the more predetermined feature points is excluded from the calculation target of the heartbeat interval time. For this reason, an excessively acquired peak point can be deleted, and an appropriate value of the first heart beat interval time or the second heart beat interval time can be obtained.

  Moreover, according to the pulse wave measuring device 10, the light amounts in the visible light source and the infrared light source are determined according to the calculated correlation and the extraction result of the predetermined feature points from the visible light waveform and the infrared light waveform. Each is determined to be increased, decreased, or maintained, and a control signal corresponding to the determination result is output to the visible light source and the infrared light source. Thereby, the light quantity of a visible light source and an infrared light source can be adjusted appropriately.

  Further, according to the pulse wave measuring device 10, a predetermined feature point is extracted from the visible light waveform or the infrared light waveform acquired while the light amount of the illumination device 30 or the infrared light source 123 is controlled by the control signal. do not do. For this reason, predetermined feature points can be appropriately extracted, and biological information can be calculated with high accuracy.

  Moreover, according to the pulse wave measuring device 10, in each of the visible light waveform and the infrared light waveform, the lighting device until two or more predetermined feature points continuous from the waveform are extracted within the second predetermined period. It waits for the output of the control signal which controls the light quantity of visible light in 30, or the control signal which controls the light quantity of the infrared light in an infrared light source. For this reason, predetermined feature points can be appropriately extracted, and biological information can be calculated with high accuracy.

[1-5. Modified example]
[1-5-1. Modification 1]
In the embodiment described above, the control pattern acquisition unit 114 stores the control pattern corresponding to the lighting device 30 in the storage 103 of the pulse wave measuring device 10 according to the product number of the lighting device 30 input by the user. Although the control pattern is acquired by selecting from a plurality of control patterns, the present invention is not limited to this. For example, the control pattern acquisition unit 114 may read the control pattern of the illumination device 30 by communicating with the illumination device 30 using infrared rays. Specifically, the pulse wave measuring device 10 transmits a control signal included in a plurality of control patterns by infrared rays or the like, so that the response of the lighting device 30 to the transmitted signal is changed according to a change in the light amount of the lighting device 30. A control pattern corresponding to the illumination device 30 may be specified by making a determination in the control pattern acquisition unit 114. Thereby, even if it does not receive the input of a product number by a user, the control pattern of the illuminating device 30 can be specified automatically.

  Specifically, the pulse wave measuring apparatus 10 may perform the operation shown in FIG.

  FIG. 36 is a flowchart of the control pattern recognition process in the modification.

  In the pulse wave measuring device 10, the light source control unit 115 transmits a predetermined control signal to the illumination device 30 (S401). The light source control unit 115 transmits a plurality of types of control signals to the illumination device 30. For example, the light source control unit 115 transmits a 16-bit signal from “0000” to “1111”.

  Next, the visible light waveform calculation unit 111 acquires a light amount change of the illumination device 30 from the acquired visible light image (S402).

  Then, the light source control unit 115 performs matching processing for selecting an optimal control pattern from a plurality of control patterns stored in advance according to the change in the amount of light acquired by the visible light waveform calculation unit 111 (S403).

  The light source control unit 115 performs matching processing until an optimal control pattern is recognized by the matching processing (S404).

[1-5-2. Modification 2]
In the above embodiment, in the switching control, the user can select whether to place importance on the accuracy of pulse wave acquisition or on the speed until the lighting device 30 is turned off. However, the present invention is not limited to this. . For example, the light source control method may be automatically changed according to the number of times the user uses the device.

  Specifically, when the initial setting or once setting is used about 10 times, accuracy is emphasized, and an accurate pulse wave is generated while carefully switching between the visible light source and the infrared light source. You may make it obtainable.

  On the other hand, since it is conceivable that the environment and conditions hardly change several times after being set once, the light amounts of visible light and infrared light in the light source switching control are stored in advance and stored. By making fine adjustments in the vicinity of the light quantity, the control that places importance on the speed (that is, the switching control in the time reduction mode) may be performed.

  In this way, when necessary at the minimum, by accurately comparing the pulse waves with emphasis on accuracy, it is possible to accurately perform biological sensing without disturbing the user's sleep.

  As described above, the present disclosure can acquire control means for an external lighting device and perform switching with an accompanying infrared light source. Biological sensing during sleep can be performed.

[1-5-3. Modification 3]
Although not particularly mentioned in the above embodiment, the lighting device 30 may be controlled so that the light amount becomes an initial value set in advance when starting from the light amount 0. Thereby, when there is an illuminance that the user likes or an illuminance in which a pulse wave can be easily acquired by the user, the state can be immediately set.

[1-5-4. Modification 4]
The visible light waveform calculation unit 111 records the light amount of the lighting device 30 when the visible light waveform can be acquired and the inclination between the vertex and the bottom of the visible light waveform is the largest, and the user can enter the room. You may control so that the light quantity of the illuminating device 30 may become the recorded value whenever it enters.

[1-5-5. Modification 5]
Although not specifically mentioned in the above embodiment, if the infrared light is continuously irradiated to the human eye, the visual acuity may be lowered. For this reason, the infrared light source 123 may irradiate infrared light by limiting the ROI to a region of the user's face excluding human eyes. For example, when the infrared light source 123 irradiates the user's face with light, the pulse wave is particularly easily acquired on the cheek. For this reason, the light source control unit 115 may specify, for example, a portion under the user's eyes and irradiate the infrared light source 123 with infrared light toward the portion. For example, the light source control unit 115 recognizes the user's face by analyzing the image captured by the infrared light imaging unit 124, and specifies the lower part of the user's eyes using the result of the face recognition. In addition, the light source control unit 115 sets the light amount of the infrared light to a predetermined value when the power of the infrared light emitted from the infrared light source 123 is equal to or greater than a predetermined threshold and has passed for a predetermined time. You may adjust the light quantity of the infrared light source 123 so that it may be suppressed to less than light quantity. In addition, as described above, in the case of infrared light, the user's visual acuity is affected. You can squeeze it.

[1-5-6. Modification 6]
Moreover, in the said embodiment, although the pulse-wave calculating apparatus 100 in the pulse-wave measuring apparatus 10 is incorporated in the pulse-wave measuring apparatus 10, it is not restricted to this. For example, the pulse wave calculation device 100 may be realized as an external server device, may be realized by the mobile terminal 200, or may be realized by an information terminal such as a PC. That is, in this case, if the image captured from the visible light imaging unit 122 and the infrared light imaging unit 124 can be acquired and the light amount of the illumination device 30 and the infrared light source 123 can be controlled, the pulse wave calculation is performed. The device 100 may be realized by any device.

[1-5-7. Modification 7]
Each component included in the pulse wave measuring device or the like may be a circuit. These circuits may constitute one circuit as a whole, or may be separate circuits. Each of these circuits may be a general-purpose circuit or a dedicated circuit. That is, in each of the above embodiments, each component may be configured by dedicated hardware, or may be realized by executing a software program suitable for each component.

  Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software that realizes the display control method of each of the above embodiments is the following program.

  That is, this program is a pulse wave measurement method executed by a pulse wave measurement device including a processor and a memory in a computer, and defines a first correspondence relationship from an illumination device provided outside the pulse wave measurement device. The first correspondence relationship indicates the color temperature of visible light output from the illumination device corresponding to each of a plurality of instructions, and the first color temperature held by the pulse wave measurement device is obtained. A first instruction corresponding to the information to be indicated is determined with reference to the first control pattern, the first instruction is output to the lighting device, and the lighting device has a visible color temperature corresponding to the first instruction. The user irradiated with light is imaged in a visible light region to obtain a plurality of first visible light images, a first plurality of hues are calculated from the plurality of first visible light images, and the first plurality of the plurality of first visible light images are calculated. First from hue When a phase waveform is extracted and the amplitude of the first hue waveform does not belong to a predetermined hue range, a second instruction corresponding to a second color temperature different from the first color temperature is referred to the first control pattern. The second instruction is output to the lighting device, and the user irradiated with visible light having a color temperature corresponding to the second instruction by the lighting device is imaged in the visible light region, and a plurality of images are captured. A second visible light image is acquired, a second plurality of hues are calculated from the plurality of second visible light images, a second hue waveform is extracted from the second plurality of hues, and the second hue waveform is calculated. When the amplitude belongs to the predetermined hue range, a first process is performed, and the first process is a plurality of images obtained by imaging the user irradiated with infrared light from an infrared light source in an infrared light region. The first infrared light image of A first visible light waveform that is a waveform indicating the pulse wave of the user is extracted from a second visible light image, and a first waveform that indicates the pulse wave of the user is obtained from the acquired first infrared light images. 1 infrared light waveform is extracted, the degree of correlation between the extracted first visible light waveform and the extracted first infrared light waveform is calculated, and the infrared light source according to the degree of correlation An infrared light control signal for controlling the amount of infrared light emitted from the illumination device is output to the infrared light source, and the visible light control signal for controlling the amount of visible light emitted by the illumination device is controlled according to the degree of correlation. A plurality of third visible light images obtained by imaging in a visible light region a user irradiated with visible light based on the visible light control signal by the lighting device and acquired by the lighting device are obtained, and infrared light is obtained. The user is irradiated with infrared light based on the infrared light control signal by a light source. A plurality of second infrared light images obtained by imaging the image in the infrared light region, and a second waveform indicating the pulse wave of the user from the obtained third visible light images. A visible light waveform is extracted, and a second infrared light waveform, which is a waveform indicating the pulse wave of the user, is extracted from the acquired second infrared light images, and a feature amount of the second visible light waveform and A pulse wave measurement method including calculating first biological information from at least one of the feature quantities of the second infrared light waveform and outputting the calculated first biological information is executed.

  As mentioned above, although the pulse wave measuring device etc. which concern on the one or some aspect were demonstrated based on embodiment, this indication is not limited to this embodiment. Unless it deviates from the gist of the present disclosure, various modifications conceived by those skilled in the art have been made in this embodiment, and forms constructed by combining components in different embodiments are also within the scope of one or more aspects. May be included.

  For example, in the above-described embodiment, a process executed by a specific component may be executed by another component instead of the specific component. Further, the order of the plurality of processes may be changed, and the plurality of processes may be executed in parallel.

  The present disclosure is useful as a pulse wave measuring device that can accurately calculate biological information.

DESCRIPTION OF SYMBOLS 1 Pulse wave measurement system 10 Pulse wave measuring device 20 Case 22 Visible light camera 23 Infrared light LED
24 Infrared camera 30 Illumination device 31 Visible light LED
32 controller 40 presentation device 100 pulse wave calculation device 101 CPU
102 Main memory 103 Storage 104 Communication IF
111 Visible Light Waveform Calculation Unit 112 Infrared Light Waveform Calculation Unit 113 Correlation Level Calculation Unit 114 Control Pattern Acquisition Unit 115 Light Source Control Unit 116 Biometric Information Calculation Unit 121 Visible Light Source 122 Visible Light Imaging Unit 123 Infrared Light Source 124 Infrared Light Imaging unit 200 Portable terminal 201 CPU
202 Main memory 203 Storage 204 Display 205 Communication IF
206 Input IF

Claims (12)

A pulse wave measuring device comprising a processor,
The processor is
A first control pattern that defines a first correspondence relationship is obtained from an illumination device provided outside the pulse wave measurement device, and the first correspondence relationship is output by the illumination device corresponding to each of a plurality of instructions. A first instruction indicating a color temperature of visible light and corresponding to information indicating a first color temperature held by the pulse wave measuring device is determined with reference to the first control pattern, and the first instruction is determined by the illumination. Outputting to a device, and capturing a plurality of first visible light images by imaging a user irradiated with visible light having a color temperature corresponding to the first instruction by the lighting device in a visible light region,
Calculating a first plurality of hues from the plurality of first visible light images, extracting a first hue waveform from the first plurality of hues;
If the amplitude of the first hue waveform does not belong to a predetermined hue range, a second instruction corresponding to a second color temperature different from the first color temperature is determined with reference to the first control pattern, and the first 2 instructions are output to the lighting device,
Imaging the user irradiated with visible light having a color temperature corresponding to the second instruction by the lighting device in the visible light region to obtain a plurality of second visible light images;
A second plurality of hues are calculated from the plurality of second visible light images, a second hue waveform is extracted from the second plurality of hues, and an amplitude of the second hue waveform belongs to the predetermined hue range If the first process,
The first process includes
Obtaining a plurality of first infrared light images obtained by imaging the user irradiated with infrared light by an infrared light source in an infrared light region;
Extracting a first visible light waveform, which is a waveform indicating the pulse wave of the user, from the acquired second visible light images;
Extracting a first infrared light waveform, which is a waveform indicating the pulse wave of the user, from the plurality of first infrared light images acquired;
Calculating the degree of correlation between the extracted first visible light waveform and the extracted first infrared light waveform;
According to the degree of correlation, an infrared light control signal for controlling the amount of infrared light emitted from the infrared light source is output to the infrared light source,
According to the degree of correlation, the visible light control signal for controlling the amount of visible light emitted by the lighting device is output to the lighting device,
Obtaining a plurality of third visible light images obtained by imaging in a visible light region a user irradiated with visible light based on the visible light control signal by the illumination device;
Acquiring a plurality of second infrared light images obtained by imaging the user irradiated with infrared light based on the infrared light control signal from an infrared light source in an infrared light region;
Extracting a second visible light waveform, which is a waveform indicating the pulse wave of the user, from the acquired plurality of third visible light images;
Extracting a second infrared light waveform, which is a waveform indicating the pulse wave of the user, from the plurality of second infrared light images obtained;
First biological information is calculated from at least one of the feature quantity of the second visible light waveform and the feature quantity of the second infrared light waveform,
Outputting the calculated first biological information;
Pulse wave measuring device. The pulse wave measuring device according to claim 1, wherein the predetermined hue range is a range in which the hue is not less than 0 degrees and not more than 60 degrees. The pulse wave measurement device according to claim 2, wherein the predetermined hue range is a hue range based on a hue of 30 degrees. The processor, in the calculation of the correlation degree,
(1) A plurality of first peak points in a plurality of first unit times included in the plurality of first unit waveforms are extracted, and the plurality of first peak points are included in the plurality of first unit waveforms. A first maximum value point or a plurality of first minimum value points included in the plurality of first unit waveforms, wherein the first visible light waveform includes the plurality of first unit waveforms; And the plurality of first unit waveforms respectively correspond to the plurality of first minimum value points and the plurality of first unit waveforms, respectively, and the plurality of first unit waveforms and the plurality of first unit waveforms. Each of the first unit times corresponds to
(2) A plurality of second peak points in a plurality of second unit times included in the plurality of second unit waveforms are extracted, and the plurality of second peak points are included in the plurality of second unit waveforms. A second maximum value point or a plurality of second minimum value points included in the plurality of second unit waveforms, wherein the first infrared light waveform includes the plurality of second unit waveforms; 2 maximum value points and the plurality of second unit waveforms respectively correspond to each other, the plurality of second minimum value points and the plurality of second unit waveforms correspond to each other, and the plurality of second unit waveforms and The plurality of second unit times correspond to each other,
(3) A plurality of first heart beat interval times are calculated based on the plurality of first unit times, and the plurality of first heart beat interval times are times between a first time and a second time, The first unit time includes the first time and the second time, and the time included in the plurality of first unit times does not exist between the first time and the second time,
(4) A plurality of second heart beat interval times are calculated based on the plurality of second unit times, and the plurality of second heart beat interval times are times between a third time and a fourth time, The second unit time includes the third time and the fourth time, and the time included in the plurality of second unit times does not exist between the third time and the fourth time,
The pulse wave measuring apparatus according to any one of claims 1 to 3, wherein the correlation is calculated using the following (Equation 1).
The processor is
Calculating second biological information from at least one of the feature quantity of the first visible light waveform and the feature quantity of the first infrared light waveform;
The pulse wave measuring device according to any one of claims 1 to 4, wherein the calculated second biological information is output. The processor is
In the case where the illumination device is a device that is further dimmed by a second control pattern that adjusts the amount of light in an on / off stage
In the output of the infrared light control signal, a control signal that increases the amount of infrared light emitted from the infrared light source by a predetermined first change amount is used as the infrared light control signal. Output to
The pulse wave measuring device according to any one of claims 1 to 5, wherein, in the output of the visible light control signal, a control signal for turning off the lighting device is output to the lighting device as the visible light control signal. The processor is
When the illumination device is a device that is further dimmed by a third control pattern that adjusts the amount of light in two stages, a first visible light amount and a second visible light amount that is smaller than the first visible light amount. ,
In the output of the infrared light control signal, the amount of infrared light emitted from the infrared light source is controlled to a second infrared light amount that is increased from the first infrared light amount by a predetermined second change amount. A control signal is output to the infrared light source as the infrared light control signal,
A control signal for changing the illumination device from the first visible light amount to the second visible light amount is output to the illumination device as the visible light control signal,
In accordance with the luminance change of the infrared light obtained from the first and second infrared light images and the luminance change of the visible light obtained from the second and third visible light images, in the amount of the infrared light Determine the third change,
From the second infrared light amount, a control signal for controlling the third infrared light amount increased by the determined third change amount is output to the infrared light source as the infrared light control signal,
The pulse wave measuring device according to any one of claims 1 to 6, wherein a second-stage control signal for turning off the lighting device is output to the lighting device as the visible light control signal. The processor is
In the case where the illumination device is a device that is further dimmed by a fourth control pattern that adjusts the amount of light in a stepless manner,
When the calculated degree of correlation is a predetermined threshold or more,
In the output of the infrared light control signal, a control signal for increasing the amount of infrared light in the infrared light source is output to the infrared light source as the infrared light control signal,
In the output of the visible light control signal, a control signal for reducing the amount of visible light in the illumination device is output to the illumination device as the visible light control signal,
After the acquisition of the third visible light image, the extraction of the second visible light waveform, the acquisition of the second infrared light image, and the extraction of the second infrared light waveform, the calculation of the correlation degree is repeated. Done
Control that turns off the lighting device when the light intensity of the lighting device is equal to or smaller than a second threshold value and the correlation level is equal to or larger than the predetermined threshold value as a result of repeated calculation of the correlation level. A pulse wave measuring device given in any 1 paragraph of Claims 1-6 which output a signal to said illuminating device as said visible light control signal. The processor is
In the case where the illumination device is a device that is dimmed by a fourth control pattern that adjusts the light amount in a stepless manner,
(I) When the calculated degree of correlation is a predetermined threshold value or more,
In the output of the infrared light control signal, a control signal for increasing the amount of infrared light in the infrared light source at a first speed is output to the infrared light source as the infrared light control signal,
In the output of the visible light control signal, a control signal for reducing the amount of visible light in the lighting device at a second speed is output to the lighting device as the visible light control signal,
After the acquisition of the third visible light image, the extraction of the second visible light waveform, the acquisition of the second infrared light image, and the extraction of the second infrared light waveform, the calculation of the correlation degree is repeated. Normal processing to do,
(Ii) When the calculated degree of correlation is a predetermined threshold value or more,
In the output of the infrared light control signal, a control signal for increasing the amount of infrared light in the infrared light source at a third speed that is at least twice as fast as the first speed is used as the red light control signal. Output to an external light source,
In the output of the visible light control signal, a control signal for reducing the amount of visible light in the lighting device at a fourth speed that is twice or more faster than the second speed is output to the lighting device as the visible light control signal,
After the acquisition of the third visible light image, the extraction of the second visible light waveform, the acquisition of the second infrared light image, and the extraction of the second infrared light waveform, the calculation of the correlation degree is repeated. The pulse wave measuring device according to any one of claims 1 to 6, wherein any one of the short-time processing to be performed is executed. A pulse wave measuring method of a pulse wave measuring device,
A first control pattern that defines a first correspondence relationship is obtained from an illumination device provided outside the pulse wave measurement device, and the first correspondence relationship is output by the illumination device corresponding to each of a plurality of instructions. Indicates the color temperature of visible light,
Determining a first instruction corresponding to information indicating the first color temperature held by the pulse wave measuring device with reference to the first control pattern;
Outputting the first instruction to the lighting device;
A user who has been irradiated with visible light having a color temperature corresponding to the first instruction by the lighting device is imaged in a visible light region to obtain a plurality of first visible light images,
Calculating a first plurality of hues from the plurality of first visible light images, extracting a first hue waveform from the first plurality of hues;
If the amplitude of the first hue waveform does not belong to a predetermined hue range, a second instruction corresponding to a second color temperature different from the first color temperature is determined with reference to the first control pattern, and the first 2 instructions are output to the lighting device,
Imaging the user irradiated with visible light having a color temperature corresponding to the second instruction by the lighting device in the visible light region to obtain a plurality of second visible light images;
A second plurality of hues are calculated from the plurality of second visible light images, a second hue waveform is extracted from the second plurality of hues, and an amplitude of the second hue waveform belongs to the predetermined hue range If the first process,
The first process includes
Obtaining a plurality of first infrared light images obtained by imaging the user irradiated with infrared light by an infrared light source in an infrared light region;
Extracting a first visible light waveform, which is a waveform indicating the pulse wave of the user, from the acquired second visible light images;
Extracting a first infrared light waveform, which is a waveform indicating the pulse wave of the user, from the plurality of first infrared light images acquired;
Calculating the degree of correlation between the extracted first visible light waveform and the extracted first infrared light waveform;
According to the degree of correlation, an infrared light control signal for controlling the amount of infrared light emitted from the infrared light source is output to the infrared light source,
According to the degree of correlation, the visible light control signal for controlling the amount of visible light emitted by the lighting device is output to the lighting device,
Obtaining a plurality of third visible light images obtained by imaging in a visible light region a user irradiated with visible light based on the visible light control signal by the illumination device;
Acquiring a plurality of second infrared light images obtained by imaging the user irradiated with infrared light based on the infrared light control signal from an infrared light source in an infrared light region;
Extracting a second visible light waveform, which is a waveform indicating the pulse wave of the user, from the acquired plurality of third visible light images;
Extracting a second infrared light waveform, which is a waveform indicating the pulse wave of the user, from the plurality of second infrared light images obtained;
First biological information is calculated from at least one of the feature amount of the second visible light waveform and the feature amount of the second infrared light waveform,
A pulse wave measuring method including outputting the calculated first biological information. A pulse wave measuring method of a pulse wave measuring device,
A first control pattern that defines a first correspondence relationship is obtained from an illumination device provided outside the pulse wave measurement device, and the first correspondence relationship is output by the illumination device corresponding to each of a plurality of instructions. Indicates the color temperature of visible light,
Determining a first instruction corresponding to information indicating the first color temperature held by the pulse wave measuring device with reference to the first control pattern;
Outputting the first instruction to the lighting device;
A user who has been irradiated with visible light having a color temperature corresponding to the first instruction by the lighting device is imaged in a visible light region to obtain a plurality of first visible light images,
Calculating a first plurality of hues from the plurality of first visible light images, extracting a first hue waveform from the first plurality of hues;
If the amplitude of the first hue waveform does not belong to a predetermined hue range, a second instruction corresponding to a second color temperature different from the first color temperature is determined with reference to the first control pattern, and the first 2 instructions are output to the lighting device,
Imaging the user irradiated with visible light having a color temperature corresponding to the second instruction by the lighting device in the visible light region to obtain a plurality of second visible light images;
A second plurality of hues are calculated from the plurality of second visible light images, a second hue waveform is extracted from the second plurality of hues, and an amplitude of the second hue waveform belongs to the predetermined hue range If the first process,
The first process includes
Obtaining a plurality of first infrared light images obtained by imaging the user irradiated with infrared light by an infrared light source in an infrared light region;
Extracting a first visible light waveform, which is a waveform indicating the pulse wave of the user, from the acquired second visible light images;
Extracting a first infrared light waveform, which is a waveform indicating the pulse wave of the user, from the plurality of first infrared light images acquired;
Calculating the degree of correlation between the extracted first visible light waveform and the extracted first infrared light waveform;
According to the degree of correlation, an infrared light control signal for controlling the amount of infrared light emitted from the infrared light source is output to the infrared light source,
According to the degree of correlation, the visible light control signal for controlling the amount of visible light emitted by the lighting device is output to the lighting device,
Obtaining a plurality of third visible light images obtained by imaging in a visible light region a user irradiated with visible light based on the visible light control signal by the illumination device;
Acquiring a plurality of second infrared light images obtained by imaging the user irradiated with infrared light based on the infrared light control signal from an infrared light source in an infrared light region;
Extracting a second visible light waveform, which is a waveform indicating the pulse wave of the user, from the acquired plurality of third visible light images;
Extracting a second infrared light waveform, which is a waveform indicating the pulse wave of the user, from the plurality of second infrared light images obtained;
First biological information is calculated from at least one of the feature quantity of the second visible light waveform and the feature quantity of the second infrared light waveform,
A program for causing a computer to execute a pulse wave measurement method including outputting the calculated first biological information. A pulse wave measuring method of a pulse wave measuring device,
A first control pattern that defines a first correspondence relationship is obtained from an illumination device provided outside the pulse wave measurement device, and the first correspondence relationship is output by the illumination device corresponding to each of a plurality of instructions. Indicates the color temperature of visible light,
Determining a first instruction corresponding to information indicating the first color temperature held by the pulse wave measuring device with reference to the first control pattern;
Outputting the first instruction to the lighting device;
A user who has been irradiated with visible light having a color temperature corresponding to the first instruction by the lighting device is imaged in a visible light region to obtain a plurality of first visible light images,
Calculating a first plurality of hues from the plurality of first visible light images, extracting a first hue waveform from the first plurality of hues;
If the amplitude of the first hue waveform does not belong to a predetermined hue range, a second instruction corresponding to a second color temperature different from the first color temperature is determined with reference to the first control pattern, and the first 2 instructions are output to the lighting device,
Imaging the user irradiated with visible light having a color temperature corresponding to the second instruction by the lighting device in the visible light region to obtain a plurality of second visible light images;
A second plurality of hues are calculated from the plurality of second visible light images, a second hue waveform is extracted from the second plurality of hues, and an amplitude of the second hue waveform belongs to the predetermined hue range If the first process,
The first process includes
Obtaining a plurality of first infrared light images obtained by imaging the user irradiated with infrared light by an infrared light source in an infrared light region;
Extracting a first visible light waveform, which is a waveform indicating the pulse wave of the user, from the acquired second visible light images;
Extracting a first infrared light waveform, which is a waveform indicating the pulse wave of the user, from the plurality of first infrared light images acquired;
Calculating the degree of correlation between the extracted first visible light waveform and the extracted first infrared light waveform;
According to the degree of correlation, an infrared light control signal for controlling the amount of infrared light emitted from the infrared light source is output to the infrared light source,
According to the degree of correlation, the visible light control signal for controlling the amount of visible light emitted by the lighting device is output to the lighting device,
Obtaining a plurality of third visible light images obtained by imaging in a visible light region a user irradiated with visible light based on the visible light control signal by the illumination device;
Acquiring a plurality of second infrared light images obtained by imaging the user irradiated with infrared light based on the infrared light control signal from an infrared light source in an infrared light region;
Extracting a second visible light waveform, which is a waveform indicating the pulse wave of the user, from the acquired plurality of third visible light images;
Extracting a second infrared light waveform, which is a waveform indicating the pulse wave of the user, from the plurality of second infrared light images obtained;
First biological information is calculated from at least one of the feature amount of the second visible light waveform and the feature amount of the second infrared light waveform,
A computer-readable non-transitory recording medium storing a program for causing a computer to execute a pulse wave measurement method including outputting the calculated first biological information.