JPWO2020070808A1 - Pulse wave calculation device, pulse wave calculation method and pulse wave calculation program - Google Patents

Abstract

Continuously obtain pulse waves suitable for analysis. The pulse wave calculation device (10) extracts a pulse wave waveform corresponding to each of the plurality of regions from the time-series data in the plurality of regions of the body acquired by the sensor (11). Further, the pulse wave calculation device (10) calculates each autocorrelation coefficient of the pulse wave waveform for each time series section. Further, the pulse wave calculation device (10) calculates the difference between each of the pulse wave waveforms and the reference waveform. Further, the pulse wave calculation device 10 selects one of the pulse wave waveforms whose autocorrelation coefficient is equal to or higher than the threshold value from the pulse wave waveforms for each section. Further, the pulse wave calculation device (10) corrects the selected pulse wave waveform by using the difference.

Description

The present invention relates to a pulse wave calculation device, a pulse wave calculation method, and a pulse wave calculation program.

Conventionally, there is known a technique for acquiring a pulse wave used for analysis from a signal related to the body obtained by a predetermined sensor. Here, the pulse wave is a change in volume or the like that occurs in a blood vessel with the beating of the heart. In addition, information such as pulse interval, waveform, and propagation velocity is derived from the pulse wave. Furthermore, by analyzing the information derived from the pulse wave, the stress index, blood vessel age, blood pressure, and the like can be grasped.

In addition, the correlation coefficient between the waveform of the signal in the predetermined target section and the waveform of the signal in the other section is calculated, and the signal in the target section is determined by whether or not the magnitude of the correlation coefficient is a certain value or more. A method for determining whether or not it is suitable for pulse wave analysis is known. Further, there is known a method of calculating a pulse wave based on a change in a luminance signal in a moving image of a specific part of the body taken by a camera.

Japanese Unexamined Patent Publication No. 2014-073159 Japanese Unexamined Patent Publication No. 2014-176584 JP-A-2017-000612

However, in the conventional technique, it may be difficult to continuously obtain a pulse wave suitable for analysis due to noise mixed in the signal and a missing section in which it is difficult to calculate the pulse wave.

For example, if there is no waveform in any of the other sections whose correlation coefficient with the signal waveform of the target section is equal to or higher than a certain value, the target section is a missing section. In addition, the background or shadow reflected in the moving image of a specific portion captured by the camera may generate noise when calculating the pulse wave.

In one aspect, it is an object of the present invention to provide a pulse wave calculation device, a pulse wave calculation method, and a pulse wave calculation program capable of continuously obtaining pulse waves suitable for analysis.

In one embodiment, the pulse wave calculation device extracts the pulse wave waveform corresponding to each of the plurality of regions from the time series data in the plurality of regions of the body acquired by the sensor. The pulse wave calculation device calculates each autocorrelation coefficient of the pulse wave waveform for each time series section. The pulse wave calculation device calculates the difference between each pulse wave waveform and the reference waveform. The pulse wave calculation device selects one of the pulse wave waveforms whose autocorrelation coefficient is equal to or higher than the threshold value from the pulse wave waveforms for each section. The pulse wave calculation device corrects the pulse wave waveform selected by the selection unit using the difference.

According to one aspect, pulse waves suitable for analysis can be continuously obtained.

FIG. 1 is a diagram showing an example of a usage scene of the pulse wave calculation device according to the first embodiment. FIG. 2 is a block diagram showing an example of the functional configuration of the pulse wave calculation device according to the first embodiment. FIG. 3 is a diagram showing an example of a data structure of difference information according to the first embodiment. FIG. 4 is a diagram showing an example of the overall flow of the pulse wave calculation process according to the first embodiment. FIG. 5 is a diagram showing an example of a pulse wave waveform extraction process and a section setting process according to the first embodiment. FIG. 6 is a diagram showing an example of linear interpolation processing. FIG. 7 is a diagram showing an example of statistical processing for each wavelength component. FIG. 8 is a diagram for explaining the zero cross point. FIG. 9 is a diagram showing an example of the correlation calculation process according to the first embodiment. FIG. 10 is a diagram showing an example of the difference calculation process according to the first embodiment. FIG. 11 is a diagram showing an example of a pulse wave waveform selection process and a correction process according to the first embodiment. FIG. 12 is a diagram showing an example of waveform features. FIG. 13 is a flowchart showing an example of the pulse wave calculation process according to the first embodiment. FIG. 14 is a flowchart showing an example of the acquisition process of the pulse wave waveform extraction data according to the first embodiment. FIG. 15 is a flowchart showing an example of the pulse wave waveform extraction process according to the first embodiment. FIG. 16 is a flowchart showing an example of the section setting process according to the first embodiment. FIG. 17 is a flowchart showing an example of the correlation calculation process according to the first embodiment. FIG. 18 is a flowchart showing an example of the difference calculation process according to the first embodiment. FIG. 19 is a flowchart showing an example of the pulse wave waveform selection process and the correction process according to the first embodiment. FIG. 20 is a diagram showing an example of a usage scene of the pulse wave calculation device according to another embodiment. FIG. 21 is a diagram showing an example of a computer that executes a pulse wave calculation program.

Hereinafter, examples of the pulse wave calculation device, the pulse wave calculation method, and the pulse wave calculation program disclosed in the present application will be described in detail with reference to the drawings. The present invention is not limited to this embodiment. In addition, the examples shown below may be appropriately combined as long as they do not cause a contradiction.

[Overall configuration and usage scene]
The pulse wave calculation device 10 according to the first embodiment can calculate a pulse wave with less noise based on the data acquired from a predetermined sensor and provide the calculated pulse wave to the user. Specifically, the pulse wave calculation device 10 calculates the pulse wave waveform or the characteristics of the pulse wave waveform. Further, for example, the pulse wave calculation device 10 acquires data from a photoelectric pulse wave sensor or a camera.

The photoelectric pulse wave sensor is attached to a user's finger or the like and acquires time-series fluctuations of reflected light such as infrared rays radiated to peripheral blood vessels as a signal. The pulse wave calculation device 10 can calculate the pulse wave based on the signal acquired from the photoelectric pulse wave sensor.

In addition, the camera captures a moving image of a specific part of the user's body and acquires moving image data. Here, the moving image data acquired by the camera is the luminance signal of each pixel for each frame in the time series. Therefore, the pulse wave calculation device 10 can calculate the pulse wave based on the change in the luminance signal.

Here, unlike the conventional method, the pulse wave calculation device 10 of the present embodiment acquires time-series data in a plurality of regions of the body from the sensor. For example, the pulse wave calculation device 10 can acquire a signal from each of the photoelectric pulse wave sensors mounted on a plurality of different parts of the body. Further, for example, the pulse wave calculation device 10 can specify a region corresponding to a plurality of parts of the body from an image taken by a camera, and can acquire a luminance signal from each of the specified regions. Further, the pulse wave calculation device 10 may acquire the luminance signal from each of the images of different parts of the body taken by a plurality of cameras.

A usage scene of the pulse wave calculation device 10 of this embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an example of a usage scene of the pulse wave calculation device according to the first embodiment. In the example of FIG. 1, the pulse wave calculation device 10 is a smartphone. Further, in the example of FIG. 1, the camera provided in the smartphone functions as the sensor 11. Further, in the example of FIG. 1, the touch panel display provided in the smartphone functions as the output unit 14.

The user photographs the measurement target (for example, himself / herself) with the sensor 11 of the pulse wave calculation device 10. Then, the pulse wave calculation device 10 calculates the pulse wave based on the captured image, and outputs the calculated pulse wave and the information about the pulse wave to the output unit 14 as an image. In the example of FIG. 1, the pulse wave calculation device 10 outputs the heart rate and the pulse wave waveform. Further, the pulse wave calculation device 10 outputs an image in which a marker indicating a region specified for pulse wave calculation is superimposed on the image of the measurement target taken by the user to the output unit 14. In the example of FIG. 1, the pulse wave calculation device 10 specifies the area between the eyebrows, the neck, and the palm of the user to be measured. In addition to this, the pulse wave calculation device 10 may specify areas such as the earlobe, cheeks, and forehead of the user to be measured.

[Functional configuration]
The functional configuration of the pulse wave calculation device 10 will be described with reference to FIG. FIG. 2 is a block diagram showing an example of the functional configuration of the pulse wave calculation device according to the first embodiment. As shown in FIG. 2, the pulse wave calculation device 10 includes a sensor 11, a storage unit 12, a control unit 13, and an output unit 14.

The sensor 11 acquires a signal regarding the user's body. For example, the sensor 11 is a photoelectric pulse wave sensor or a camera capable of capturing a moving image. Further, the pulse wave calculation device 10 may include a plurality of sensors 11. In this embodiment, the sensor 11 is built-in, but it may be externally attached or only the measurement result is input.

The storage unit 12 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. The storage unit 12 has the difference information 121. Further, the storage unit 12 stores information used for processing by the control unit 13.

The difference information 121 is information indicating the difference between the pulse wave waveforms between the parts of the body. The pulse wave waveform between body parts can be obtained in an extraction process described later. FIG. 3 is a diagram showing an example of a data structure of difference information according to the first embodiment. As shown in FIG. 3, the difference information 121 includes the phase difference and the amplitude ratio of each part when a predetermined part is used as a reference. For example, FIG. 3 shows that the phase difference of the pulse wave waveform of the site B with respect to the pulse wave waveform of the site A is -11 msec, and the amplitude ratio is 0.91. The difference in phase difference and amplitude ratio is caused by the difference in the distance from the heart of each part of the body and the strength of the signal that can be acquired.

Returning to the description of FIG. 2, in the control unit 13, for example, a program stored in an internal storage device is executed by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like using the RAM as a work area. It is realized by. Further, the control unit 13 may be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 13 includes an acquisition unit 131, an extraction unit 132, a setting unit 133, a correlation calculation unit 134, a difference calculation unit 135, a selection unit 136, a correction unit 137, and an output control unit 138. Realize or execute the functions and actions of. The internal configuration of the control unit 13 is not limited to the configuration shown in FIG. 1, and may be another configuration as long as it is a configuration for performing information processing described later.

Here, with reference to FIG. 4, the overall flow of processing of the pulse wave calculation device 10 and the details of processing of each unit included in the control unit 13 will be described. FIG. 4 is a diagram showing an example of the overall flow of the pulse wave calculation process according to the first embodiment. The arrows in FIG. 4 represent the flow of data.

The acquisition unit 131 is a signal acquired by the sensor 11 and acquires time-series signal data of each part of the body. The acquisition unit 131 may acquire the signals of each part as individual data or may acquire the signals as one data. For example, the acquisition unit 131 may acquire a plurality of different moving image data obtained by photographing each part of the body, or may acquire one moving image data obtained by photographing an area including each part of the body. Further, the acquisition unit 131 may collectively acquire signals for a certain period of time, or may continuously acquire signals in real time.

When the sensor 11 is a photoelectric pulse wave sensor, the acquisition unit 131 acquires a signal indicating a pulse wave acquired at a predetermined sampling rate (for example, 128 Hz (hertz)). On the other hand, when the sensor 11 is a camera, the acquisition unit 131 acquires the luminance signal of each pixel for each predetermined frame in the time series (for example, 30 Hz).

Here, when the sensor 11 is a camera, the acquisition unit 131 can acquire the data as shown in the graph 11G and the graph 21G of FIG. FIG. 5 is a diagram showing an example of a pulse wave waveform extraction process and a section setting process according to the first embodiment. Graphs 11G and 21G show changes in the luminance signal for each wavelength band of B (Blue), G (Green), and R (Red). The horizontal axis of each graph in FIG. 5 represents time. Further, the vertical axis of each graph in FIG. 5 represents the signal strength. However, in this embodiment, the unit on the horizontal axis of each graph in FIG. 5 is seconds. Further, the unit on the vertical axis of each graph in FIG. 5 may be an arbitrary unit common to each graph.

In addition, the acquisition unit 131 detects each part of the body from the moving image data, identifies the area of the detected part, and then obtains the imaging time series data of each area. For example, the acquisition unit 131 can detect a face, cheeks, palms, ear lobes, neck, etc. as parts, and can specify a region on the image of each of the detected parts.

Returning to the description of FIG. 4, the extraction unit 132 extracts the pulse wave waveform corresponding to each of the plurality of regions from the time series data in the plurality of regions of the body acquired by the sensor 11. Here, the extraction unit 132 receives the time-series data in the plurality of regions of the body acquired by the sensor 11 from the acquisition unit 131. Further, the extraction unit 132 extracts the pulse wave waveform as shown in the graph 12G and the graph 22G of FIG.

When the sensor 11 is a photoelectric pulse wave sensor, the extraction unit 132 sets a frequency band (for example, 40-240 bpm (beats per minute)) that the pulse wave can take with respect to the time-series waveform data acquired by the photoelectric pulse wave sensor. ) Can be specified to extract the pulse wave waveform. Note that bpm is the heart rate per minute, and 1 Hz = 60 bpm holds.

On the other hand, when the sensor 11 is a camera, the extraction unit 132 extracts the pulse wave waveform from the data of the moving image of the body taken by the camera. In this case, the extraction unit 132 obtains the pulse wave waveform by performing at least one of the statistical processing of the brightness for each wavelength component and the bandpass filtering processing for designating the frequency band that the pulse wave can take. Extract.

First, statistical processing of luminance for each wavelength component will be described. The extraction unit 132 can perform linear interpolation processing on the imaging time series data acquired by the acquisition unit 131. The linear interpolation processing is an example of statistical processing. FIG. 6 is a diagram showing an example of linear interpolation processing. As shown in FIG. 6, in the imaging time series data acquired by the acquisition unit 131, the time series intervals of the luminance signals Red, Blue, and Green for each wavelength band are not constant.

Here, the extraction unit 132 performs linear interpolation or spline interpolation to the imaging time series data, and calculates the luminance signals Red2, Green2, and Blue2 for each wavelength band in which the time series is constant. Further, the extraction unit 132 calculates dbGreen and dbRed as data for the filtering process described later. In addition, dbGreen is Green2 itself. Further, dbRed is calculated by (Red2 + Blue2) / 2.

Further, the extraction unit 132 extracts the pulse wave waveform by performing statistical processing for each wavelength component on dbGreen and dbRed. FIG. 7 is a diagram showing an example of statistical processing for each wavelength component. As shown in FIG. 7, the extraction unit 132 has a noise region extraction unit 132a and a pulse wave region extraction unit 132b.

The noise region extraction unit 132a extracts signal data in the noise region from the input dbGreen and dbRed by bandpass filter processing and lowpass filter processing. Further, the pulse wave region extraction unit 132b extracts signal data candidates in the pulse wave region from the input dbGreen and dbRed by bandpass filter processing. Further, the pulse wave region extraction unit 132b outputs the signal data obtained by removing noise from the extracted signal data candidates using the signal data extracted by the noise region extraction unit 132a as a pulse wave waveform.

Here, the processing of the noise region extraction unit 132a and the pulse wave region extraction unit 132b of FIG. 7 will be described. First, the BPF (Band-pass filter) 1321a extracts signal data in a frequency band of, for example, 12-18 bpm from dbGreen and dbRed. Note that 12-18 bpm is an example of a frequency band in which noise due to respiration is likely to appear. Next, ABS (absolute value) 1322a converts the input signal data into an absolute value. Then, the LPF (Low-pass filter) 1323a performs a smoothing process. LPF1323a extracts signal data in a frequency band of, for example, 0.015 Hz (0.9 bpm) or less. The division unit 1324a calculates k = Red / Green. Red and Green here are signal data extracted by LPF1323a.

BPF1321b extracts signal data in the frequency band, for example 42-150 bpm, from dbGreen and dbRed. 42-150 bpm is an example of a frequency band that a pulse wave can take. Next, the division unit 1322b calculates Red'= Red / k. Red here is the signal data extracted by BPF1321b. Further, k is a value calculated by the division unit 1324a. Then, the subtraction unit 1323b calculates Green-Red'. Green here is signal data extracted by BPF1321b. Here, BFP1324b extracts signal data in a frequency band of, for example, 42-150 bpm from Green-Red'.

Next, a case where bandpass filtering processing for specifying a frequency band that can be taken by a pulse wave is performed will be described. In this case, the extraction unit 132 extracts the pulse wave waveform by using a filter that passes, for example, 42-210 bpm as a frequency band that the pulse wave can take with respect to the luminance signal Green.

Returning to the description of FIG. 4, the setting unit 133 sets a time-series section of the period corresponding to the pulse wave waveform extracted by the extraction unit 132. As shown in the graph 13G and the graph 23G of FIG. 5, the setting unit 133 may set a section every 2 seconds, for example, from a predetermined time. In the example of FIG. 5, the setting unit 133 sets the time 27 as the start time, sets the section from time 27 to time 29, the section from time 29 to time 31, and the section from time 31 to time 33.

Further, the setting unit 133 may set a section in which the time corresponding to the zero cross point, the maximum point and the minimum point of the pulse wave waveform is set as the start time and the end time. FIG. 8 is a diagram for explaining the zero cross point. In the example of FIG. 8, the setting unit 133 sets a section in which the first zero cross point is the start time and the fifth zero cross point is the end time. The zero crossing point is an intersection of a zero value line representing 0 or a threshold value and a curve representing a waveform. Further, the setting unit 133 may set the section using the maximum value and the minimum value.

The correlation calculation unit 134 calculates the autocorrelation coefficient of each pulse wave waveform for each time series section. Here, the autocorrelation coefficient is an index showing the pulse wave-likeness of the waveform extracted as the pulse wave waveform by the extraction unit 132. Further, for example, the correlation calculation unit 134 can calculate the autocorrelation coefficient by the method described in Patent Document 2.

As shown in FIG. 9, the correlation calculation unit 134 calculates the autocorrelation for each graph and each section. FIG. 9 is a diagram showing an example of the correlation calculation process according to the first embodiment. Further, since the graph shows the pulse wave waveform for each part, the correlation calculation unit 134 calculates the autocorrelation coefficient for each part and each section.

The autocorrelation coefficient indicates the degree of correlation between the pulse wave waveform and the pulse wave waveform obtained by shifting the pulse wave waveform by a predetermined period. Therefore, the autocorrelation coefficient increases as the change in the period and amplitude of the pulse wave waveform becomes smaller. Further, the smaller the noise at the time of signal acquisition by the sensor 11, the more stable the pulse wave waveform and the larger the autocorrelation coefficient. On the contrary, the more noise at the time of signal acquisition by the sensor 11, the more unstable the pulse wave waveform becomes and the smaller the autocorrelation coefficient becomes.

For example, the correlation calculation unit 134 calculates the autocorrelation coefficient in the section from time 27 to time 29 in the graph 13G as 0.8. Further, for example, the correlation calculation unit 134 calculates the autocorrelation coefficient of the section from time 29 to time 31 in the graph 13G as 0.2. For example, the correlation calculation unit 134 calculates the autocorrelation coefficient in the section from time 31 to time 33 in the graph 13G as 0.8. In this way, even if the pulse wave waveform is at the same site, the autocorrelation coefficient may change significantly depending on the section.

Returning to the description of FIG. 4, the difference calculation unit 135 calculates the difference between each of the pulse wave waveforms and the reference waveform. Specifically, the difference calculation unit 135 calculates the phase difference and the amplitude ratio of the pulse wave waveform from the reference waveform. Further, the difference calculation unit 135 stores the calculated difference as the difference information 121 in the storage unit 12.

A method of calculating the difference by the difference calculation unit 135 will be described with reference to FIG. FIG. 10 is a diagram showing an example of the difference calculation process according to the first embodiment. In Figure 10 example, the amplitude of the pulse waveform 1w is _{S 1.} The time of the zero-crossing points of the pulse waveform 1w is t _1. The amplitude of the pulse waveform 1w is _{S 2.} The time at the zero crossing point of the pulse wave waveform 2w is t ₂ .

At this time, the difference calculation unit 135 calculates the phase difference of the pulse wave waveform 2w with respect to the pulse wave waveform 1w as t ₂ −t ₁ when the pulse wave waveform 1w is used as a reference. Further, the difference calculating unit 135, when with respect to the pulse waveform 1w, the amplitude ratio of the pulse waveform 1w pulse waveform _2w, calculated as _S 2 / S _1. Depending on how the reference is determined, the difference calculation unit 135 may calculate a negative value as the phase difference.

Here, even if the heartbeat that is the source of the pulse wave is common, it is conceivable that the time at which the contraction of the blood vessel due to the beat is transmitted may be different for each site. Therefore, when the setting unit 133 sets the section based on the zero cross point or the extreme value of the pulse wave waveform, the time difference until the contraction of the blood vessel is transmitted to each site causes a delay time. Then, this delay time appears in the phase difference between the pulse wave waveforms.

Returning to the description of FIG. 4, the selection unit 136 selects one of the pulse wave waveforms whose autocorrelation coefficient is equal to or higher than the threshold value from the pulse wave waveforms for each section. Further, the correction unit 137 corrects the pulse wave waveform selected by the selection unit 136 by using the difference.

The pulse wave waveform selection process and the correction process will be described with reference to FIG. FIG. 11 is a diagram showing an example of a pulse wave waveform selection process and a correction process according to the first embodiment. As shown in FIG. 11, the selection unit 136 selects either the pulse wave waveform of the graph 13G or the pulse wave waveform of the graph 23G in each section. Further, here, the threshold value is set to 0.6 as an example.

In the example of FIG. 11, the autocorrelation coefficient exceeds the threshold value in any of the pulse wave waveforms in the section from time 27 to time 29. In such a case, the selection unit 136 selects the pulse wave waveform corresponding to the region having the highest rank specified in advance for each region among the pulse waveforms having the autocorrelation coefficient equal to or higher than the threshold value. May be good. Here, the selection unit 136 selects the pulse wave waveform of the graph 13G. The selection unit 136 may select the pulse wave waveform having the largest autocorrelation coefficient.

In the section from time 29 to time 31, the autocorrelation coefficient of the pulse wave waveform of the graph 13G is equal to or less than the threshold value, and the autocorrelation coefficient of the pulse wave waveform of the graph 23G exceeds the threshold value. Therefore, the selection unit 136 selects the pulse wave waveform of the graph 23G.

Further, the correction unit 137 corrects the phase and amplitude of the pulse wave waveform selected by the selection unit 136 based on the phase difference and the amplitude factor. For example, as shown in FIG. 3, the phase difference between the pulse wave waveform of the site B and the pulse wave waveform of the site A is −11 msec, and the amplitude ratio is 0.91. Here, when the pulse wave waveform of the portion B is selected by the selection unit 136, the correction unit 137 shifts the selected pulse wave waveform by 11 msec in the traveling direction of the time series and multiplies the amplitude by 1 / 0.91. Make corrections by doing so.

Further, the correction unit 137 synthesizes the corrected pulse wave waveform and generates a pulse wave waveform for analysis. That is, the correction unit 137 corrects the pulse wave waveform selected from each part for each section and then joins the pulse wave waveform to generate the pulse wave waveform for analysis. The correction unit 137 may generate an analysis pulse wave waveform without correction, or may generate an analysis pulse wave waveform by joining the pulse wave waveforms corrected only for the phase difference.

Returning to the description of FIG. 4, the output control unit 138 causes the output unit 14 to output the characteristics of the analysis pulse wave waveform or the analysis pulse wave waveform. For example, the output control unit 138 causes the output unit 14 to output the graph 30G of FIG. Further, the output control unit 138 may cause the output unit 14 to output the waveform feature.

Waveform features will be described with reference to FIG. FIG. 12 is a diagram showing an example of waveform features. As shown in FIG. 12, waveform features include width between peak points, amplitude, top line, baseline, and the like. Further, the output control unit 138 may output the waveform characteristics of the first derivative or the second derivative of the pulse wave waveform.

[Processing flow]
The flow of the pulse wave calculation process of this embodiment will be described with reference to FIGS. 13 to 19. FIG. 13 is a flowchart showing an example of the pulse wave calculation process according to the first embodiment. As shown in FIG. 13, first, the pulse wave calculation device 10 acquires data for extracting a pulse wave waveform from the signal of the sensor 11 (step S1). Next, the pulse wave waveforms of a plurality of sites are extracted from the acquired data (step S2).

Here, the pulse wave calculation device 10 sets a section in the extracted pulse wave waveform (step S3). Then, the pulse wave calculation device 10 calculates the autocorrelation coefficient of the pulse wave waveform in each section for each part (step S4).

Next, the pulse wave calculation device 10 calculates the difference in phase and amplitude between the parts (step S5). Then, the pulse wave calculation device 10 generates a pulse wave waveform for analysis in each section (step S6). After that, the pulse wave calculation device 10 outputs the characteristics of the analysis pulse wave waveform or the analysis pulse wave waveform (step S7).

The details of the process (step S1) in which the pulse wave calculation device 10 acquires data will be described with reference to FIG. FIG. 14 is a flowchart showing an example of the acquisition process of the pulse wave waveform extraction data according to the first embodiment. The process shown in FIG. 14 is for the case where the sensor 11 is a camera. When the sensor 11 is a photoelectric pulse wave sensor, the pulse wave calculation device 10 can acquire the data itself acquired from the sensor 11 as data for extracting the pulse wave waveform data.

As shown in FIG. 14, first, the pulse wave calculation device 10 acquires an image of the camera (step S101). Next, the pulse wave calculation device 10 detects each part from the acquired image (step S102). Here, the pulse wave calculation device 10 specifies a region of the detected portion (step S103). Then, the pulse wave calculation device 10 calculates the region average of the brightness of the specified region (step S104).

Here, the pulse wave calculation device 10 determines whether or not there is an unprocessed portion (step S105). When there is an untreated portion (step S105, Yes), the pulse wave calculation device 10 returns to step S103 and repeats the processing for the unprocessed portion.

On the other hand, when there is no unprocessed portion (step S105, No), the pulse wave calculation device 10 proceeds to step S106. Then, the pulse wave calculation device 10 determines whether or not there is an unprocessed image (step S106). When there is an unprocessed image (step S106, Yes), the pulse wave calculation device 10 returns to step S101 and repeats the process for the unprocessed image. On the other hand, when there is no unprocessed image (step S106, No), a luminance waveform is generated from the region average (step S107).

Here, the unprocessed image in step S106 is, for example, an image of an unprocessed frame in a moving image acquired in time series. That is, the pulse wave calculation device 10 repeats the processes from steps S101 to S106 for the image of each frame of the moving image for a predetermined period.

Next, the details of the process (step S2) in which the pulse wave calculation device 10 extracts the pulse wave waveform will be described with reference to FIG. FIG. 15 is a flowchart showing an example of the pulse wave waveform extraction process according to the first embodiment. The process shown in FIG. 15 is for the case where the sensor 11 is a camera. However, even when the sensor 11 is a photoelectric pulse wave sensor, the pulse wave calculation device 10 can perform noise filter processing or the like in step S2.

As shown in FIG. 15, first, the pulse wave calculation device 10 extracts a noise region of the luminance waveform (step S201). Next, the pulse wave calculation device 10 extracts the pulse wave region of the luminance waveform (step S202). Then, the pulse wave calculation device 10 removes the noise region from the pulse wave region and extracts the pulse wave waveform (step S203).

Next, the details of the process (step S3) in which the pulse wave calculation device 10 sets the section will be described with reference to FIG. FIG. 16 is a flowchart showing an example of the section setting process according to the first embodiment. As shown in FIG. 16, the pulse wave calculation device 10 determines the start point of the pulse wave waveform (step S301). Next, the pulse wave calculation device 10 determines the end point of the pulse wave waveform (step S302). Then, the pulse wave calculation device 10 sets a section based on the start point and the end point. Here, the start point and the end point may be predetermined predetermined times, or may be times corresponding to the zero crossing point and the extreme value of the pulse wave waveform.

Next, the details of the process of calculating the correlation (step S4) will be described with reference to FIG. FIG. 17 is a flowchart showing an example of the correlation calculation process according to the first embodiment. As shown in FIG. 17, first, the pulse wave calculation device 10 selects a section (step S401). Next, the pulse wave calculation device 10 selects a site (step S402). Here, the pulse wave calculation device 10 calculates the autocorrelation coefficient in the selected section of the pulse wave waveform of the selected portion (step S403).

Here, the pulse wave calculation device 10 determines whether or not there is an unprocessed portion (step S404). When there is an unprocessed portion (step S404, Yes), the pulse wave calculation device 10 returns to step S402, selects the unprocessed portion, and repeats the process.

On the other hand, when there is no unprocessed portion (step S404, No), the pulse wave calculation device 10 proceeds to step S405. Then, the pulse wave calculation device 10 determines whether or not there is an unprocessed section (step S405). When there is an unprocessed section (step S405, Yes), the pulse wave calculation device 10 returns to step S401, selects the unprocessed section, and repeats the process. On the other hand, when there is no unprocessed section (step S405, No), the pulse wave calculation device 10 ends the process of calculating the correlation.

Next, the details of the process (step S5) in which the pulse wave calculation device 10 calculates the difference will be described with reference to FIG. FIG. 18 is a flowchart showing an example of the difference calculation process according to the first embodiment. As shown in FIG. 18, first, the pulse wave calculation device 10 selects a site (step S501). Next, the pulse wave calculation device 10 determines whether or not the maximum value of the autocorrelation coefficient in each section of the pulse wave waveform of the selected portion is larger than the threshold value (step S502). When the maximum value of the autocorrelation coefficient is larger than the threshold value (step S502, Yes), the pulse wave calculation device 10 calculates the phase difference and the amplitude ratio of the selected portion from the reference portion (step S503). On the other hand, when the maximum value of the autocorrelation coefficient is equal to or less than the threshold value (step S502, No), the pulse wave calculation device 10 proceeds to step S504 without calculating the phase difference and the amplitude ratio.

Here, the pulse wave calculation device 10 determines whether or not there is an unprocessed portion (step S504). When there is an untreated part (step S504, Yes), the pulse wave calculation device 10 returns to step S501, selects the untreated part, and repeats the process. On the other hand, when there is no unprocessed portion (step S504, No), the pulse wave calculation device 10 ends the process of calculating the difference.

Next, the details of the process (step S6) for generating the pulse wave waveform for analysis will be described with reference to FIG. First, as shown in FIG. 19, the pulse wave calculation device 10 selects a section (step S601). Next, the pulse wave calculation device 10 selects sites in order of priority (step S602). That is, in step S602, the pulse wave calculation device 10 selects the portion having the highest predetermined priority among the unselected portions.

Here, the pulse wave calculation device 10 determines whether or not the autocorrelation coefficient of the pulse wave waveform corresponding to the selected portion in the selected section is larger than the threshold value (step S603). When the autocorrelation coefficient is larger than the threshold value (step S603, Yes), the pulse wave waveform corresponding to the selected portion in the selected section is selected and held (step S604), and the process proceeds to step S607.

On the other hand, when the autocorrelation coefficient is equal to or less than the threshold value (step S603, No), the pulse wave calculation device 10 determines whether or not there is an unprocessed portion (step S605). When there is an untreated part (step S605, Yes), the pulse wave calculation device 10 returns to step S602, selects the unprocessed part in the order of priority, and repeats the processing.

On the other hand, when there is no untreated portion (step S605, No), the pulse wave calculation device 10 sets the selected section as the missing section (step S606). Then, the pulse wave calculation device 10 determines whether or not there is an unprocessed section (step S607). When there is an unprocessed section (step S607, Yes), the pulse wave calculation device 10 returns to step S601, selects the unprocessed section, and repeats the process.

On the other hand, when there is no unprocessed section (step S607, No), the pulse wave calculation device 10 corrects and synthesizes the held pulse wave waveform (step S608). Here, the pulse wave calculation device 10 corrects using the difference calculated in step S5.

[effect]
As described above, the pulse wave calculation device 10 extracts the pulse wave waveform corresponding to each of the plurality of regions from the time series data in the plurality of regions of the body acquired by the sensor 11. Further, the pulse wave calculation device 10 calculates each autocorrelation coefficient of the pulse wave waveform for each time series section. Further, the pulse wave calculation device 10 calculates the difference between each of the pulse wave waveforms and the reference waveform. Further, the pulse wave calculation device 10 selects one of the pulse wave waveforms whose autocorrelation coefficient is equal to or higher than the threshold value from the pulse wave waveforms for each section. Further, the pulse wave calculation device 10 corrects the selected pulse wave waveform by using the difference. In this way, the pulse wave calculation device 10 can calculate the pulse wave based on a plurality of time series data. Therefore, even if there is a section in which a large amount of noise is generated in some of the plurality of data, the pulse wave calculation device 10 can supplement the pulse wave in the section with other data. Therefore, the pulse wave calculation device 10 can continuously obtain pulse waves suitable for analysis.

Further, the pulse wave calculation device 10 extracts a pulse wave waveform from the data of the moving image of the body taken by the camera. As a result, the pulse wave calculation device 10 can obtain a pulse wave suitable for analysis even in a situation where it is difficult to use the photoelectric pulse wave sensor.

Further, in the pulse wave calculation device 10, the extraction unit 132 extracts the pulse wave waveform from the data of the moving image of the body taken by the camera. Further, the pulse wave calculation device 10 performs at least one of a statistical process of brightness for each wavelength component and a bandpass filtering process for designating a frequency band that the pulse wave can take on the data, so that the pulse wave waveform can be waveformd. Is extracted. As a result, the pulse wave calculation device 10 can remove noise generated in the image taken by the camera and calculate the pulse wave with high accuracy.

Further, the pulse wave calculation device 10 calculates the autocorrelation coefficient for each section in which the time corresponding to the zero cross point, the maximum point and the minimum point of the pulse wave waveform is set as the start time and the end time. As a result, the pulse wave calculation device 10 can easily set the section.

Further, the pulse wave calculation device 10 calculates the phase difference and the amplitude ratio of the pulse wave waveform from the reference waveform. Further, the pulse wave calculation device 10 corrects the phase and amplitude of the selected pulse wave waveform based on the phase difference and the amplitude factor. As a result, the pulse wave calculation device 10 can generate a pulse wave waveform for analysis by correcting the difference even if there is a delay or a difference in intensity in the pulse wave waveforms of a plurality of parts.

Further, the pulse wave calculation device 10 selects the pulse wave waveform corresponding to the region having the highest rank designated in advance for each region among the pulse wave waveforms having the autocorrelation coefficient equal to or higher than the threshold value. This makes it easier for the pulse wave calculation device 10 to select a pulse wave waveform of a desired portion as a reference.

[Other Examples]
So far, as shown in FIG. 1, an example in which the pulse wave calculation device 10 includes the sensor 11 and the output unit 14 has been described. On the other hand, the pulse wave calculation device of the present invention may be realized by another configuration as shown in FIG. FIG. 20 is a diagram showing an example of a usage scene of the pulse wave calculation device according to another embodiment. As shown in FIG. 20, the pulse wave calculation device 10a is a server connected to the network N. Further, the photographing device 20 and the output device 30 are connected to the network N. For example, the photographing device 20 is a smartphone, a Web camera, or the like. The output device 30 is a smartphone, a personal computer, or the like.

Further, the pulse wave calculation device 10a has the same function as the pulse wave calculation device 10 of the first embodiment. However, the pulse wave calculation device 10a acquires the moving image data acquired by the photographing device 20 via the network N. Further, the pulse wave calculation device 10a transmits the pulse wave waveform and the like calculated based on the moving image data to the output device 30 via the network N. Then, the output device 30 outputs a pulse wave waveform or the like.

[system]
It is also possible to manually perform a part of the processes described as being automatically performed among the processes described in each embodiment. Alternatively, all or part of the processing described as being performed manually can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific forms of distribution and integration of each device are not limited to those shown in the figure. That is, all or a part thereof can be functionally or physically distributed / integrated in an arbitrary unit according to various loads, usage conditions, and the like. For example, the correlation calculation unit 134 and the difference calculation unit 135 shown in FIG. 2 may be integrated. Further, the extraction unit 132 shown in FIG. 2 may be dispersed in the noise region extraction unit 132a and the pulse wave region extraction unit 132b. Further, each processing function performed by each device may be realized by a processor and a program analyzed and executed by the processor, or may be realized as hardware by wired logic.

The various processes described in the above examples can be realized by executing a program prepared in advance on a computer. Therefore, in the following, an example of a computer that executes a program having the same function as that of the above embodiment will be described. FIG. 21 is a diagram showing an example of a computer that executes a pulse wave calculation program.

The computer 100 has an operation unit 110a, a sensor 110b, a display 120, and a communication unit 130. Further, the computer 100 includes a processor 150, a ROM (Read Only Memory) 160, an external storage device 170, and a RAM (Random Access Memory) 180. Each part of these 110 to 180 is connected via a bus 140.

Examples of the RAM 180 include memories such as SDRAM (Synchronous Dynamic Random Access Memory) and flash memory. Examples of the processor 150 include a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), a PLD (Programmable Logic Device), and the like.

As shown in FIG. 21, the external storage device 170 stores in advance a pulse wave calculation program 170a that exhibits the same functions as each functional unit shown in the above embodiment. The pulse wave calculation program 170a may be integrated or separated as appropriate, as in the case of each component shown in FIG. That is, as for each data stored in the external storage device 170, it is not always necessary that all the data is stored in the external storage device 170, and only the data necessary for processing needs to be stored in the external storage device 170.

Then, the processor 150 reads the pulse wave calculation program 170a from the external storage device 170 and deploys it in the RAM 180. As a result, as shown in FIG. 21, the pulse wave calculation program 170a functions as the pulse wave calculation process 180a. The pulse wave calculation process 180a appropriately expands various data read from the external storage device 170 into an area assigned to itself on the RAM 180, and executes various processes based on the expanded various data. The pulse wave calculation process 180a includes a process executed by each functional unit shown in FIG. 2, for example, a process shown in FIGS. 13 to 19. Further, as for each processing unit virtually realized on the processor 150, it is not always necessary for all the processing units to operate on the processor 150, and only the processing units necessary for processing need to be virtually realized.

The pulse wave calculation program 170a does not necessarily have to be stored in the external storage device 170 or the ROM 160 from the beginning. For example, each program can be stored in a "portable physical medium" inserted into the computer 100. As the portable physical medium, any medium such as a flexible disk, so-called FD, CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), magneto-optical disk, or IC (Integrated Circuit) card can be adopted. Then, the computer 100 may acquire and execute each program from these portable physical media. Further, each program is stored in another computer or server device connected to the computer 100 via a public line, the Internet, LAN, WAN, etc., so that the computer 100 acquires and executes each program from these. You may do it.

10, 10a Pulse wave calculation device 11 Sensor 12 Storage unit 13 Control unit 14 Output unit 20 Imaging device 30 Output device 121 Difference information 131 Acquisition unit 132 Extraction unit 133 Setting unit 134 Correlation calculation unit 135 Difference calculation unit 136 Selection unit 137 Correction unit 138 Output control unit

Claims (8)

An extraction unit that extracts pulse wave waveforms corresponding to each of the plurality of regions from time-series data acquired by the sensor in a plurality of regions of the body.
A correlation calculation unit that calculates the auto-correlation coefficient of each of the pulse wave waveforms for each time-series section,
A difference calculation unit that calculates the difference between each of the pulse wave waveforms and the reference waveform, and
For each section, a selection unit that selects one of the pulse wave waveforms whose autocorrelation coefficient is equal to or higher than the threshold value from the pulse wave waveforms.
A correction unit that corrects the pulse wave waveform selected by the selection unit using the difference, and a correction unit.
A pulse wave calculation device characterized by having. The pulse wave calculation device according to claim 1, wherein the extraction unit extracts the pulse wave waveform from data of a moving image of the body taken by a camera. The extraction unit extracts the pulse wave waveform by performing at least one of a statistical process of brightness for each wavelength component and a bandpass filtering process for designating a frequency band that a pulse wave can take on the data. The pulse wave calculation device according to claim 2, wherein the pulse wave calculation device is characterized. Claim 1 is characterized in that the correlation calculation unit calculates the auto-correlation coefficient for each section whose start time and end time are the times corresponding to the zero cross point, the maximum point, and the minimum point of the pulse wave waveform. The pulse wave calculation device according to. The difference calculation unit calculates the phase difference and the amplitude ratio of the pulse wave waveform from the reference waveform.
The pulse wave calculation device according to claim 1, wherein the correction unit corrects the phase and amplitude of the pulse wave waveform selected by the selection unit based on the phase difference and the amplitude factor. The claim is characterized in that the selection unit selects a pulse wave waveform corresponding to a region having the highest rank designated in advance for each of the regions among the pulse waveforms having the autocorrelation coefficient equal to or higher than the threshold value. Item 1. The pulse wave calculation device according to item 1. The computer
From the time-series data in multiple regions of the body acquired by the sensor, the pulse wave waveform corresponding to each of the multiple regions of the body is extracted.
The autocorrelation coefficient of each of the pulse wave waveforms was calculated for each time series section.
The difference between each of the pulse wave waveforms and the reference waveform is calculated.
For each section, one of the pulse wave waveforms having the autocorrelation coefficient equal to or higher than the threshold value is selected from the pulse wave waveforms.
The pulse wave waveform selected by the selection process is corrected by using the difference.
A pulse wave calculation method characterized by executing processing. On the computer
From the time-series data in a plurality of regions of the body acquired by the sensor, the pulse wave waveform corresponding to each of the plurality of regions is extracted, and the pulse wave waveform is extracted.
The autocorrelation coefficient of each of the pulse wave waveforms was calculated for each time series section.
The difference between each of the pulse wave waveforms and the reference waveform is calculated.
For each section, one of the pulse wave waveforms having the autocorrelation coefficient equal to or higher than the threshold value is selected from the pulse wave waveforms.
The pulse wave waveform selected by the selection process is corrected by using the difference.
A pulse wave calculation program characterized by executing processing.

Images