WO2023214457A1

WO2023214457A1 - Pulse wave estimation device and pulse wave estimation method

Info

Publication number: WO2023214457A1
Application number: PCT/JP2022/019552
Authority: WO
Inventors: 遼平村地; 雄大中村
Original assignee: 三菱電機株式会社
Priority date: 2022-05-06
Filing date: 2022-05-06
Publication date: 2023-11-09
Also published as: JPWO2023214457A1

Abstract

This pulse wave estimation device is provided with: a captured image acquisition unit (11) for acquiring a captured image obtained by imaging a human; a skin region detection unit (12) for detecting a skin region in the human from the captured image; a measurement region setting unit (13) for setting a plurality of measurement regions in a region corresponding to the skin region on the captured image, in which each of the measurement regions is used for extracting therefrom a pulse wave original signal that shows the change in brightness over time in a first period; a pulse wave original signal extraction unit (14) for extracting a pulse wave original signal with respect to each of the measurement regions on the basis of the change in brightness in the first period in each of the measurement regions; a segment generation unit (15) for generating a plurality of pulse wave original signal segments with respect to the measurement regions on the basis of the pulse wave original signal extracted from each of the measurement regions in accordance with a segment generation condition, in which each of the pulse wave original signal segments is a signal produced by partially extracting a pulse wave original signal for a second period from the pulse wave original signals in the first period; and a pulse wave estimation unit (16) for estimating the pulse wave of the human on the basis of the plurality of purse wave original signal segments generated with respect to the measurement regions.

Description

Pulse wave estimation device and pulse wave estimation method

The present disclosure relates to a pulse wave estimation device and a pulse wave estimation method.

Conventionally, human skin is measured based on luminance signals of multiple areas (hereinafter referred to as "measurement areas") set in an area containing human skin (hereinafter referred to as "skin area") in a captured image captured by an imaging device. There is a known technology for estimating a person's pulse wave by extracting minute changes in brightness on the surface and separating the pulse wave component from the signal indicating the brightness change using signal separation techniques such as independent component analysis or principal component analysis. (For example, Patent Document 1). Such technology utilizes the principle that the amount of light absorbed in a person's bloodstream changes in response to pulsations.

JP2017-93760A

Blood leaves the heart in arteries, circulates throughout the body, and then returns to the heart through veins. Due to the characteristics of such human blood flow, if there is a large difference in distance from the heart between multiple measurement areas set for estimating pulse waves, the multiple measurement areas may A phase difference occurs in the luminance signals extracted from the regions. If a phase difference occurs in the brightness signals extracted between a plurality of measurement regions, the pulse wave component cannot be separated from the signal indicating the brightness change, and there is a possibility that the accuracy of estimating the pulse wave will decrease.
With conventional technology, if a phase difference occurs in the luminance signals extracted from multiple measurement regions, there is a problem that it may not be possible to estimate a person's pulse wave based on the luminance signal with the phase difference. .

The present disclosure has been made to solve the above-mentioned problems, and even if there is a phase difference between the luminance signals extracted from multiple measurement regions, a person's pulse wave can be estimated based on the luminance signals. The purpose of the present invention is to provide a pulse wave estimating device that can perform the following steps.

The pulse wave estimating device according to the present disclosure includes a captured image acquisition unit that acquires a captured image of a person, a skin area detection unit that detects a skin area of the person from the captured image, and a skin area that corresponds to the skin area on the captured image. A measurement region setting unit that sets a plurality of measurement regions in the region for extracting pulse wave source signals indicating time-series luminance changes in the first period; a pulse wave source signal extraction unit that extracts a pulse wave source signal based on the change; and a pulse wave source signal extracting unit that extracts a pulse wave source signal for each measurement region, based on the pulse wave source signal extracted from the measurement region, and according to segment generation conditions, pulse wave in the first period. A segment generation unit that generates a plurality of pulse wave source signal segments that are signals obtained by partially extracting the pulse wave source signal for a second period from the original signal, and a plurality of pulse wave source signals generated for each measurement area. and a pulse wave estimator that estimates a person's pulse wave based on the segments.

According to the present disclosure, even if a phase difference occurs in the luminance signals extracted from a plurality of measurement regions, a person's pulse wave can be estimated based on the luminance signals.

1 is a diagram showing a configuration example of a pulse wave estimating device according to Embodiment 1. FIG. FIG. 3 is a diagram for explaining a phase difference in luminance signals caused by characteristics of human blood flow that occurs between a plurality of measurement regions. 3A, FIG. 3B, and FIG. 3C are diagrams for explaining an example of a method for setting a measurement region by a measurement region setting section in the pulse wave estimating device according to the first embodiment. FIG. 3 is a diagram for explaining an example of a pulse wave source signal segment generated by a segment generation unit in the first embodiment. FIG. 7 is a diagram for explaining a flow in which the signal separation unit analyzes a principal component based on a plurality of pulse wave source signal segments and generates a separated signal indicating the analyzed principal component in the first embodiment. FIG. 3 is a diagram for explaining an example of details of a separated signal indicating a main component, which is generated by a signal separation unit in the first embodiment. 3 is a flowchart for explaining the operation of the pulse wave estimating device according to the first embodiment. 8 is a flowchart for explaining details of pulse wave estimation processing by the pulse wave estimator in step ST6 of FIG. 7. FIG. 9A and 9B are diagrams illustrating an example of the hardware configuration of the pulse wave estimating device according to the first embodiment. FIG. 3 is a diagram illustrating a configuration example of a pulse wave estimating device according to a second embodiment. 7 is a flowchart for explaining the operation of the pulse wave estimating device according to Embodiment 2. FIG. FIG. 7 is a diagram illustrating a configuration example of a pulse wave estimating device according to Embodiment 3. FIG. 7 is a diagram for explaining pulse wave source signal segments used by the pulse wave estimating device to estimate a subject's pulse wave in Embodiment 3; 12 is a flowchart for explaining the operation of the pulse wave estimating device according to Embodiment 3. In Embodiments 1 to 3, the pulse wave estimating device sets component weights for a plurality of separated signals indicating a plurality of principal components estimated using a known general signal separation technique, and It is a figure which shows the example of a structure of the pulse wave estimating part in the case where it has the function of restoring the pulse wave source signal for each measurement area based on the component weight for each separated signal which was set. FIG. 3 is a diagram illustrating an example of projection coefficient time series information generated by a component weight setting unit. In Embodiments 1 to 3, the pulse wave estimating device sets component weights for a plurality of separated signals indicating a plurality of principal components estimated using a known general signal separation technique, and 12 is a flowchart for explaining the operation of the pulse wave estimating section when it has a function of restoring the pulse wave source signal for each measurement area based on the set component weight for each separated signal.

Embodiments of the present disclosure will be described in detail below with reference to the drawings.
Embodiment 1.
FIG. 1 is a diagram showing a configuration example of a pulse wave estimating device 1 according to the first embodiment.

The pulse wave estimating device 1 estimates the pulse wave of a person based on a captured image of the person. In the following description, a person whose pulse wave is estimated by the pulse wave estimating device 1 is also referred to as a "subject."
The pulse wave estimating device 1 captures a series of frames at a predetermined frame rate Fr, capturing images of at least a range where a skin region that includes the subject's skin should exist (hereinafter referred to as "skin existing range"). A captured image consisting of Im(k) is acquired. Here, k indicates a frame number assigned to each frame. For example, the frame given at the next timing after frame Im(k) is frame Im(k+1).
In the first embodiment, the skin area is an area corresponding to the subject's face. Note that this is just an example, and the skin area may be an area other than the subject's face. For example, the skin area may be an area corresponding to a part of the face such as the subject's eyes, eyebrows, nose, mouth, forehead, cheeks, or chin. Furthermore, the skin area may be an area corresponding to a body part other than the face, such as the subject's head, shoulders, hands, neck, or feet. The skin area may be a plurality of areas, such as a skin area corresponding to the cheek, a skin area corresponding to the forehead, or a skin area corresponding to the neck.

Then, the pulse wave estimating device 1 generates a series of frames Im( The pulse wave of the subject is estimated from k−Tp+1) to Im(k), and pulse wave estimation result P(t), which is information indicating the estimated pulse wave (hereinafter referred to as “pulse wave information”), is output. Specifically, the pulse wave estimating device 1 sets a plurality of regions (hereinafter referred to as "measurement regions") for the subject's skin region in a series of frames Im(k-Tp+1) to Im(k). Then, the pulse wave estimating device 1 performs independent component analysis (hereinafter referred to as "ICA") or Pulse wave components are separated using a general signal separation technique such as principal component analysis (hereinafter referred to as "PCA"), and the subject's pulse wave is estimated from the separated pulse wave components. In separating the pulse wave components, the pulse wave estimation device 1 partially extracts the pulse wave source signal of a preset period (hereinafter referred to as "second period") from the pulse wave source signal of the first period. A plurality of pulse wave source signals (hereinafter referred to as "pulse wave source signal segments") are generated, and ICA or PCA is performed on the plurality of pulse wave source signal segments.

Here, t indicates an output number assigned to each specific number of frames Tp acquired in the first period. For example, the pulse wave estimation result given at the next timing of the pulse wave estimation result P(t) is the pulse wave estimation result P(t+1). The frame number k and the output number t are integers of 1 or more. The number of frames Tp is an integer of 2 or more.

It is effective to estimate the subject's pulse wave from the luminance changes of multiple measurement areas set in the skin area. However, due to the characteristics of human blood flow, if the distance between the plurality of measurement regions is large, a phase difference may occur between the luminance signals extracted from the plurality of measurement regions.

Here, FIG. 2 is a diagram for explaining a phase difference in luminance signals due to characteristics of human blood flow that occurs between a plurality of measurement regions.
In FIG. 2, the left side of the figure is a diagram showing a skin area in a captured image of a subject. In FIG. 2, the subject is indicated by "H" and the skin area is indicated by "sr". It is assumed that four measurement regions are set in the skin region. In FIG. 2, four measurement regions are indicated by "R1", "R2", "R3", and "R4". Note that the details of the method of detecting the skin area and the method of setting the measurement area will be described later.
The upper right side of the figure shows a diagram (indicated by 22 in FIG. 2) on the time axis of the luminance value indicated by the luminance signal extracted from the measurement region indicated by "R1", and a diagram showing the luminance value indicated by the luminance signal extracted from the measurement region indicated by "R3" on the time axis. FIG. 3 is a diagram (indicated by 21 in FIG. 2) showing the luminance values indicated by the luminance signals in the time axis.
The subject's blood is pumped from the heart and flows from the neck to the forehead. Therefore, when the distance between the measurement regions in the direction of blood flow is large, such as the measurement region indicated by "R1" and the measurement region indicated by "R3", in other words, the difference in distance from the heart between the measurement regions is large. Then, a phase difference occurs in the luminance signal extracted from the measurement area.
If a phase difference occurs between multiple luminance signals extracted from multiple measurement regions, it is possible to use a general signal separation technique such as ICA or PCA on the pulse wave source signal for each measurement region. If the pulse wave component is separated, the separation of the pulse wave component ends in failure.
The pulse wave estimating device 1 according to the first embodiment generates a plurality of pulse wave source signal segments and performs ICA on the generated pulse wave source signal segments in consideration of the occurrence of the phase difference as described above. Alternatively, by separating pulse wave components using a general signal separation technique such as PCA, even if a phase difference occurs between multiple luminance signals extracted from multiple measurement regions, the phase difference can be The pulse wave of the subject can be estimated by absorbing it.

Note that the number of subjects included in the captured image may be one or more. To simplify the explanation, in the following first embodiment, the number of subjects included in the captured image is one.

In Embodiment 1, as an example, it is assumed that the pulse wave estimating device 1 is mounted on a vehicle (not shown), and the subject is a driver of the vehicle. That is, the pulse wave estimating device 1 estimates the pulse wave of the driver of the vehicle. The driver's pulse wave information estimated by the pulse wave estimating device 1 is output to, for example, a state estimating device (not shown) mounted on the vehicle. The state estimating device estimates the driver's state based on the driver's pulse wave estimation result P(t) output from the pulse wave estimating device 1. For example, when the state estimating device estimates that the driver's state is not suitable for driving, it outputs a warning sound to the driver.

As shown in FIG. 1, the pulse wave estimation device 1 includes a captured image acquisition section 11, a skin region detection section 12, a measurement region setting section 13, a pulse wave source signal extraction section 14, a segment generation section 15, and a pulse wave estimation section 16. , and an output section 17. The pulse wave estimating section 16 includes a signal separating section 161, a restoring section 162, and an estimating section 163.
The pulse wave estimating device 1 is connected to an imaging device 3 mounted on a vehicle. The imaging device 3 is installed to be able to image the skin area of the subject, here the driver. For example, the imaging device 3 may be shared with an imaging device included in a so-called "Driver Monitoring System (DMS)" that is installed in a vehicle to monitor the condition of the driver in the vehicle. .

The captured image acquisition unit 11 acquires a captured image of the subject from the imaging device 3.
The captured image acquisition unit 11 outputs the acquired captured image to the skin area detection unit 12.

The skin area detection unit 12 detects the skin area of the subject from the frame Im(k) included in the captured image acquired by the captured image acquisition unit 11. The skin area detection unit 12 may detect the skin area using a known means. For example, the skin area detection unit 12 can detect a skin area using a cascade type face detector using Haar-like features.
The skin area detection unit 12 generates skin area information S(k) indicating the detected skin area.
The skin area information S(k) can include information indicating whether or not a skin area has been detected, and information indicating the position and size of the detected skin area on the captured image. In the first embodiment, it is assumed that the skin area is represented by a rectangular area on the captured image, and the skin area information S(k) includes information indicating the position and size of the rectangular area on the captured image.
Specifically, when the skin area is an area corresponding to the subject's face, the skin area information S(k) includes, for example, whether or not the subject's face is detected and the rectangle surrounding the subject's face on the captured image. The center coordinates Fc (Fcx, Fcy) of the rectangle, and the width Fcw and height Fch of this rectangle are shown. The presence or absence of detection of the subject's face is represented by, for example, "1" if it is detected, and "0" if it is not detected. Furthermore, the center coordinates of the rectangle surrounding the face are expressed in the coordinate system of frame Im(k).
Note that the skin area detection unit 12 can also detect multiple skin areas.
The skin area detection unit 12 outputs the generated skin area information S(k) to the measurement area setting unit 13.

The measurement area setting unit 13 determines the area on the frame Im(k) based on the frame Im(k) of the captured image acquired by the captured image acquisition unit 11 and the skin area information S(k) outputted by the skin area detection unit 12. A plurality of measurement regions are set in the image region corresponding to the skin region indicated by the skin region information S(k) of , for extracting the pulse wave source signal indicating the time-series luminance change in the first period. Note that the measurement area setting unit 13 may acquire the captured image acquired by the captured image acquisition unit 11 via the skin area detection unit 12.
When the measurement area setting unit 13 sets a plurality of measurement areas, it generates measurement area information R(k) indicating the plurality of measurement areas that have been set. The measurement area information R(k) includes information indicating the positions and sizes of N (an integer greater than or equal to 2) measurement areas on the captured image. Each measurement area is defined as a measurement area ri(k) (i=2, . . . , N). In the first embodiment, the measurement region ri(k) is a quadrilateral, and the position and size of the measurement region ri(k) are the coordinate values of the four vertices of the quadrilateral on the captured image.

Here, FIG. 3A, FIG. 3B, and FIG. 3C are diagrams for explaining an example of a method for setting the measurement region ri(k) by the measurement region setting section 13 in the pulse wave estimating device 1 according to the first embodiment. be.
An example of a method by which the measurement area setting unit 13 sets a plurality of measurement areas ri(k) will be described with reference to FIG. 3.
First, as shown in FIGS. 3A and 3B, the measurement region setting unit 13 selects facial organs such as the outer corners of the eyes, inner corners of the eyes, nose, and mouth in the skin region sr indicated by the skin region information S(k). Detect Ln (positive integer) landmarks. In FIGS. 3A and 3B, landmarks are shown as circles. The measurement area setting unit 13 sets a vector storing the coordinate values of the detected landmarks to L(k).
Note that the measurement area setting unit 13 may detect facial organs using a known method such as using a model called a Constrained Local Model (CLM).

Next, the measurement area setting unit 13 sets the coordinates of the vertices of the quadrilateral of the measurement area ri(k) using the detected landmark as a reference. For example, the measurement area setting unit 13 sets the vertex coordinates of a quadrilateral as shown in FIG. 3C, and sets N measurement areas ri(k).

To explain using an example in which the measurement area setting unit 13 sets the measurement area ri(k) in the part corresponding to the cheek of the skin area sr, the measurement area setting unit 13 sets the landmark LA1 of the facial contour and the nose. Select landmark LA2. The measurement area setting unit 13 first selects the nasal landmark LA2, and then selects the facial contour landmark LA1 closest to the nasal landmark LA2.
Then, the measurement area setting unit 13 sets auxiliary landmarks a1, a2, and a3 so as to equally divide the line segment between the landmark LA1 and the landmark LA2 into four.
Similarly, the measurement area setting unit 13 selects the facial contour landmark LB1 and the nose landmark LB2. Furthermore, the measurement area setting unit 13 sets auxiliary landmarks b1, b2, and b3 so as to equally divide the line segment between the landmark LB1 and the landmark LB2 into four. Note that the landmarks LB1 and LB2 may be selected from, for example, the contour of the face or the landmark of the nose adjacent to the landmarks LA1 and LA2, respectively.
The measurement area setting unit 13 sets a quadrilateral area surrounded by the auxiliary landmarks a1, b1, b2, and a2 as one measurement area R5. The auxiliary landmarks a1, b1, b2, and a2 each have vertex coordinates corresponding to the measurement region R5.
Similarly, the measurement area setting unit 13 sets one measurement area R6 surrounded by the auxiliary landmarks a2, b2, b3, and a3 and the vertex coordinates of the measurement area R6.

Although an example has been described here in which the measurement region ri(k) is set on the corresponding part of the cheek, the measurement region setting unit 13 can set the measurement region ri(k) on the part corresponding to the cheek. Similarly, the measurement region ri(k) and the vertex coordinates of the measurement region ri(k) are set for the region sr. Although not shown in FIG. 3C, the measurement region setting unit 13 sets a measurement region ri(k ) may be set.

Note that the measurement region setting unit 13 may set the measurement region ri(k) using a method other than CLM. For example, the measurement region setting unit 13 may set the measurement region ri(k) using a tracking technique such as a Kanade-Lucas-Tomasi (KLT) tracker. Specifically, the measurement area setting unit 13 detects the coordinates of facial organ points using CLM for the skin area of the first frame Im(1) of the series of frames Im(k-Tp+1) to Im(k). However, from the skin region of the next frame Im(2) onward, the facial organ points may be tracked by the KLT tracker and the facial organ points for the skin region of each frame Im(k) may be calculated. In this case, since detection errors due to tracking accumulate, the measurement area setting unit 13 may perform a reset process such as executing CLM once every several frames and resetting the coordinate positions of facial organ points.

The measurement area setting unit 13 outputs the generated measurement area information R(k) to the pulse wave source signal extraction unit 14.

The pulse wave source signal extraction unit 14 extracts the frame Im( A pulse wave source signal indicating a luminance change in the first period is extracted from each of a plurality of measurement regions ri(k) shown by measurement region information R(k) on k). Note that the pulse wave source signal is a signal that is the source of the pulse wave. The pulse wave estimating device 1 estimates the pulse wave of the subject using the pulse wave source signal. The pulse wave estimation unit 16 estimates the pulse wave of the subject. Details of the pulse wave estimation section 16 will be described later.
The pulse wave source signal extraction unit 14 may acquire the captured image acquired by the captured image acquisition unit 11 via the skin area detection unit 12 and the measurement area setting unit 13.
After extracting the pulse wave source signal, the pulse wave source signal extraction unit 14 generates pulse wave source signal information W(t) indicating the extracted pulse wave source signal.

The pulse wave source signal information W(t) includes information indicating the pulse wave source signal wi(t) extracted in the measurement region ri(k). The pulse wave source signal wi(t) is time-series data for Tp, for example, frames Im(k-Tp+1), Im(k-Tp+2), ..., Im(k) for the past Tp, It is extracted based on the measurement area information R(k-Tp+1), R(k-Tp+2), . . . , R(k).
In extracting the pulse wave source signal wi(t), the pulse wave source signal extracting unit 14 extracts the difference between the previous frame Im(k-1) and the previous frame Im(k-1) for each frame Im(k) of the captured image. A difference Gi(j) (j=k−Tp+1, k−Tp+2, . . . , k) between the luminance features of each measurement region ri(k) is calculated. The luminance feature amount is a value calculated for each measurement region ri(j) based on the luminance value on frame Im(j) of the captured image. The brightness feature amount is the average or variance of the brightness values of pixels included in the measurement region ri(j). In the first embodiment, as an example, the brightness feature amount is the average brightness value of pixels included in the measurement region ri(j).
The pulse wave source signal extracting unit 14 arranges Gi(j) calculated for each frame Im(k) of the captured image acquired in the first period in chronological order to obtain a pulse wave source signal wi(t). That is, the pulse wave source signal extraction unit 14 sets the pulse wave source signal wi(t)=[Gi(k−Tp+1), Gi(k−Tp+2), . . . , Gi(k)].

The pulse wave source signal extraction unit 14 extracts the pulse wave source signal wi(t) by calculating the luminance feature amount of each measurement region ri(k) for each frame Im(k) of the captured image. May be extracted.

The pulse wave source signal extraction unit 14 generates pulse wave source signal information W(t) indicating the pulse wave source signal wi(t) in each measurement region ri(k). The pulse wave source signal information W(t) includes the pulse wave source signal wi(t) in each measurement region ri(k) and the pulse wave source signal wi(t) from which measurement region ri(k). and information indicating whether the wave source signal wi(t) is the wave source signal wi(t).
The pulse wave source signal extraction section 14 outputs the generated pulse wave source signal information W(t) to the segment generation section 15.

The segment generation unit 15 generates preset conditions (hereinafter referred to as “segment generation conditions”) for each measurement region ri(k) based on the pulse wave source signal information W(t) output from the pulse wave source signal extraction unit 14. ), the pulse wave source signal segment is a signal obtained by partially extracting the pulse wave source signal wi(t) for the second period from the pulse wave source signal wi(t) for the first period. Generate multiple.

The segment generation conditions are generated in advance by an administrator or the like, and are held by the segment generation unit 15.
For example, the following conditions (1) to (5) are set as the segment generation conditions. The segment generation unit 15 generates a plurality of pulse wave source signal segments so as to satisfy all of (1) to (5).
(1) The second period is shorter than the first period. (2) Regarding multiple pulse wave source signal segments in a certain measurement area, one pulse wave source signal segment is connected to other pulse wave source signal segments on the time axis. (3) Regarding multiple pulse wave source signal segments in a certain measurement area, one pulse wave source signal segment partially overlaps with another pulse wave source signal segment on the time axis. (4) In each measurement area, the length of partial overlap between a certain pulse wave source signal segment and another pulse wave source signal segment on the time axis is the same (5) Multiple pulse wave source signal segments in each measurement area The lengths of the pulse wave source signal segments on the time axis are the same.

Here, FIG. 4 is a diagram for explaining an example of a pulse wave source signal segment generated by the segment generation unit 15 in the first embodiment.
The upper left diagram in FIG. 4 is a diagram (hereinafter referred to as "skin area diagram") showing the skin region sr in which the measurement region ri(k) is set. In the skin area diagram, the subject is indicated by "H". As shown in the skin region diagram, it is assumed that four measurement regions ri(k) are now set in the skin region sr. Specifically, two measurement regions ri(k) (indicated by “n ₁ ” and “n ₂ ” on the skin area map) are placed on the subject's chin area, and two measurement areas ri(k) are placed on the subject's forehead area. ) (indicated by "f ₁ " and "f ₂ " on the skin area diagram) are set. In the following description, the measurement region ri(k) indicated by "n ₁ " is referred to as the "first measurement region", the measurement region ri(k) indicated by "n ₂ " is referred to as the "second measurement region", and the measurement region ri(k) indicated by "f ₁ " is referred to as the "first measurement region". The measurement region ri(k) shown is called a "third measurement region", and the measurement region ri(k) shown by "f ₂ " is called a "fourth measurement region".

The middle diagram in the upper part of FIG. 4 is a diagram for explaining an example of a pulse wave source signal segment generated by the segment generation unit 15 for the first measurement area and the third measurement area (hereinafter referred to as a "segment generation diagram"). ). In the segment generation diagram, the luminance value indicated by the pulse wave source signal wi(t) in the first period (indicated by "X" in the segment generation diagram) is shown on the time axis.
In the upper part of the segment generation diagram, the segment generation unit 15 generates the pulse wave source signal for the second period from the pulse wave source signal wi(t) (indicated by "401" on the segment generation diagram) extracted from the first measurement area. A plurality of pulse wave source signal segments (=p1-1, p1-2, p1-3, p1-4, p1-5,...) for the first measurement region are generated by partially extracting wi(t). . A plurality of pulse wave source signal segments (=p3-1, p3-2, p3-3, p3-4, p1-5,..., p3-M) is shown below.
For each measurement region ri(k), the segment generation unit 15 generates a pulse wave source signal wi(t ), M (M is an integer of 2 or more) pulse wave source signal segments are generated by partially extracting the pulse wave source signal wi(t) for the second period.

In addition, each pulse wave source signal segment is generated from the pulse wave source signal wi(t) of which measurement region ri(k) and the pulse wave source signal is generated from which pulse wave source signal wi(t) in which measurement region ri(k) Information indicating whether it is a segment and information indicating the time range on the time axis of the time-series pulse wave source signal wi(t) that is the extraction source are provided. For example, the pulse wave source signal segment p1-1 is the first pulse wave signal segment on the time axis among the plurality of pulse wave source signal segments generated for the pulse wave source signal wi(t) in the first measurement area. This indicates that it is the original signal segment, in other words, the pulse wave original signal segment in the earliest time period. For example, information indicating that the pulse wave source signal segment p1-1 is the pulse wave source signal wi(t) of the first measurement region from 0:00:00 minutes to △ hours, △ minutes, and △ seconds is attached to the pulse wave source signal segment p1-1. has been done.

In the segment generation diagram of FIG. 4, M pulse wave source signal segments for the first measurement area and the third measurement area are illustrated for convenience, but the segment generation unit 15 M pulse wave source signal segments are also generated for each region. That is, the segment generation unit 15 generates M pulse wave source signal segments (=p2-1, p2-2, p2-3, p2-4, p2-5, . . . , p2-M ) and M pulse wave source signal segments (=p4-1, p4-2, p4-3, p4-4, p4-5, . . . , p4-M) for the fourth measurement region are also generated.

As described above, when the difference in distance from the heart between the measurement regions ri(k) is large, a phase difference occurs in the luminance signals extracted from the measurement regions ri(k). On the other hand, when there is almost no difference in distance from the heart between the measurement regions ri(k), no phase difference occurs in the luminance signals extracted from the measurement regions ri(k). It can be said that the second measurement area has almost no difference in distance from the heart from the first measurement area, so the M pulse wave source signal segments for the second measurement area are the same as the M pulse wave source signals for the first measurement area. The signal has the same content as the original signal segment. Furthermore, it can be said that there is almost no difference in distance from the heart between the fourth measurement region and the third measurement region, so the M pulse wave source signal segments for the fourth measurement region are the same as the M pulse wave source signal segments for the third measurement region. The signal has the same content as the pulse wave source signal segment. Note that in the first embodiment, "the same" is not limited to completely matching, but includes "substantially matching" within a permissible range.

After generating a plurality of pulse wave source signal segments for each measurement region ri(k), the segment generation unit 15 generates a trajectory matrix from the generated pulse wave source signal segments, and stores this in a storage unit (not shown). Stack. The storage unit is provided at a location that can be referenced by the pulse wave estimating device 1.
In FIG. 4, there are four measurement regions ri(k), so when the segment generation unit 15 generates M pulse wave source signal segments for each measurement region ri(k), M×4 rows of pulse wave source signal segments are generated. Trajectory matrix (=[p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M], ..., [p4-1, p4-2, p4-3 , p4-4, p4-5, ..., p4-M]) (see the upper right diagram in FIG. 4).

Note that here, the number of measurement regions ri(k) is four, but this is only an example, and N measurement regions ri(k) may be set. The segment generation unit 15 generates a trajectory matrix of M×N rows (=[p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M], ..., [ pN-1, pN-2, pN-3, pN-4, pN-5, ..., pN-M]).

When the segment generation unit 15 stacks the trajectory matrices, it notifies the pulse wave estimation unit 16 of this fact.

When the pulse wave estimation unit 16 is notified by the segment generation unit 15 that the trajectory matrices have been stacked, the pulse wave estimation unit 16 calculates the pulse wave of the subject based on the plurality of pulse wave source signal segments generated for each measurement region ri(k). Estimate.
Specifically, first, the signal separation unit 161 generates a signal (hereinafter referred to as a "separated signal") indicating a plurality of signal components (hereinafter referred to as "principal component") based on a plurality of pulse wave source signal segments. . Specifically, the signal separation unit 161 analyzes a plurality of principal components using a general signal separation technique such as PCA or ICA, and generates a separated signal indicating the analyzed plurality of principal components. By analyzing signal components using general signal separation techniques such as PCA or ICA, the signal separation unit 161 separates components that appear to be pulse wave components and components that appear to be noise components from a plurality of pulse wave source signal segments. To separate.
Note that the signal separation unit 161 generates separated signals indicating a plurality of principal components based on a plurality of pulse wave source signal segments corresponding to all measurement regions ri(k).

Here, FIG. 5 shows the flow in Embodiment 1 until the signal separation unit 161 analyzes a principal component based on a plurality of pulse wave source signal segments and generates a separated signal indicating the analyzed principal component. It is a figure for explaining.
Here, as the measurement area ri(k), there are four measurement areas ri(k) as shown in FIG. 4: the first measurement area, the second measurement area, the third measurement area, and the fourth measurement area. Assume that it is set.
In FIG. 5, a plurality of pulse wave source signal segments are generated for each of the first measurement region, second measurement region, third measurement region, and fourth measurement region shown on the left side of the figure, and a trajectory matrix of M×4 rows is generated. The flow of processing up to stacking is the flow of processing performed by the segment generation unit 15, and has already been explained using FIG. 4, so repeated explanation will be omitted.
The signal separation unit 161 performs principal component analysis on the M×4 row trajectory matrix stacked by the segment generation unit 15 using a known general signal separation technique such as PCA or ICA. As a result, the signal separation unit 161 analyzes C (positive integer) principal components and generates a separated signal indicating the analyzed C principal components. Note that in general signal separation techniques, main components are generated in order from the one with the most information possible. That is, the first principal component c1, the second principal component c2, the third principal component c3, . . . , the C-th principal component cC are arranged in order of increasing information amount.

FIG. 6 is a diagram for explaining an example of details of a separated signal indicating a main component, which is generated by the signal separation unit 161 in the first embodiment.
Note that FIG. 6 shows the separated signals (=[c1, c2, c3, c4, c5, ..., cC]) shown in FIG. 5, in other words, the first measurement area, the second measurement area shown in FIG. 4, The separated signal (=[c1, c2, c3, c4, c5, ..., cC]).
Each separated signal includes a locus matrix of a plurality of pulse wave source signal segments from which the amount of information contained in the principal components indicated by the separated signal is analyzed. In the example shown in FIG. 6, the separated signal (=[c1, c2, c3, c4, c5, ..., cC]) indicating the first principal component c1 to the C-th principal component cC is the first measurement area Pulse wave source signal segments (=p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M) generated from the pulse wave source signal wi(t), Pulse wave source signal segments (=p2-1, p2-2, p2-3, p2-4, p2-5, ..., p2-M ), pulse wave source signal segments (=p3-1, p3-2, p3-3, p3-4, p3-5, ..., p3-M) and pulse wave source signal segments (=p4-1, p4-2, p4-3, p4-4, p4-5) generated from the pulse wave source signal wi(t) of the fourth measurement area , ..., p4-M). Therefore, the separated signals indicating the first principal component c1 to the C-th principal component cC are each formed by a trajectory matrix of M×4 rows (=[p1-1, p1-2, p1-3, p1-4, p1-5 , ..., p1-M], ..., [p4-1, p4-2, p4-3, p4-4, p4-5, ..., p4-M]).

Furthermore, a projection coefficient is assigned to each separated signal (=[c1, c2, c3, c4, c5, . . . , cC]). The projection coefficient is a coefficient assigned according to the information amount of the principal component, and is assigned in the process of analyzing the principal component using a known general signal separation technique.
A projection coefficient is assigned to each pulse wave source signal segment in each measurement region ri(k) for each separated signal. For example, in the example shown in FIG. 6, for the first principal component c1, projection coefficients corresponding to M pulse wave source signal segments in the first measurement region and M pulse wave source signal segments in the second measurement region are used. projection coefficients respectively corresponding to the pulse wave source signal segments; projection coefficients corresponding to each of the M pulse wave source signal segments in the third measurement area; and projection coefficients corresponding to each of the M pulse wave source signal segments in the fourth measurement area. A projection coefficient is given.
For example, in FIG. 6, the projection coefficient (q1, 1-1) given to the separated signal indicating the first principal component c1 is the projection coefficient (q1, 1-1) given to the separated signal indicating the first principal component c1. This indicates that the projection coefficient corresponds to the pulse wave source signal segment. For example, in FIG. 6, the projection coefficient (q1, 4-M) given to the separated signal indicating the first principal component c1 is M on the time axis in the fourth measurement region of the first principal component c1. This indicates that the projection coefficient corresponds to the th pulse wave source signal segment. For example, in FIG. 6, the projection coefficient (qC, 1-1) given to the separated signal indicating the C-th principal component cC is 1 on the time axis in the first measurement region of the C-th principal component cC. This indicates that the projection coefficient corresponds to the th pulse wave source signal segment.

The signal separation unit 161 outputs to the restoration unit 162 the plurality of generated separated signals, specifically, separated signal information Sep(t) regarding the plurality of separated signals indicating the generated plurality of principal components.
The separated signal information Sep(t) includes C separated signals generated from a plurality (M×N) of pulse wave source signal segments generated based on the pulse wave source signal information W(t).

The restoration unit 162 restores the pulse wave source signal wi(t) for each measurement region ri(k) based on the separated signal information Sep(t) output from the signal separation unit 161.
As described above, each separated signal included in the separated signal information Sep(t) includes a plurality of pulse wave sources generated based on each pulse wave source signal wi(t) of each measurement region ri(k). Since the signal segment is included, the restoring unit 162 can restore the pulse wave source signal wi(t) for each measurement region ri(k) from the plurality of separated signals included in the separated signal information Sep(t). .

When restoring the pulse wave source signal wi(t) for each measurement area ri(k), the restoration unit 162 generates a restored pulse wave source indicating the restored pulse wave source signal wi(t) for each measurement area ri(k). Generate signal information RW(t).
The restored pulse wave source signal information RW(t) includes the pulse wave source signal wi(t) (hereinafter referred to as "restored pulse wave source signal") for each restored measurement region ri(k).
The restoration unit 162 outputs the generated restored pulse wave source signal information RW(t) to the estimation unit 163.

The estimation unit 163 estimates the subject's pulse wave based on the restored pulse wave source signal information RW(t) output from the restoration unit 162.
Estimating section 163 outputs pulse wave estimation result P(t), which is pulse wave information indicating the estimated pulse wave, to output section 17 .
The pulse wave information may be, for example, time series data of the subject's pulse wave estimated by the estimation unit 163, the subject's pulse rate, or the subject's pulse interval. Here, in order to simplify the explanation, the pulse wave information is assumed to be the subject's pulse rate (number of pulses per minute).

A method for estimating a subject's pulse wave by the estimation unit 163 will be specifically described.
The estimation unit 163 calculates, for example, the S/N ratio of the restored pulse wave source signal of each measurement region ri(k). The estimation unit 163 weights the restored pulse wave source signal of each measurement region ri(k) based on the S/N ratio obtained, and then calculates the restored pulse wave signal corresponding to each measurement region ri(k). Composite pulse wave signal information D(t) is calculated by summing the original signals. That is, the estimation unit 163 calculates one composite pulse wave signal information D(t) for all measurement regions ri(k). Since weighting is performed based on the S/N ratio, it is assumed that the composite pulse wave signal information D(t) is a signal that is likely to be a pulse wave component with noise components removed.
Then, the estimation unit 163 performs Fourier transform on the composite pulse wave signal information D(t), and calculates the peak frequency in the frequency power spectrum within a predetermined frequency range as the pulse rate. The predetermined frequency range is set in consideration of the range of human heart rates.

The output unit 17 outputs the pulse wave estimation result P(t) output from the pulse wave estimation unit 16 to, for example, a state estimation device.
Note that the function of the output section 17 may be provided in the pulse wave estimating section 16.

The operation of the pulse wave estimating device 1 according to the first embodiment will be explained.
FIG. 7 is a flowchart for explaining the operation of the pulse wave estimating device 1 according to the first embodiment.
For example, when the vehicle is powered on, the pulse wave estimating device 1 repeats the process shown in the flowchart of FIG. 7 until the vehicle is powered off.

The captured image acquisition unit 11 acquires a captured image of the subject from the imaging device 3 (step ST1).
The captured image acquisition unit 11 outputs the acquired captured image to the skin area detection unit 12.

The skin area detection unit 12 detects the subject's skin area from the frame Im(k) included in the captured image acquired by the captured image acquisition unit 11 in step ST1 (step ST2).
The skin area detection unit 12 outputs the generated skin area information S(k) to the measurement area setting unit 13.

The measurement area setting unit 13 uses the frame Im(k) of the captured image acquired by the captured image acquisition unit 11 in step ST1 and the skin area information S(k) outputted by the skin area detection unit 12 in step ST2. Based on the image area corresponding to the skin area indicated by the skin area information S(k) on the frame Im(k), a pulse wave source signal wi(t) indicating a time-series luminance change in the first period is extracted. A plurality of measurement regions ri(k) are set for the measurement (step ST3).
The measurement area setting unit 13 outputs the generated measurement area information R(k) to the pulse wave source signal extraction unit 14.

The pulse wave source signal extraction unit 14 extracts the frame Im(k) of the captured image acquired by the captured image acquisition unit 11 in step ST1 and the measurement area information R(k) output from the measurement area setting unit 13 in step ST3. ), a pulse wave source signal wi(t ) is extracted (step ST4).
The pulse wave source signal extraction unit 14 generates pulse wave source signal information W(t) indicating the pulse wave source signal wi(t) in each measurement region ri(k).
The pulse wave source signal extraction section 14 outputs the generated pulse wave source signal information W(t) to the segment generation section 15.

Based on the pulse wave source signal information W(t) outputted from the pulse wave source signal extractor 14 in step ST4, the segment generation section 15 generates the first segment generation condition for each measurement region ri(k). A plurality of pulse wave source signal segments, which are signals obtained by partially extracting the pulse wave source signal wi(t) for the second period from the pulse wave source signal wi(t) in the period, are generated (step ST5).
After generating a plurality of pulse wave source signal segments for each measurement region ri(k), the segment generation section 15 generates a trajectory matrix from the generated pulse wave source signal segments and stacks this in the storage section.
When the segment generation unit 15 stacks the trajectory matrices, it notifies the pulse wave estimation unit 16 of this fact.

When the pulse wave estimating unit 16 is notified in step ST5 that the trajectory matrices have been stacked from the segment generating unit 15, the pulse wave estimating unit 16 calculates the following based on the plurality of pulse wave source signal segments generated for each measurement region ri(k). Pulse wave estimation processing for estimating the subject's pulse wave is performed (step ST6).
Pulse wave estimation section 16 outputs pulse wave estimation result P(t), which is pulse wave information indicating the estimated pulse wave, to output section 17 .
The output unit 17 outputs the pulse wave estimation result P(t) output from the pulse wave estimation unit 16 to, for example, a state estimation device.

FIG. 8 is a flowchart for explaining details of the pulse wave estimation process by the pulse wave estimation unit 16 in step ST6 of FIG. 7.

The signal separation unit 161 generates separated signals indicating a plurality of principal components based on the plurality of pulse wave source signal segments (step ST11).
The signal separation section 161 outputs the plurality of generated separated signals, specifically, separated signal information Sep(t) regarding the generated separated signals indicating the plurality of generated principal components, to the restoration section 162.

The restoring unit 162 restores the pulse wave source signal wi(t) for each measurement region ri(k) based on the separated signal information Sep(t) output from the signal separating unit 161 in step ST11 (step ST12 ).
The restoring unit 162 outputs the restored pulse wave source signal information RW(t) to the estimating unit 163.

The estimation unit 163 estimates the subject's pulse wave based on the restored pulse wave original signal information RW(t) output from the restoration unit 162 in step ST12 (step ST13).
Estimating section 163 outputs pulse wave estimation result P(t), which is pulse wave information indicating the estimated pulse wave, to output section 17 .
The output unit 17 outputs the pulse wave estimation result P(t) output from the pulse wave estimation unit 16 to, for example, a state estimation device.

In this way, the pulse wave estimation device 1 according to the first embodiment sets a plurality of measurement regions ri(k) in regions corresponding to skin regions on the captured image detected from the captured image of the subject, For each measurement region ri(k), a pulse wave source signal wi(t) is extracted based on the luminance change in a first period, which is a period during which pulse wave estimation is performed once, in the measurement region ri(k). The pulse wave estimating device 1 calculates the pulse wave source in the first period for each measurement area ri(k) based on the pulse wave source signal wi(t) extracted from the measurement area ri(k) and according to the segment generation conditions. A plurality of pulse wave source signal segments are generated by partially extracting the pulse wave source signal wi(t) for the second period from the signal wi(t), and the test subject's pulse wave source signal segment is determined based on the plurality of pulse wave source signal segments. Estimate the pulse wave.
The pulse wave estimating device 1 extracts a pulse wave source signal wi(t) for a second period shorter than the first period based on the pulse wave source signal wi(t) for the first period. By generating a plurality of signal segments, the pulse wave source signal wi(t) can be brought into a state in which there are pairs whose phases are aligned. Then, the pulse wave estimating device 1 estimates the subject's pulse wave based on the pulse wave source signal wi(t) in a state where there is a pair with the same phase, in other words, a plurality of pulse wave source signal segments. Separation of pulse wave components using a known general signal separation technique such as ICA or PCA for a plurality of pulse wave source signals wi(t) can be prevented from failing, and as a result, the pulse wave of the subject can be estimated. In this way, the pulse wave estimating device 1 can estimate the subject's pulse wave based on the luminance signals even if there is a phase difference in the luminance signals extracted from the plurality of measurement regions ri(k). .

9A and 9B are diagrams showing an example of the hardware configuration of the pulse wave estimating device 1 according to the first embodiment.
In the first embodiment, the captured image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave source signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output The functions of the unit 17 are realized by the processing circuit 101. That is, the pulse wave estimating device 1 generates a pulse wave source signal wi(t) indicating the brightness change of a plurality of measurement regions ri(k) set for the subject's skin region in the captured image acquired in the first period. A plurality of pulse wave source signal segments are generated by partially extracting the pulse wave signal, and pulse wave components are separated from the pulse wave source signal segments using techniques such as independent component analysis or principal component analysis. A processing circuit 101 is provided to perform control for estimating the pulse wave of the user.
The processing circuit 101 may be dedicated hardware as shown in FIG. 9A, or may be a processor 104 that executes a program stored in memory as shown in FIG. 9B.

When the processing circuit 101 is dedicated hardware, the processing circuit 101 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Circuit). Gate Array), or a combination of these.

When the processing circuit is the processor 104, the captured image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave source signal extraction unit 14, the segment generation unit 15, and the pulse wave estimation unit 16. , the functions of the output unit 17 are realized by software, firmware, or a combination of software and firmware. Software or firmware is written as a program and stored in memory 105. The processor 104 reads out and executes the program stored in the memory 105, thereby controlling the captured image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave source signal extraction unit 14, and the segment The functions of the generation unit 15, pulse wave estimation unit 16, and output unit 17 are executed. That is, the pulse wave estimating device 1 includes a memory 105 for storing a program that, when executed by the processor 104, results in the execution of steps ST1 to ST6 in FIG. Further, the programs stored in the memory 105 include the captured image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave source signal extraction unit 14, the segment generation unit 15, and the pulse wave estimation unit 12. It can also be said that the computer is caused to execute the processing procedure or method of the unit 16 and the output unit 17. Here, the memory 105 includes, for example, RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically EEPROM). Nonvolatile or volatile such as rasable Programmable Read-Only Memory) This includes semiconductor memory, magnetic disks, flexible disks, optical disks, compact disks, mini disks, DVDs (Digital Versatile Discs), and the like.

Note that the functions of the captured image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave source signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output unit 17 A portion of the information may be realized using dedicated hardware, and a portion may be realized using software or firmware. For example, the functions of the captured image acquisition section 11 and the output section 17 are realized by a processing circuit 101 as dedicated hardware, and the functions of the captured image acquisition section 11 and the output section 17 are realized by a processing circuit 101 as dedicated hardware, and a skin region detection section 12, a measurement region setting section 13, and a pulse wave source signal extraction section 14. The functions of the segment generating section 15 and the pulse wave estimating section 16 can be realized by the processor 104 reading and executing programs stored in the memory 105.
A storage unit (not shown) is configured by, for example, the memory 105.
The pulse wave estimating device 1 also includes a device such as an imaging device 3, and an input interface device 102 and an output interface device 103 that perform wired or wireless communication.

Note that in Embodiment 1 above, the subject was a vehicle driver, but this is only an example. The subject may be a passenger other than the driver of the vehicle.

Further, in the first embodiment described above, the pulse wave estimation device 1 is an in-vehicle device, and includes a captured image acquisition section 11, a skin region detection section 12, a measurement region setting section 13, a pulse wave source signal extraction section 14, The segment generation section 15, the pulse wave estimation section 16, and the output section 17 were included in the on-vehicle device.
The invention is not limited to this, but the captured image acquisition section 11, the skin region detection section 12, the measurement region setting section 13, the pulse wave source signal extraction section 14, the segment generation section 15, the pulse wave estimation section 16, and the output section Of the 17, a part may be installed in an on-vehicle device of a vehicle, and the other part may be provided in a server connected to the on-vehicle device via a network, thereby configuring a system with the on-vehicle device and the server.
In addition, the captured image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave source signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output unit 17 are all It may be provided in the server.

Further, the pulse wave estimating device 1 according to the first embodiment described above is not limited to an in-vehicle device mounted on a vehicle, but can also be applied to, for example, home appliances. Moreover, the test subject is not limited to the occupant of the vehicle, but can be a variety of other people.
To give a specific example, the pulse wave estimating device 1 may be installed in a television set in the living room of a residence. In this case, the subject is a user, such as a resident of the residence. The pulse wave estimating device 1 estimates a user's pulse wave based on a captured image captured by an imaging device installed in a television.

As described above, the pulse wave estimation device 1 according to the first embodiment includes the captured image acquisition unit 11 that acquires a captured image of a person (subject), and the captured image acquisition unit 11 that detects the skin area of the person (subject) from the captured image. The skin area detection unit 12 and a plurality of measurement areas ri(k) for extracting a pulse wave source signal wi(t) indicating a time-series luminance change in the first period are provided in areas corresponding to skin areas on the captured image. ), and a pulse wave generator that extracts a pulse wave source signal wi(t) based on the luminance change in the first period in the measurement region ri(k) for each measurement region ri(k). The original signal extraction unit 14 generates a pulse wave source in the first period based on the pulse wave source signal wi(t) extracted from the measurement area ri(k) for each measurement area ri(k) according to the segment generation conditions. a segment generation unit 15 that generates a plurality of pulse wave source signal segments by partially extracting the pulse wave source signal wi(t) for the second period from the signal wi(t); The pulse wave estimation unit 16 estimates the pulse wave of a person (subject) based on the plurality of generated pulse wave source signal segments. Therefore, the pulse wave estimating device 1 can estimate a person's pulse wave based on the luminance signals even if there is a phase difference in the luminance signals extracted from the plurality of measurement regions ri(k).

Embodiment 2.
In the first embodiment, the length of the second period used to generate the pulse wave source signal segment is uniform.
In Embodiment 2, an embodiment will be described in which the length of the second period used to generate the pulse wave source signal segment is adjusted.
In addition, in the following Embodiment 2, similarly to Embodiment 1, it is assumed that the pulse wave estimating device is installed in a vehicle, and the subject is the driver of the vehicle.

FIG. 10 is a diagram showing a configuration example of a pulse wave estimating device 1a according to the second embodiment.
Regarding the configuration of the pulse wave estimating device 1a according to Embodiment 2, the same components as the pulse wave estimating device 1 described in Embodiment 1 using FIG. do.
The pulse wave estimating device 1a according to the second embodiment differs from the pulse wave estimating device 1 according to the first embodiment in that it includes a parameter setting section 18.

The parameter setting section 18 sets a segment parameter that includes at least information indicating the length of the second period, based on the subject's pulse wave estimated by the pulse wave estimating section 16.
Specifically, the parameter setting unit 18 sets the length of the second period based on the pulse wave of the subject estimated by the pulse wave estimating unit 16 so that the second period corresponds to one period of the subject's pulse wave. A segment parameter including information indicating the calculated length of the second period is set.
Note that the second period does not have to be the same as the time for one cycle of the subject's pulse wave, and the parameter setting unit 18 sets the second period to be longer than the time for one cycle of the subject's pulse wave. Therefore, it is sufficient to calculate the second period.
In addition to the information indicating the length of the second period, the parameter setting unit 18 sets, for example, information regarding the size of partial overlap of the second period as a segment parameter. From the viewpoint of processing load, the parameter setting unit 18 can variably set the amount of overlap between pulse wave source signal segments depending on the length of the pulse wave source signal segments.
The parameter setting section 18 outputs the set segment parameters to the segment generation section 15.

In the second embodiment, the estimation unit 163 outputs the pulse wave estimation result P(t), which is pulse wave information indicating the estimated pulse wave, to the output unit 17 and the parameter setting unit 18.
Furthermore, in the second embodiment, the segment generation unit 15 generates the pulse wave source signal segment using the second period set by the parameter setting unit 18 based on the segment parameters.
Specifically, based on the segment parameters, the segment generation unit 15 sets the length of the second period set by the parameter setting unit 18 to the length of the second period used to generate the pulse wave source signal segment. Above, for each measurement region ri(k), the pulse wave source signal wi(t) for the second period is partially extracted from the pulse wave source signal wi(t) for the first period according to the segment generation conditions. Generate multiple extracted pulse wave source signal segments.

Note that the information on the length of the second period set by the parameter setting unit 18 will be used in the generation of pulse wave source signal segments by the segment generation unit 15 after the parameter setting unit 18 sets the segment parameters. . Therefore, when the segment generation unit 15 generates the pulse wave source signal segment, such as immediately after the pulse wave estimation device 1a starts estimating the subject's pulse wave, there may be cases where the segment parameters are not obtained from the parameter setting unit 18. . In this case, the segment generation unit 15 may generate the pulse wave source signal segment using, for example, the initial value of the second period that is set in advance by the administrator or the like and held by the segment generation unit 15.
Thereafter, if the parameter setting unit 18 has not obtained information for segment parameter setting, in other words, pulse wave estimation results, the parameter setting unit 18 uses the most recently set segment parameters among the previously set segment parameters. Continuously set the segment parameters or reset the second period to the initial value.

The operation of the pulse wave estimating device 1a according to the second embodiment will be explained.
FIG. 11 is a flowchart for explaining the operation of the pulse wave estimating device 1a according to the second embodiment.
For example, when the vehicle is powered on, the pulse wave estimating device 1a repeats the process shown in the flowchart of FIG. 11 until the vehicle is powered off.
In FIG. 11, the specific operations of steps ST21 to ST24 and steps ST28 to ST29 are respectively the same as the specific operations of steps ST1 to ST6 in FIG. 7, which have already been explained in the first embodiment. Since they are similar, redundant explanation will be omitted.

When the pulse wave source signal information W(t) is output from the pulse wave source signal extraction section 14 in step ST24, the segment generation section 15 determines whether the segment parameters have been set by the parameter setting section 18, or in other words, whether the segment parameters have been set by the parameter setting section 18, or in other words, the parameter setting It is determined whether segment parameters have been output from the unit 18 (step ST25).

If the segment generation unit 15 determines that the segment parameters have been set (“YES” in step ST25), the segment generation unit 15 sets the second period set by the parameter setting unit 18 based on the segment parameters. The length is set to the length of the second period used to generate the pulse wave source signal segment (step ST26).

If the segment generation unit 15 determines that the segment parameters have not been set (“NO” in step ST25), the segment generation unit 15 sets the initial value in the second period (step ST27).

When the pulse wave estimating section 16 estimates the subject's pulse wave in step ST29, the parameter setting section 18 sets information indicating the length of the second period based on the subject's pulse wave estimated by the pulse wave estimating section 16. The segment parameters to be included are set (step ST30).

In this way, the pulse wave estimating device 1a calculates the second period based on the subject's pulse wave estimated by the pulse wave estimation unit 16, and sets a segment parameter including information indicating the length of the second period. , and setting a second period according to the set segment parameters. Then, the pulse wave estimating device 1a extracts the pulse wave source signal wi(t) for the second period from among the pulse wave source signals wi(t) in the first period using the second period set according to the segment parameters. A plurality of partially extracted pulse wave source signal segments are generated.
When estimating pulse waves, it is necessary to capture the subject's pulse wave components that repeatedly appear. However, the period of the subject's pulse wave varies depending on the subject's condition or the subject. For example, if a change occurs in the condition of the subject even if it is the same subject, or if there is a change in the subject, the pulse wave estimating device 1a will take one cycle of the subject's pulse wave to estimate. change. For example, if the subject's pulse rate is 60 bpm (beats per minute), the time for one cycle of the subject's pulse wave is 1 second, and if the subject's pulse rate is 100 bpm, the time for one cycle of the subject's pulse wave is 1 second. The time is 0.6 seconds.
As described above, the pulse wave estimating device 1a according to the second embodiment determines the length of the second period used to generate the pulse wave source signal segment for estimating the pulse wave of the subject based on the already estimated pulse wave of the subject. The length can be adjusted in consideration of the wave period. As a result, the pulse wave estimating device 1a can estimate the pulse wave of the subject from the pulse wave source signal segment generated using the second period set without considering the cycle of the subject's pulse wave. The pulse wave of the subject can be estimated with high accuracy.

In the second embodiment described above, once the pulse wave estimation device 1a starts estimating the pulse wave of the subject, it continues to estimate the pulse wave of the subject for a certain period of time, such as repeating the first period several times, and then sets the parameters. Although the unit 18 assumes that the second period is calculated based on the subject's pulse wave estimated by the estimation unit 163 in the previous first period, this is only an example. For example, before the pulse wave estimating device 1a starts estimating the subject's pulse wave, the past pulse wave estimation result P(t) of the subject estimated by the pulse wave estimating device 1a is accumulated in the storage unit. The parameter setting unit 18 may be configured to be able to calculate the second period based on the pulse wave estimation result P(t) stored in the storage unit.
Further, for example, the subject may be able to register information regarding the subject's average pulse wave in the storage unit, and the parameter setting unit 18 may be configured based on the information regarding the subject's average pulse wave registered in the storage unit. It may also be possible to calculate the second period.
For example, the parameter setting unit 18 calculates the second period based on the pulse wave estimation result P(t) obtained by referring to the storage unit or the subject's average pulse wave and the subject's pulse wave estimated by the estimation unit 163. may be calculated.

The hardware configuration of the pulse wave estimating device 1a according to the second embodiment is the same as the hardware configuration of the pulse wave estimating device 1 described using FIGS. 9A and 9B in the first embodiment, so illustration thereof is omitted. do.
In the second embodiment, the captured image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave source signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output The functions of the section 17 and the parameter setting section 18 are realized by the processing circuit 101. That is, the pulse wave estimating device 1a generates a pulse wave source signal wi(t) indicating the luminance change of a plurality of measurement regions ri(k) set for the subject's skin region in the captured image acquired in the first period. A plurality of pulse wave source signal segments are generated by partially extracting the pulse wave signal, and pulse wave components are separated from the pulse wave source signal segments using techniques such as independent component analysis or principal component analysis. A processing circuit 101 is provided to perform control for estimating the pulse wave of the user.

The processing circuit 101 reads out and executes a program stored in the memory 105, thereby controlling the captured image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave source signal extraction unit 14, The functions of the segment generation section 15, pulse wave estimation section 16, output section 17, and parameter setting section 18 are executed. That is, the pulse wave estimating device 1a includes a memory 105 for storing a program that, when executed by the processing circuit 101, results in the execution of steps ST21 to ST30 in FIG. 11 described above. Further, the programs stored in the memory 105 include the captured image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave source signal extraction unit 14, the segment generation unit 15, and the pulse wave estimation unit 12. It can also be said that the computer is caused to execute the processing procedure or method of the section 16, the output section 17, and the parameter setting section 18.
A storage unit (not shown) is configured by, for example, the memory 105.
The pulse wave estimating device 1a also includes a device such as the imaging device 3, and an input interface device 102 and an output interface device 103 that perform wired or wireless communication.

Note that in the second embodiment above, the test subject is a vehicle driver, but this is only an example. The subject may be a passenger other than the driver of the vehicle.

Further, in the second embodiment described above, the pulse wave estimation device 1a is an in-vehicle device, and includes a captured image acquisition section 11, a skin region detection section 12, a measurement region setting section 13, a pulse wave source signal extraction section 14, The segment generation section 15, the pulse wave estimation section 16, the output section 17, and the parameter setting section 18 were included in the on-vehicle device.
The invention is not limited to this, but the captured image acquisition section 11, the skin region detection section 12, the measurement region setting section 13, the pulse wave source signal extraction section 14, the segment generation section 15, the pulse wave estimation section 16, and the output section 17 and the parameter setting unit 18, part of which is installed in the on-vehicle device of the vehicle, and the other part is provided in a server connected to the in-vehicle device via a network, so that the in-vehicle device and the server constitute a system. It's okay.
Further, a captured image acquisition section 11, a skin region detection section 12, a measurement region setting section 13, a pulse wave source signal extraction section 14, a segment generation section 15, a pulse wave estimation section 16, an output section 17, The entire parameter setting unit 18 may be provided in the server.

Further, the pulse wave estimating device 1a according to the second embodiment described above is not limited to an in-vehicle device mounted on a vehicle, but can also be applied to, for example, home appliances. Moreover, the test subject is not limited to the occupant of the vehicle, but can be a variety of other people.

As described above, according to the second embodiment, the pulse wave estimating device 1a has the configuration of the pulse wave estimating device 1 according to the first embodiment, and also includes the pulse wave estimating device 1a that has the pulse wave estimating device 1a according to the first embodiment. The segment generation unit 15 includes a parameter setting unit 18 that calculates a second period based on the wave and sets a segment parameter including information indicating the length of the second period, and a segment generation unit 15 that sets the second period according to the segment parameter. It was configured as follows. Therefore, the pulse wave estimating device 1a is more accurate than when estimating a person's pulse wave from the pulse wave source signal segment generated using the second period set without considering the period of the person's pulse wave. A person's pulse wave can be estimated.

Specifically, in the pulse wave estimating device 1a, the parameter setting unit 18 sets the second period so that the second period corresponds to one cycle of the pulse wave of the person (subject) estimated by the pulse wave estimating unit 16. is calculated, and a segment parameter including information indicating the length of the second period is set. Then, the segment generation unit 15 sets the second period according to the segment parameters. Therefore, the pulse wave estimating device 1a is more accurate than when estimating a person's pulse wave from the pulse wave source signal segment generated using the second period set without considering the period of the person's pulse wave. A person's pulse wave can be estimated.

Embodiment 3.
In the first embodiment, the pulse wave estimating device uses the plurality of generated pulse wave source signal segments as they are for estimating the pulse wave of the subject.
In Embodiment 3, an embodiment will be described in which a plurality of pulse wave source signal segments are weighted and a pulse wave of a subject is estimated.
In addition, in the following Embodiment 3, similarly to Embodiment 1, it is assumed that the pulse wave estimating device is installed in a vehicle, and the subject is a driver of the vehicle.

FIG. 12 is a diagram showing a configuration example of a pulse wave estimating device 1b according to the third embodiment.
Regarding the configuration of the pulse wave estimating device 1b according to Embodiment 3, the same components as the pulse wave estimating device 1 described in Embodiment 1 using FIG. do.
The pulse wave estimating device 1b according to the third embodiment differs from the pulse wave estimating device 1 according to the first embodiment in that it includes a weighting coefficient calculating section 19.

The weighting coefficient calculation unit 19 calculates a weighting coefficient (hereinafter referred to as “segment weight”) for each of the plurality of pulse wave source signal segments generated by the segment generation unit 15 for each measurement region ri(k).
In detail, the weighting coefficient calculation unit 19 calculates the weighting factor for each of the plurality of pulse wave source signal segments based on the magnitude of signal fluctuation in the pulse wave source signal wi(t) from which the pulse wave source signal segment is extracted. Calculate segment weights.
In the third embodiment, the segment generation unit 15 generates information on a plurality of generated pulse wave source signal segments of each measurement region ri(k) and the pulse wave source signal generated by the pulse wave source signal extraction unit 14. The information W(t) is output to the weighting coefficient calculating section 19.

For example, the weighting coefficient calculation unit 19 calculates a preset threshold value (hereinafter referred to as "variation determination threshold value") for the time-series pulse wave source signal wi(t) of the first period from which the pulse wave source signal segment is extracted. ) or not. When it is determined that the pulse wave source signal wi(t) has fluctuated by more than the fluctuation determination threshold, the weighting coefficient calculation unit 19 calculates the relevant pulse wave source signal wi(t) from the time series pulse wave source signal wi(t) of the first period. A segment weight of "0" is calculated for the pulse wave source signal segment whose extraction source is the pulse wave source signal wi(t) of the portion where the fluctuation occurred. It can be said that a signal in a portion of the pulse wave source signal wi(t) that fluctuates by more than the fluctuation determination threshold is a signal that contains a lot of noise. The weighting coefficient calculation unit 19 sets a segment weight of "1" to the pulse wave source signal segment whose extraction source is the pulse wave source signal wi(t) in a portion other than the portion where the fluctuation occurs.

On the other hand, when the weighting coefficient calculation unit 19 determines that there is no fluctuation greater than the fluctuation determination threshold in the time-series pulse wave source signal wi(t) of the first period from which the pulse wave source signal segment is extracted. , a segment weight of "1" is calculated for all the pulse wave source signal segments whose extraction source is the time-series pulse wave source signal wi(t) of the first period in which there is no variation.

The weighting coefficient calculation unit 19 receives information regarding segment weights (hereinafter referred to as “segment weight information”) calculated for each of the plurality of pulse wave source signal segments generated by the segment generation unit 15 for each measurement region ri(k). , is output to the pulse wave estimation section 16.
The segment weight information is information in which information that allows the measurement region ri(k) to be specified, information that allows the identification of the pulse wave source signal segment, and the segment weight of the pulse wave source signal segment are associated with each other.

The segment weight calculated by the weighting coefficient calculation unit 19 is used by the pulse wave estimation unit 16 to determine whether or not to use the pulse wave source signal segment generated by the segment generation unit 15 when estimating the subject's pulse wave. This will serve as an indicator.
The pulse wave estimating unit 16 does not use the pulse wave source signal segment to which segment weight "0" is assigned for estimating the pulse wave of the subject. Specifically, the signal separation unit 161 of the pulse wave estimation unit 16 discards the pulse wave source signal segment to which segment weight “0” has been assigned, and performs PCA based on the remaining plurality of pulse wave source signal segments. Alternatively, a common signal separation technique such as ICA is used to analyze the plurality of principal components and generate a separated signal indicative of the analyzed plurality of principal components.
In this way, in the third embodiment, the pulse wave estimation unit 16 estimates the subject's pulse wave based on the pulse wave source signal segment and the segment weight.

Here, FIG. 13 is a diagram for explaining pulse wave source signal segments used by the pulse wave estimating device 1b to estimate the pulse wave of the subject in the third embodiment.
FIG. 13 shows the luminance values indicated by the time-series pulse wave source signal wi(t) of the first period extracted from a certain measurement region ri(k) among the plurality of measurement regions ri(k) on the time axis. The figure is shown in .
In FIG. 13, among the time-series pulse wave source signals wi(t) of the first period, the pulse wave source signals wi(t) in the portion surrounded by the dotted line have large signal fluctuations.
The weighting factor calculation unit 19 extracts the pulse wave source signal wi(t) of the portion surrounded by the dotted line from among the plurality of pulse wave source signal segments generated by the segment generation unit 15 for a certain measurement region ri(k). The segment weight for the original pulse wave source signal segment is calculated as "0".
The weighting coefficient calculation unit 19 calculates the pulse wave source signal wi(t) other than the portion surrounded by the dotted line among the plurality of pulse wave source signal segments generated by the segment generation unit 15 for a certain measurement region ri(k). The segment weight for the pulse wave source signal segment to be extracted is calculated as "1".
As a result, the pulse wave estimation unit 16 uses the pulse wave source signal segment whose extraction source is the pulse wave source signal wi(t) of the part surrounded by the dotted line and whose segment weight is "0" to estimate the subject's pulse wave. Not used.

The operation of the pulse wave estimating device 1b according to the third embodiment will be explained.
FIG. 14 is a flowchart for explaining the operation of the pulse wave estimating device 1b according to the third embodiment.
For example, when the vehicle is powered on, the pulse wave estimating device 1b repeats the process shown in the flowchart of FIG. 14 until the vehicle is powered off.
In FIG. 14, the specific operations of steps ST31 to ST35 are the same as the specific operations of steps ST1 to ST5 in FIG. 7, which have already been explained in Embodiment 1. omitted.

The weighting coefficient calculating unit 19 calculates segment weights for each of the plurality of pulse wave source signal segments generated by the segment generating unit 15 in step ST35 (step ST36).
In detail, the weighting coefficient calculation unit 19 calculates the weighting factor for each of the plurality of pulse wave source signal segments based on the magnitude of signal fluctuation in the pulse wave source signal wi(t) from which the pulse wave source signal segment is extracted. Calculate segment weights.
The weighting coefficient calculating section 19 outputs the segment weighting information to the pulse wave estimating section 16.

When the pulse wave estimating unit 16 is notified in step ST35 that the trajectory matrices have been stacked from the segment generating unit 15, the pulse wave estimating unit 16 calculates the following based on the plurality of pulse wave source signal segments generated for each measurement region ri(k). Perform pulse wave estimation processing to estimate the subject's pulse wave. At this time, the pulse wave estimation unit 16 determines, based on the segment weight information output from the weighting coefficient calculation unit 19, that the pulse wave source signal segments to which segment weight “0” is assigned are used for estimating the subject's pulse wave. (step ST37). Note that the pulse wave estimating unit 16 may receive a notification that the trajectory matrices have been stacked, for example, via the weighting coefficient calculating unit 19.
Specifically, when generating the separated signal (see step ST11 in FIG. 8), the signal separation unit 161 discards pulse wave source signal segments with a segment weight of "0" and removes the remaining pulse wave source signal segments. Based on the segments, multiple principal components are analyzed using common signal separation techniques, such as PCA or ICA, to generate separated signals indicative of the analyzed multiple principal components.

In this way, the pulse wave estimating device 1b calculates segment weights for each of a plurality of pulse wave source signal segments, and estimates the subject's pulse wave based on the pulse wave source signal segments and the segment weights. Specifically, the pulse wave estimating device 1b determines whether or not each of the plurality of pulse wave source signal segments is determined based on the magnitude of signal fluctuation in the pulse wave source signal wi(t) from which the pulse wave source signal segment is extracted. Calculate segment weights.
As a result, when the pulse wave source signal wi(t) extracted from the plurality of measurement regions ri(k) is a signal containing many noise components, the pulse wave estimating device 1b can remove the subject's pulse wave can be estimated. Therefore, the pulse wave estimating device 1b can improve the accuracy of the test for the subject compared to the case where it does not take into consideration whether or not the pulse wave source signal wi(t) extracted from the plurality of measurement regions ri(k) is a signal containing many noise components. It is possible to estimate the pulse wave of

The hardware configuration of pulse wave estimating device 1b according to Embodiment 3 is the same as the hardware configuration of pulse wave estimating device 1 described using FIGS. 9A and 9B in Embodiment 1, so illustration is omitted. do.
In the third embodiment, the captured image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave source signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output The functions of the section 17 and the weighting coefficient calculation section 19 are realized by the processing circuit 101. That is, the pulse wave estimating device 1b generates a pulse wave source signal wi(t) indicating the luminance change of a plurality of measurement regions ri(k) set for the subject's skin region in the captured image acquired in the first period. A plurality of pulse wave source signal segments are generated by partially extracting the pulse wave signal, and pulse wave components are separated from the pulse wave source signal segments using techniques such as independent component analysis or principal component analysis. A processing circuit 101 is provided to perform control for estimating the pulse wave of the user.

The processing circuit 101 reads out and executes a program stored in the memory 105, thereby controlling the captured image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave source signal extraction unit 14, The functions of the segment generation section 15, pulse wave estimation section 16, output section 17, and weighting coefficient calculation section 19 are executed. That is, the pulse wave estimating device 1b includes a memory 105 for storing a program that, when executed by the processing circuit 101, results in the execution of steps ST31 to ST37 in FIG. 14 described above. Further, the programs stored in the memory 105 include the captured image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave source signal extraction unit 14, the segment generation unit 15, and the pulse wave estimation unit 12. It can also be said that the computer is caused to execute the processing procedure or method of the unit 16, the output unit 17, and the weighting coefficient calculation unit 19.
A storage unit (not shown) is configured by, for example, the memory 105.
The pulse wave estimating device 1b also includes a device such as the imaging device 3, and an input interface device 102 and an output interface device 103 that perform wired or wireless communication.

Note that in the third embodiment above, the subject was a vehicle driver, but this is only an example. The subject may be a passenger other than the driver of the vehicle.

Further, in the third embodiment described above, the pulse wave estimation device 1b is an in-vehicle device, and includes a captured image acquisition section 11, a skin region detection section 12, a measurement region setting section 13, a pulse wave source signal extraction section 14, The segment generation section 15, the pulse wave estimation section 16, the output section 17, and the weighting coefficient calculation section 19 were included in the vehicle-mounted device.
The invention is not limited to this, but the captured image acquisition section 11, the skin region detection section 12, the measurement region setting section 13, the pulse wave source signal extraction section 14, the segment generation section 15, the pulse wave estimation section 16, and the output section 17 and the weighting coefficient calculation unit 19, a part of which is installed in an on-vehicle device of a vehicle, and the other part is provided in a server connected to the in-vehicle device via a network, and the in-vehicle device and the server constitute a system. You may.
Further, a captured image acquisition section 11, a skin region detection section 12, a measurement region setting section 13, a pulse wave source signal extraction section 14, a segment generation section 15, a pulse wave estimation section 16, an output section 17, All of the weighting coefficient calculation units 19 may be included in the server.

Further, the pulse wave estimating device 1b according to the third embodiment described above is not limited to an in-vehicle device mounted on a vehicle, but can also be applied to, for example, home appliances. Moreover, the test subject is not limited to the occupant of the vehicle, but can be a variety of other people.

As described above, according to the third embodiment, the pulse wave estimating device 1b has a weighting coefficient (segment weight) for each of a plurality of pulse wave source signal segments in addition to the configuration of the pulse wave estimating device 1 according to the first embodiment ), and the pulse wave estimation section 16 was configured to estimate the pulse wave of a person (subject) based on the pulse wave source signal segment and the weighting coefficient (segment weight). Therefore, the pulse wave estimating device 1b can be used more accurately than when it does not take into consideration whether or not the pulse wave source signal wi(t) extracted from the plurality of measurement regions ri(k) is a signal containing many noise components. It is possible to estimate the pulse wave of

Specifically, in the pulse wave estimating device 1b, the weighting coefficient calculation unit 19 calculates a plurality of A weighting coefficient (segment weight) for each pulse wave source signal segment is calculated. Therefore, the pulse wave estimating device 1b can be used more accurately than when it does not take into consideration whether or not the pulse wave source signal wi(t) extracted from the plurality of measurement regions ri(k) is a signal containing many noise components. It is possible to estimate the pulse wave of

In the first to third embodiments described above, the pulse wave estimating devices 1, 1a, and 1b apply weighting coefficients ( (hereinafter referred to as "component weight"), and restores the pulse wave source signal wi(t) for each measurement region ri(k) based on a plurality of separated signals and the set component weight for each separated signal. May have.

FIG. 15 shows how the pulse wave estimating devices 1, 1a, and 1b, in Embodiments 1 to 3, calculate components for a plurality of separated signals indicating a plurality of principal components estimated using a known general signal separation technique. In the case of having a function of setting weights and restoring the pulse wave source signal wi(t) for each measurement region ri(k) based on a plurality of separated signals and the set component weight for each separated signal, It is a figure showing an example of composition of pulse wave estimating part 16a.
When the pulse wave estimating devices 1, 1a, 1b have the above functions, the pulse wave estimating devices 1, 1a, 1b explained using FIG. 1, FIG. 10, or FIG. 12 in the above embodiments 1 to 3. In the configuration example shown in FIG. 1, a pulse wave estimating section 16a as shown in FIG. 15 is provided instead of the pulse wave estimating section 16.

The restoration section 162 of the pulse wave estimation section 16a includes a component weight setting section 1621 that sets component weights for each separated signal output from the signal separation section 161.
The restoration unit 162 generates the pulse wave source signal wi for each measurement region ri(k) based on the plurality of separated signals output from the signal separation unit 161 and the component weight for each separated signal set by the component weight setting unit 1621. (t) is restored.
The estimating unit 163 estimates the subject's pulse wave based on the generated pulse wave original signal wi(t) that is restored by the restoring unit 162 based on the plurality of separated signals and the component weights for each separated signal.

Some examples of how the component weight setting unit 1621 sets component weights will be described.

The component weight setting unit 1621 sets a component weight for each separated signal, for example, based on the similarity of frequency characteristics between a plurality of separated signals.
Specifically, the component weight setting unit 1621 first performs fast Fourier transform or the like on each separated signal, and calculates the peak frequency in the frequency power spectrum of each separated signal.
Next, the component weight setting unit 1621 creates a pair of adjacent separated signals for the plurality of separated signals indicating the plurality of principal components, and calculates the difference in peak frequencies of the adjacent separated signals (hereinafter referred to as "peak difference"). calculate. Note that "adjacent principal components" refer to separated signals that are arranged in order of increasing amount of information when the separated signals are arranged in descending order of the amount of information they contain. Then, the component weight setting unit 1621 determines whether the calculated peak difference is larger than a preset threshold (hereinafter referred to as "peak difference determination threshold"). When the peak difference is larger than the threshold for peak difference determination, the component weight setting unit 1621 sets the component weight of the separated signal for which the peak difference is calculated to "0", and sets the component weight of the separated signal for which the peak difference is calculated to be "0", If it is below the threshold, the component weight is set to "1".

When the component weight is "0", it can be said that the similarity in frequency characteristics between the separated signals is low, and when the component weight is "1", it can be said that the similarity in the frequency characteristics between the separated signals is high. Since the pulse wave component has the characteristic of appearing repeatedly at a certain period, the high similarity of the frequency characteristics means that the separated signal with the frequency characteristics contains a large amount of the pulse wave component.In other words, It is assumed that this is a separated signal that appears to be a pulse wave component. The component weight setting unit 1621 assigns a large component weight to the separated signal that appears to be a pulse wave component, thereby allowing the estimation unit 163 to estimate the subject's pulse wave based on the separated signal that appears to be a pulse wave component.

To give a specific example, for example, the plurality of separated signals output from the signal separation unit 161 are the first principal components c1, . Assume that there are C separated signals indicating the C-th principal component cC. Here, the separated signal indicating the first principal component c1 is the first separated signal, the separated signal indicating the second principal component c2 is the second separated signal, the separated signal indicating the third principal component c3 is the third separated signal, and the separated signal indicating the third principal component c3 is the third separated signal. A separated signal indicating the principal component c4 is a fourth separated signal, a separated signal indicating the fifth principal component c5 is a fifth separated signal, ..., a separated signal indicating the (C-1)th principal component c (C-1) Let be the (C-1)th separated signal, and let the signal indicating the C-th principal component cC be the C-th separated signal.
In this case, the component weight setting unit 1621 creates a pair of the first separated signal and the second separated signal, calculates the peak difference between the peak frequency of the first separated signal and the peak frequency of the second separated signal, and calculates the peak frequency of the calculated peak frequency. It is determined whether the difference is larger than a peak difference determination threshold. The component weight setting unit 1621 sets the component weights of both the first separated signal and the second separated signal to "0" when the peak difference is larger than the peak difference judgment threshold, and when the peak difference is less than or equal to the peak difference judgment threshold. In this case, the component weights of the first separated signal and the second separated signal are both set to "1".
Further, the component weight setting unit 1621 creates a pair of the third separated signal and the fourth separated signal, calculates the peak difference between the peak frequency of the third separated signal and the peak frequency of the fourth separated signal, and calculates the peak difference between the peak frequency of the third separated signal and the peak frequency of the fourth separated signal. It is determined whether or not is larger than a peak difference determination threshold. The component weight setting unit 1621 sets the component weights of both the third separated signal and the fourth separated signal to "0" when the peak difference is larger than the peak difference judgment threshold, and when the peak difference is less than or equal to the peak difference judgment threshold. In this case, the component weights of the third separated signal and the fourth separated signal are both set to "1".
The component weight setting unit 1621 sets a pair of a fifth separated signal and a sixth separated signal, a pair of a seventh separated signal and an eighth separated signal, a pair of a (C-1)th separated signal and a C-th separated signal, etc. , the above-described processing is also performed, and component weights are set for all separated signals.

Among the plurality of separated signals output from the signal separation section 161, the restoration section 162 discards the separated signals for which the component weight setting section 1621 has set the component weight "0", and the component weight setting section 1621 has set the component weight "0". The pulse wave source signal wi(t) for each measurement region ri(k) is restored based only on the separated signal set to ``1''.
Then, the estimating unit 163 estimates the subject's pulse wave based on the pulse wave original signal wi(t) that the restoring unit 162 restored based only on the separated signal with the component weight “1”. Since the estimation unit 163 can estimate the subject's pulse wave based on the separated signal that is likely to be a pulse wave component, it is possible to improve the estimation accuracy of the subject's pulse wave.

Note that in the above-described specific example, the component weight setting unit 1621 creates a pair of adjacent separated signals and calculates the peak difference between the peak frequencies of the adjacent separated signals, but this is only an example. For example, the component weight setting unit 1621 may calculate the differences between all pairs and use the weighted average value as the component weight.
For example, the component weight setting unit 1621 calculates the difference in peak frequency of a certain separated signal (to be referred to as a reference separated signal) among the C separated signals and the other (C-1) separated signals. , the weighted average value of the calculated (C-1) differences is set as the difference value of the reference separated signal. Then, the component weight setting unit 1621 sets the component weight for the reference separated signal to "0" if the difference value of the reference separated signal is larger than the threshold value, and sets the component weight to the reference separated signal to "1" if it is less than or equal to the threshold value. The component weight setting unit 1621 performs the above processing using all separated signals as reference separated signals. The component weight setting unit 1621 may set a value corresponding to the magnitude of the weighted average value as the component weight.

The component weight setting unit 1621 calculates, for example, a correlation coefficient of projection coefficients between a plurality of separated signals based on a projection coefficient given to the separated signals when the plurality of separated signals are generated, and calculates the correlation coefficient of the projection coefficients between the plurality of separated signals. The component weight for each separated signal may be set based on the above.

Specifically, the component weight setting unit 1621 first assigns, for each measurement region ri(k), to a plurality of pulse wave source signal segments corresponding to the measurement region ri(k) for each separated signal. Information expressing projection coefficients in time series (hereinafter referred to as "projection coefficient time series information") is generated. The component weight setting unit 1621 generates projection coefficient time series information for each separated signal for the number of measurement regions ri(k) (N pieces). That is, assuming that there are C separated signals, the component weight setting unit 1621 generates C×N projection coefficient time series information.
Next, the component weight setting unit 1621 determines the correlation of the generated projection coefficient time series information for each separated signal, and calculates the correlation coefficient. Specifically, the component weight setting unit 1621 creates all pairs of N pieces of projection coefficient time series information generated in a certain separated signal, and sets the correlation coefficient for each pair of projection coefficient time series information. Calculate each. Note that the correlation coefficient indicates the degree of phase matching of the projection coefficient time series information. The component weight setting unit 1621 sets the average value of the correlation coefficients calculated for each pair as the component weight of the separated signal.

Here, the pulse wave estimating devices 1, 1a, and 1b assume that signal components that are in phase are likely pulse wave components, and that the time-series changes in projection coefficients are similar between measurement regions ri(k). In other words, if the phases of the projection coefficient time series information are aligned between measurement regions ri(k), such projection coefficients for each pulse wave source signal segment of each measurement region ri(k) are It is assumed that the applied separated signal is a separated signal that contains many pulse wave components and is likely to be a pulse wave component. The component weight setting unit 1621 assigns a large weighting coefficient to the separated signal that appears to be a pulse wave component, thereby allowing the estimation unit 163 to estimate the subject's pulse wave based on the separated signal that appears to be a pulse wave component.

To give a specific example, for example, the plurality of separated signals output from the signal separation unit 161 are the first principal components c1, . Assume that there are C separated signals indicating the C-th principal component cC. Here, the separated signal indicating the first principal component c1 is the first separated signal, the separated signal indicating the second principal component c2 is the second separated signal, the separated signal indicating the third principal component c3 is the third separated signal, and the separated signal indicating the third principal component c3 is the third separated signal. A separated signal indicating the principal component c4 is a fourth separated signal, a separated signal indicating the fifth principal component c5 is a fifth separated signal, ..., a separated signal indicating the (C-1)th principal component c (C-1) Let be the (C-1)th separated signal, and let the separated signal showing the C-th principal component cC be the C-th separated signal.
In this case, the component weight setting unit 1621 first projects each measurement region ri(k) (first measurement region, second measurement region, third measurement region, and fourth measurement region) for each separated signal. Generate coefficient time series information.

Here, FIG. 16 is a diagram illustrating an example of projection coefficient time series information generated by the component weight setting unit 1621.
FIG. 16 shows, as an example, projection coefficient time series information generated by the component weight setting unit 1621 for the first separated signal. The component weight setting unit 1621 generates projection coefficient time series information for each of the first measurement area, the second measurement area, the third measurement area, and the fourth measurement area for the first separated signal.
Although FIG. 16 shows projection coefficient time series information generated for the first separated signal as an example, the component weight setting unit 1621 also sets the measurement area ri for the second to C-th separated signals. (k) of projection coefficient time series information is generated.

Next, for example, the component weight setting unit 1621 first sets the projection coefficient time series information of the first measurement region (indicated by 1601 in FIG. 16), the projection coefficient time series information of the second measurement region ( 1602 in FIG. 16), projection coefficient time series information of the third measurement region (indicated by 1603 in FIG. 16), and projection coefficient time series information of the fourth measurement region (indicated by 1604 in FIG. 16). Create all pairs for .
Here, the component weight setting unit 1621 includes a pair (referred to as a first pair) of projection coefficient time series information of the first measurement region and projection coefficient time series information of the second measurement region, a projection coefficient time series of the first measurement region A pair of information and projection coefficient time series information of the third measurement area (referred to as the second pair), a pair of projection coefficient time series information of the first measurement area and projection coefficient time series information of the fourth measurement area (referred to as the third pair) ), a pair of projection coefficient time series information of the second measurement area and projection coefficient time series information of the third measurement area (referred to as the fourth pair), a pair of projection coefficient time series information of the second measurement area and projection coefficient time series information of the fourth measurement area A pair of projection coefficient time series information (referred to as a fifth pair) and a pair of projection coefficient time series information of the third measurement area and projection coefficient time series information of the fourth measurement area (referred to as a sixth pair) are created.
The component weight setting unit 1621 then sets the first pair of correlation coefficients, the second pair of correlation coefficients, the third pair of correlation coefficients, the fourth pair of correlation coefficients, the fifth pair of correlation coefficients, and , the correlation coefficient of the sixth pair is calculated. The component weight setting unit 1621 calculates the calculated correlation coefficient of the first pair, correlation coefficient of the second pair, correlation coefficient of the third pair, correlation coefficient of the fourth pair, correlation coefficient of the fifth pair, The average value of the correlation coefficients of the fifth pair is set as the component weight of the first separated signal indicating the first principal component c1.

The component weight setting unit 1621 similarly sets the projection coefficient time series information of the first measurement region, the projection coefficient time series information of the second measurement region, and the projection coefficient time series information of the third measurement region for the second to C-th separation signals. All pairs of the coefficient time series information and the projection coefficient time series information of the fourth measurement area are created, and the correlation is determined for each created pair to calculate the correlation coefficient. Then, the component weight setting unit 1621 sets the calculated average value of the correlation coefficients for each pair as the component weight.

The restoration unit 162 weights the plurality of separated signals output from the signal separation unit 161 according to the component weights set by the component weight setting unit 1621, and generates the pulse wave source signal wi for each measurement region ri(k). (t) is restored.
For example, the restoring unit 162 performs (component weight of first separated signal x [p1-1, p1-2, ..., p1-M] + component weight of second separated signal x [p1-1, p1-2 , ..., p1-M] + component weight of the third separated signal x [p1-1, p1-2, ..., p1-M] + ... + component weight of the C-th separated signal x [p1 -1, p1-2, . . . , p1-M]) is the restored pulse wave source signal (t) of the first measurement region.

Then, the estimation unit 163 estimates the subject's pulse wave based on the pulse wave original signal wi(t) restored by the restoration unit 162 by weighting according to the component weights. Since the estimation unit 163 can estimate the subject's pulse wave based on the separated signal that is likely to be a pulse wave component, it is possible to improve the estimation accuracy of the subject's pulse wave.

Note that in the above-described specific example, the component weight setting unit 1621 creates all pairs of projection coefficient time series information for each separated signal, but this is only an example. For example, the component weight setting unit 1621 may create a pair using two pieces of projection coefficient time series information among a plurality of pieces of projection coefficient time series information. In the above-mentioned specific example, the component weight setting unit 1621, for example, sets a pair of projection coefficient time series information of the first measurement region and projection coefficient time series information of the second measurement region (referred to as the seventh pair), a third Two pairs may be created: the projection coefficient time series information of the measurement area and the projection coefficient time series information of the fourth measurement area (referred to as the eighth pair). In this case, the component weight setting unit 1621 calculates the correlation coefficient of the seventh pair and the correlation coefficient of the eighth pair, and calculates the average value of the correlation coefficient of the sixth pair and the correlation coefficient of the seventh pair. Let be the component weight of the first separated signal indicating the first principal component c1.

Furthermore, in the above-described specific example, the component weight setting unit 1621 uses the average value of the calculated correlation coefficients as the component weight of the separated signal, but this is only an example. For example, the component weight setting unit 1621 may set the calculated variance value of the correlation coefficient as the component weight of the separated signal.

In the first to third embodiments described above, the pulse wave estimation devices 1, 1a, and 1b apply component weights to a plurality of separated signals indicating a plurality of principal components estimated using a known general signal separation technique. The pulse wave in the case where the function is configured to restore the pulse wave source signal wi(t) for each measurement region ri(k) based on a plurality of separated signals and the component weight for each separated signal set. The operation of the estimator 16a will be explained using FIG. 17.
In Embodiments 1 to 3, the pulse wave estimation devices 1, 1a, and 1b set component weights for a plurality of separated signals indicating a plurality of principal components estimated using a known general signal separation technique. , the first embodiment to The details of the pulse wave estimation processing in step ST6 in FIG. 7, step ST29 in FIG. 11, and step ST37 in FIG.

Note that the specific operations of step ST11 and step ST13 in FIG. 17 are the same as the specific operations of step ST11 and step ST13 in FIG. 8, which have already been explained, so a redundant explanation will be omitted.

The component weight setting section 1621 of the pulse wave estimation section 16a sets the component weight for each separated signal output from the signal separation section 161 in step ST11 (step ST12-1).

The restoring unit 162 restores the measurement region ri based on the plurality of separated signals output from the signal separating unit 161 in step ST1 and the component weight for each separated signal set by the component weight setting unit 1621 in step ST12-1. (k) pulse wave source signal wi(t) is restored (step ST12-2).

In addition, in step ST13, the estimating unit 163 calculates the pulse wave of the subject based on the generated pulse wave source signal wi(t) restored by the restoring unit 162 based on the plurality of separated signals and the component weight for each separated signal. will be estimated.

In the pulse wave estimating device 1 according to the first embodiment, the pulse wave estimating device 1a according to the second embodiment, or the pulse wave estimating device 1b according to the third embodiment, the restoring unit 162a calculates the weighting coefficient for each separated signal. The restoring unit 162a includes a component weight setting unit 1621 that sets a component weight (component weight), and a restoring unit 162a performs a pulse wave element for each measurement region ri(k) based on a plurality of separated signals and a weighting coefficient (component weight) for each separated signal. By restoring the signal wi(t), the pulse wave estimating devices 1, 1a, and 1b can estimate the subject's pulse wave based on the separated signal that seems to be a pulse wave component, thereby improving the accuracy of estimating the subject's pulse wave. be able to.

Note that it is possible to freely combine each embodiment, to modify any component of each embodiment, or to omit any component in each embodiment.

Hereinafter, various aspects of the present disclosure will be collectively described as supplementary notes.

(Additional note 1)
a captured image acquisition unit that acquires a captured image of a person;
a skin area detection unit that detects the skin area of the person from the captured image;
a measurement area setting unit that sets a plurality of measurement areas for extracting a pulse wave source signal indicating a time-series luminance change in a first period in an area corresponding to the skin area on the captured image;
a pulse wave source signal extracting unit that extracts the pulse wave source signal for each measurement area based on the luminance change in the first period in the measurement area;
For each measurement area, based on the pulse wave source signal extracted from the measurement area and according to the segment generation conditions, the pulse wave source signal for the second period is selected from among the pulse wave source signals in the first period. a segment generation unit that generates a plurality of pulse wave source signal segments that are partially extracted signals;
A pulse wave estimation device comprising: a pulse wave estimation unit that estimates a pulse wave of the person based on a plurality of the pulse wave source signal segments generated for each of the measurement regions.
(Additional note 2)
The conditions for generating the segment are:
the second period is a shorter period than the first period;
Regarding the plurality of pulse wave source signal segments in a certain measurement region, a certain pulse wave source signal segment is a signal that does not completely match with other pulse wave source signal segments on the time axis;
Regarding the plurality of pulse wave source signal segments in a certain measurement region, a certain pulse wave source signal segment is a signal that partially overlaps with another pulse wave source signal segment on the time axis;
In each measurement region, the length at which a certain pulse wave source signal segment partially overlaps with another pulse wave source signal segment on the time axis is the same;
The pulse wave estimating device according to supplementary note 1, further comprising: the plurality of pulse wave source signal segments in each measurement region have the same length on the time axis.
(Additional note 3)
a parameter setting unit that calculates the second period based on the pulse wave of the person estimated by the pulse wave estimation unit and sets a segment parameter that includes at least information indicating the length of the second period;
The pulse wave estimating device according to

Supplementary note

1 or 2, wherein the segment generation unit sets the second period according to the segment parameter.
(Additional note 4)
Supplementary note 3, wherein the parameter setting unit calculates the second period so that the second period corresponds to one period of the pulse wave of the person estimated by the pulse wave estimating unit. The pulse wave estimation device described.
(Appendix 5)
comprising a weighting coefficient calculation unit that calculates a weighting coefficient for each of the plurality of pulse wave source signal segments,
The pulse wave according to any one of appendices 1 to 4, wherein the pulse wave estimation unit estimates the pulse wave of the person based on the pulse wave source signal segment and the weighting coefficient. Wave estimation device.
(Appendix 6)
The weighting coefficient calculation unit calculates the weighting coefficient for each of the plurality of pulse wave source signal segments based on the magnitude of signal fluctuation in the pulse wave source signal from which the pulse wave source signal segment is extracted. The pulse wave estimating device according to appendix 5, characterized in that:
(Appendix 7)
The pulse wave estimator includes:
a signal separation unit that generates a separation signal indicating a plurality of signal components based on the plurality of pulse wave source signal segments;
a restoring unit that restores the pulse wave source signal for each of the measurement regions based on the plurality of separated signals;
The pulse wave according to any one of appendices 1 to 6, further comprising: an estimation unit that estimates the pulse wave of the person based on the restored pulse wave source signal for each of the measurement regions. Wave estimation device.
(Appendix 8)
The restoration unit is
comprising a component weight setting unit that sets a weighting coefficient for each of the separated signals,
The pulse wave estimating device according to appendix 7, wherein the pulse wave source signal for each measurement area is restored based on the plurality of separated signals and the weighting coefficient for each separated signal.
(Appendix 9)
The component weight setting section includes:
The pulse wave estimating device according to appendix 8, wherein the weighting coefficient for each of the separated signals is set based on similarity of frequency characteristics between the plurality of separated signals.
(Appendix 10)
The component weight setting section includes:
A correlation coefficient of the projection coefficients between the plurality of separated signals is calculated based on a projection coefficient given to the separated signals when the plurality of separated signals are generated, and the correlation coefficient of the projection coefficients is calculated between the plurality of separated signals. The pulse wave estimating device according to appendix 8, wherein the weighting coefficient is set for each pulse wave estimation device.
(Appendix 11)
a step in which the captured image acquisition unit acquires a captured image of a person;
a skin area detection unit detecting the skin area of the person from the captured image;
a step in which the measurement area setting unit sets a plurality of measurement areas for extracting a pulse wave source signal indicating a time-series luminance change in a first period in an area corresponding to the skin area on the captured image;
a step in which the pulse wave source signal extracting unit extracts the pulse wave source signal for each measurement area based on the brightness change in the first period in the measurement area;
A segment generation unit generates a pulse wave source signal for a second period from among the pulse wave source signals in the first period based on the pulse wave source signal extracted from the measurement area and according to segment generation conditions for each measurement area. generating a plurality of pulse wave source signal segments that are signals obtained by partially extracting the pulse wave source signal;
A pulse wave estimating method, comprising: a pulse wave estimating unit estimating a pulse wave of the person based on a plurality of the pulse wave source signal segments generated for each of the measurement regions.

The pulse wave estimating device according to the present disclosure can estimate a person's pulse wave based on the luminance signals even if there is a phase difference in the luminance signals extracted from a plurality of measurement regions.

1, 1a, 1b pulse wave estimation device, 11 captured image acquisition unit, 12 skin area detection unit, 13 measurement area setting unit, 14 pulse wave source signal extraction unit, 15 segment generation unit, 16, 16a pulse wave estimation unit, 161 Signal separation unit, 162, 162a Restoration unit, 1621 Component weight setting unit, 163 Estimation unit, 17 Output unit, 18 Parameter setting unit, 19 Weighting coefficient calculation unit, 3 Imaging device, 101 Processing circuit, 102 Input interface device, 103 Output Interface device, 104 processor, 105 memory.

Claims

a captured image acquisition unit that acquires a captured image of a person;
a skin area detection unit that detects the skin area of the person from the captured image;
a measurement area setting unit that sets a plurality of measurement areas for extracting a pulse wave source signal indicating a time-series luminance change in a first period in an area corresponding to the skin area on the captured image;
a pulse wave source signal extracting unit that extracts the pulse wave source signal for each measurement area based on the luminance change in the first period in the measurement area;
For each measurement area, based on the pulse wave source signal extracted from the measurement area and according to the segment generation conditions, the pulse wave source signal for the second period is selected from among the pulse wave source signals in the first period. a segment generation unit that generates a plurality of pulse wave source signal segments that are partially extracted signals;
A pulse wave estimation device comprising: a pulse wave estimation unit that estimates a pulse wave of the person based on a plurality of the pulse wave source signal segments generated for each of the measurement regions.
The conditions for generating the segment are:
the second period is a shorter period than the first period;
Regarding the plurality of pulse wave source signal segments in a certain measurement region, a certain pulse wave source signal segment is a signal that does not completely match with other pulse wave source signal segments on the time axis;
Regarding the plurality of pulse wave source signal segments in a certain measurement region, a certain pulse wave source signal segment is a signal that partially overlaps with another pulse wave source signal segment on the time axis;
In each measurement region, the length at which a certain pulse wave source signal segment partially overlaps with another pulse wave source signal segment on the time axis is the same;
The pulse wave estimating device according to claim 1, further comprising: the plurality of pulse wave source signal segments in each measurement region have the same length on the time axis.
a parameter setting unit that calculates the second period based on the pulse wave of the person estimated by the pulse wave estimation unit and sets a segment parameter that includes at least information indicating the length of the second period;
The pulse wave estimating device according to claim 1, wherein the segment generation unit sets the second period according to the segment parameter.
The parameter setting unit calculates the second period so that the second period corresponds to one period of the pulse wave of the person estimated by the pulse wave estimating unit. 3. The pulse wave estimation device according to 3.
comprising a weighting coefficient calculation unit that calculates a weighting coefficient for each of the plurality of pulse wave source signal segments,
The pulse wave estimating device according to claim 1, wherein the pulse wave estimation unit estimates the pulse wave of the person based on the pulse wave source signal segment and the weighting coefficient.
The weighting coefficient calculation unit calculates the weighting coefficient for each of the plurality of pulse wave source signal segments based on the magnitude of signal fluctuation in the pulse wave source signal from which the pulse wave source signal segment is extracted. The pulse wave estimating device according to claim 5, characterized in that:
The pulse wave estimator includes:
a signal separation unit that generates a separation signal indicating a plurality of signal components based on the plurality of pulse wave source signal segments;
a restoring unit that restores the pulse wave source signal for each of the measurement regions based on the plurality of separated signals;
The pulse wave estimating device according to claim 1, further comprising: an estimation unit that estimates the pulse wave of the person based on the restored pulse wave source signal for each of the measurement regions.
The restoration unit is
comprising a component weight setting unit that sets a weighting coefficient for each of the separated signals,
The pulse wave estimating device according to claim 7, wherein the pulse wave source signal for each measurement area is restored based on the plurality of separated signals and the weighting coefficient for each separated signal.
The component weight setting section includes:
The pulse wave estimating device according to claim 8, wherein the weighting coefficient for each of the separated signals is set based on similarity of frequency characteristics between the plurality of separated signals.
The component weight setting section includes:
A correlation coefficient of the projection coefficients between the plurality of separated signals is calculated based on a projection coefficient given to the separated signals when the plurality of separated signals are generated, and the correlation coefficient of the projection coefficients is calculated between the plurality of separated signals. The pulse wave estimating device according to claim 8, wherein the weighting coefficient is set for each pulse wave estimation device.
a step in which the captured image acquisition unit acquires a captured image of a person;
a skin area detection unit detecting the skin area of the person from the captured image;
a step in which the measurement area setting unit sets a plurality of measurement areas for extracting a pulse wave source signal indicating a time-series luminance change in a first period in an area corresponding to the skin area on the captured image;
a step in which the pulse wave source signal extracting unit extracts the pulse wave source signal for each measurement area based on the brightness change in the first period in the measurement area;
A segment generation unit generates a pulse wave source signal for a second period from among the pulse wave source signals in the first period based on the pulse wave source signal extracted from the measurement area and according to segment generation conditions for each measurement area. generating a plurality of pulse wave source signal segments that are signals obtained by partially extracting the pulse wave source signal;
A pulse wave estimating method, comprising: a pulse wave estimating unit estimating a pulse wave of the person based on a plurality of the pulse wave source signal segments generated for each of the measurement regions.