JP2012239661A - Heart rate/respiration rate detection apparatus, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a heart rate/respiration rate detection apparatus that accurately detects the heart rate and respiration rate from image data including a face.SOLUTION: In the heart rate/respiration rate detection apparatus 1, an optical wavelength component obtaining part 11 obtains RGB components in the image data 2, and a face recognition processing part 12 extracts a face region from the image data 2. An RGB average calculation part 13 calculates an average of the RGB components in the face region, and an independent component analysis part 14 separates components of three independent signals from the average of the RGB components. An independent signal sorting part 15 determines spectrum distribution for each independent signal of the sampled face region, pairs the spectrum distributions of independent signals with high similarity, based on the similarity of waveform of the spectrum distribution, and partitions the spectrum distributions. A heart rate/respiration rate detection part 16 traces the peak frequency of each partitioned spectrum distribution, and estimates the heart rate based on the frequency value of the peak frequency whose positions of appearance converge on a frequency value axis after the lapse of a predetermined time.

Description

  The present invention relates to a technique for detecting a heart rate and a respiration rate without contact with a human body.

  A method of detecting a heart rate in a non-contact manner from a face image taken with a camera is known. For example, the neck or face is photographed with a camera, the image density of the set area is calculated continuously, the time response waveform of the density is calculated by the FFT (Fast Fourier Transform) processing, and the maximum value of the spectrum is calculated as the respiratory rate or A method for setting the heart rate is known. In addition, after calculating each average value of RGB in the face image area and processing by independent component analysis (ICA), the heart rate is calculated from the peak frequency obtained by frequency analysis of one component waveform (second component waveform). An estimation method is known.

JP-A-2005-218507

M.M. Z. Poh, D .; J. et al. McDuff, and R.M. W. Picard, “Non-contact, Automated Cardiac Pulse Measurements Using Video Imaging and Blind Source Separation”, Opt Express, vol. 18, pp. 10762-10774, May 7, 2010

  However, it is known that the conventional technology has the following problems.

  (1) When the frequency component analysis is performed using the RGB time response waveform of the color image, the frequency component indicating the heartbeat signal may not be detected.

  (2) When independent component analysis is applied to an RGB waveform, it is unclear which calculated component contains a heartbeat signal.

  (3) In the spectrum distribution obtained by frequency analysis, it was not possible to automatically identify which peak frequency was caused by the heartbeat signal.

  In view of the above problems, the present invention is intended to detect the heart rate / respiration rate without contact with the human body, and in particular, from the spectral distribution of the time series signal extracted from the image data, the peak caused by the heart rate signal or the respiration signal. The purpose is to realize a device that automatically detects heart rate and respiration rate accurately by specifying the frequency.

  An apparatus disclosed as one embodiment of the present invention is an apparatus that detects a heart rate or a respiration rate from image data, and 1) obtains time-series data of a plurality of light wavelength components from image data obtained by imaging a detection target person. An optical wavelength component acquisition unit that performs 2) an average value calculation unit that calculates an average value of each optical wavelength component based on time-series data of the optical wavelength component, and 3) an independent component analysis is performed on the calculated average value. An independent component analyzing unit that obtains a plurality of independent signals by application, and 4) an independent signal sorting unit that rearranges and classifies the obtained plurality of independent signals so that the characteristics of each independent signal are consistently maintained. And 5) identifying an independent signal including a peak frequency at which a fluctuation of a frequency value falls within a certain range after a certain period of time from among a plurality of peak frequencies included in the spectrum distribution of the segmented independent signal. Heart rate signal or And a heart rate, respiratory rate detection unit for an independent signal including the peak frequency due to No. 吸信.

  According to the disclosed apparatus, the peak frequency caused by the heartbeat signal and the respiratory signal is identified from the time response waveform of the optical wavelength component of the image data obtained by imaging a human face and the heart rate and the respiratory rate are automatically and accurately detected. Can be detected.

It is a figure which shows the structural example in one Example of the heart rate and respiration rate detection apparatus disclosed as 1 aspect of this invention. It is a figure which shows the example of a face area | region extracted by face recognition processing in one Example. It is a figure which shows the example which applies an independent component analysis to the RGB average value of a face area | region in one Example. It is a figure which shows the example of the result (independent signal) of the independent component analysis based on the optical wavelength component at the time of the (n-1) th sampling at the time of sampling continuously in one Example, and its spectrum distribution. It is a figure which shows the example of the result (independent signal) of the independent component analysis based on the optical wavelength component at the time of the nth sampling at the time of sampling continuously in one Example, and its spectrum distribution. It is a figure which shows the concept of the overlapping rate in one Example. It is a figure which shows the example of segmentation of the spectrum distribution of the independent signal in one Example. It is a figure for demonstrating the detection process of the frequency of the heart rate signal by the particle filter in one Example. It is a figure which shows the concept of the particle | grain arrangement | positioning of the particle filter by the likelihood initial value function in one Example. It is a processing flowchart of the heart rate estimation process of the heart rate / respiration rate detecting device in one embodiment. It is a processing flowchart of the likelihood initial value function setting process of the heart rate / respiration rate detecting device in one embodiment.

  Hereinafter, a heart rate / respiration rate detection device disclosed as one embodiment of the present invention will be described.

  FIG. 1 is a diagram illustrating a configuration example in an embodiment of a heart rate / respiration rate detection device disclosed as one aspect of the present invention.

  The heart rate / respiration rate detection device 1 identifies a heartbeat signal or a respiration signal as a signal source by using time-series data of each light wavelength component of the image data 2 including the face of the subject, and the subject Automatically detect heart rate or respiration rate. Therefore, the heart rate / respiration rate detection apparatus 1 includes an optical wavelength component acquisition unit 11, a face recognition processing unit 12, an RGB average value calculation unit 13, an independent component analysis (ICA) unit 14, an independent signal sorting unit 15, a heart rate / A respiration rate detection unit 16, a likelihood initial value function storage unit 17, and a likelihood function setting unit 18 are provided.

  The light wavelength component acquisition unit 11 is a means for inputting image data 2 taken by a person to be detected with a camera or the like and acquiring time series data of a plurality of light wavelength components of the image data 2.

  The optical wavelength component acquisition unit 11 receives image data 2 output from a video camera or the like. The device for inputting the image data 2 may be other than the camera, and the image data 2 may be image data cut out from media such as video or DVD or a moving image file stored on the hard disk.

  The image data 2 is given, for example, as a rectangle of 320 horizontal pixels × 240 vertical pixels. One pixel is given by a brightness gradation value (luminance). For example, the gradation value (I) of a pixel at coordinates (i, j) indicated by integers i and j is an 8-bit digital value. I (i, j) or the like. When the image data 2 is a color image, one pixel is given by gradation values of a red component (R), a green component (G), and a blue component (B), and is represented by integers i and j, for example. The gradation values of the red component (R), green component (G), and blue component (B) of the pixel at the coordinate (i, j) are digital values R (i, j) and G (i, j), respectively. , B (i, j), etc.

  Note that the gradation value of the image data 2 may be a combination of RGB or another color system (HSV color system, YUV color system, etc.) obtained by converting RGB values.

  The face recognition processing unit 12 is a unit that detects the position and size of a human face included in the image data 2 acquired by the light wavelength component acquisition unit 11 and extracts a face region including the face.

  The face recognition processing unit 12 executes a known face recognition process and is not particularly limited to one face recognition process. In known face recognition processing, generally, the position and size of a face included in image data are detected, and the detected face area is extracted as a rectangle.

  FIG. 2 is a diagram illustrating an example of a face area extracted by the face recognition process.

A rectangle extracted as a face area from the image data 2 is expressed by, for example, four vertex coordinates as shown in FIG. For example, the face area is represented as {(i ₁ , j ₁ ), (i ₁ , j ₂ ), (i ₂ , j ₁ ), (i ₂ , j ₂ )}.

The RGB average value calculation unit 13 is a means for calculating an average value for each color component of the red component (R), green component (G), and blue component (B) of the face area detected from the image data.
The RGB average value calculation unit 13 calculates the average value using, for example, the following formulas (1) to (3).

In Expressions (1) to (3), A is a face area, N represents the number of pixels in the face area, and the values of A are i ₁ ≦ i <i ₂ and j ₁ ≦ j <j ₂ . Yes, the value of N is (i ₂ −i ₁ ) × (j ₂ −j ₁ ).

  The independent component analysis unit 14 is a means for obtaining an independent signal by applying independent component analysis (ICA) to the time response waveform of the RGB average value obtained by the RGB average value calculation unit 13.

  The independent component analysis performed by the independent component analysis unit 14 is a method of multivariate analysis, and it is assumed that the signal serving as the information source is independent, and the signal source is separated from the signals of a plurality of observation values. It is a calculation method that separates and extracts. In the independent component analysis, for example, in the blind signal separation method, it is formulated as follows.

N signal sources at time t s (t) = [s ₁ (t), s ₂ (t),..., S _n (t)] ^T is unknown. It is also assumed that the observed value x (t) is generated from the signal source s (t) using the m × n unknown mixing matrix A. That is,
x (t) = [x ₁ (t), x ₂ (t),..., x _m (t)] ^T
x (t) = As (t)
And When the observed value series x (1), x (2),... Are observed at time t = 1, 2,..., The signal source series s (1), s (2),. .

  That is, in the independent component analysis, the independence of the signal source is assumed, and the separation signal y is obtained using the estimated value A ′ of the mixing matrix. The estimated separation signal y ′ is given by the following equation (4).

In Equation (4), if the matrix A ′ can be correctly estimated, A ′ ⁻¹ A becomes a unit matrix and therefore coincides with the signal source s.

  FIG. 3 is a diagram illustrating an example in which the independent component analysis is applied to the RGB average values of the face area extracted from the image data.

  Each of R, G, and B data shown in FIG. 3 (A) includes time series data of average values of three light wavelength components of the face area detected from the image data, that is, R component, G component, and B component. Show.

  FIG. 3B shows three pieces obtained when independent component analysis is applied using time series data of average values of three components (R component, G component, and B component) shown in FIG. This is an example of the independent signal. Each data of ICA1, ICA2, and ICA3 shown in FIG. 3 (B) includes the first component, the second component, and the third component of the independent signal corresponding to the time-series data of the optical wavelength components shown in FIG. 3 (A). Time series data. In the example shown in FIG. 3B, it is assumed that the heartbeat signal is separated into the third component (ICA3) of the independent signal.

  The independent signal sorting unit 15 arranges the independent signals separated by the independent component analysis so that their features are consistent so that the heartbeat signal or respiratory signal to be detected is always included in the same independent signal. It is a means of segmentation instead.

  Independent signal analysis has the property that the order in which the separated independent signals appear is indefinite. That is, it cannot be specified which of the plurality of components (first to third components) separated by the independent component analysis is the independent signal estimated to contain the most heartbeat signal. Therefore, the independent signal sorting unit 15 performs spectrum analysis on each component (first component to third component) of the independent signal obtained by the independent component analysis to obtain a spectral distribution, and the waveform similarity between the spectral distributions of the respective components. Based on the above, a process of rearranging the spectrum distribution of the separated independent signals is performed so that the components of the independent signals including a specific signal (heartbeat signal) always belong to the same group.

  FIG. 4 and FIG. 5 are diagrams showing examples of independent component analysis results (independent signals) based on optical wavelength components at the (n−1) th and nth sampling times when sampling is performed continuously and their spectral distributions. It is.

The data shown in FIG. 4A includes independent signals ICA1 _n-1 and ICA2 _n-1 separated by application of independent component analysis to the (n-1) th (sampling preceding the current sampling) optical wavelength component. , ICA3 _n-1 . Figure 4 the data shown in (B) is independent signal _{ICA1 n-1, ICA2 n-} 1, ICA3 n-1 of each spectral distribution _{SD-ICA1 n-1, SD} -ICA2 n-1, SD-ICA3 n-1 It is. Similarly, the data shown in FIG. 5A are independent signal examples ICA1 _n , ICA2 _n , and ICA3 _n separated by applying independent component analysis to the nth (current sampling) optical wavelength component. The data shown in FIG. 5B are spectral distributions SD-ICA1 _n , SD-ICA2 _n , and SD-ICA3 _n of the independent signals ICA1 _n , ICA2 _n , and ICA3 _n, respectively.

  The independent signal sorting unit 15 extracts consecutively sampled independent signals, obtains the spectral distribution of the independent signal at the current sampling and the previous sampling, and obtains the waveform similarity between the obtained spectral distributions. .

The waveform similarity is calculated based on the overlapping rate R _pq by the following equation (5).

In equation (5),
Sp (n−1) is the spectral distribution of the pth component at the (n−1) th sampling,
Sq (n) is the spectral distribution of the q-th component at the n-th sampling,
max () is a function that outputs a large value in parentheses (),
min () represents a function that outputs a small value in parentheses ().

  FIG. 6 is a diagram showing the concept of the above overlapping rate.

As shown in FIG. 6, the overlap rate R _pq is given by the area of the portion where the two waveforms overlap. If the shapes are similar, the value is close to 1.

The independent signal sorting unit 15 calculates an overlapping rate R _pq between the spectral distribution waveform of each component obtained by the (n−1) th sampling and the spectral distribution waveform of each component obtained by the nth sampling, and the overlapping rate. The spectral distributions having a high R _pq value are paired, and the spectral distributions at the n-th sampling are rearranged.

  FIG. 7 is a diagram illustrating an example of segmentation of the spectrum distribution of independent signals.

FIG. 7A is a diagram showing an output example of the spectral distributions SD-ICA1 _n-1 , SD-ICA2 _n-1 , SD-ICA3 _n-1 of each component obtained by the (n-1) th sampling, FIG. 7B is a diagram illustrating an output example of the spectral distributions SD-ICA1 _n , SD-ICA2 _n , SD-ICA3 _n of each component obtained by the n-th sampling.

As shown in FIGS. 7A and 7B, a certain independent signal appears in the third component (ICA3 _n−1 ) in the independent component analysis of the optical wavelength component by the ( _n−1 ) th sampling, and the nth sampling. In the independent component analysis of the light wavelength component by, there may be cases where it appears in the second component (ICA2 _n ). Therefore, since the heartbeat signal is separated into the third component (ICA3 _n-1 ) in the ( _n-1 ) th sampling and separated as the second component (ICA2 _n ) in the nth sampling, a specific independent signal ( Here, when it is desired to handle the heart rate signal), it has not been possible to automatically determine what number the independent signal output should be focused on.

  The independent signal sorting unit 15 obtains the spectrum distribution of each independent signal, calculates the waveform similarity of the obtained spectrum distribution, and rearranges the results of the independent component analysis by associating the similar ones with each other, and appears as a waveform. The spectral distribution is divided so that the characteristics of the independent signals are maintained consistently.

Independent signal sorting unit 15, for example, the spectral distribution SD-ICA1 _n-1 component ICA1 _n-1 obtained in the (n-1) -th sampling, spectral distribution SD-ICA1 _n of each component obtained by the n-th sampling calculates the overlap ratio _{R pq} the _{SD-ICA2 n, SD-ICA3} n respectively, identifying the spectral distribution as the highest value. Here, it is assumed that the overlap rate R _pq with the SD-ICA3 _n is the highest value for the spectral distribution SD-ICA1 _n-1 . The independent signal sorting unit 15 pairs the spectrum distributions SD-ICA1 _n-1 and SD-ICA3 _n . Similarly, the independent signal sorting unit 15 also identifies and pairs the spectral distribution at the n-th sampling time when the overlap rate R _pq is high for SD-ICA2 _n-1 and SD-ICA3 _n-1 .

Further, the independent signal sorting unit 15 rearranges the order of the spectrum distribution at the n-th sampling based on the pairing. As a result, as shown in FIG. 7C, the arrangement of the spectral distributions of the components obtained by the nth sampling is rearranged in the order of SD-ICA3 _n , SD-ICA1 _n , SD-ICA2 _n .

Therefore, as shown in FIG. 7C, for a specific independent signal (heartbeat signal), the third component (ICA3 _n-1 ) at the ( _n-1 ) th sampling and the second component (ICA3 _n-1 ) at the nth sampling ( ICA2 _n ) can be identified.

  The heart rate / respiration rate detection unit 16 is a means for detecting the heart rate or respiration rate by specifying the peak frequency resulting from the heart rate or respiration from the spectrum distribution of the independent signal.

  When the processing for obtaining the independent signal component from the face region extracted from the image data 2 is repeated for a certain period of time to obtain the spectral distribution of the independent signal, the peak frequency of the heartbeat signal varies in magnitude and frequency value. However, it appears at almost the same position on the frequency axis. On the other hand, the peak frequency caused by noise repeatedly appears and disappears with time.

  Therefore, the heart rate / respiration rate detection unit 16 tracks the transition of the peak frequency of the spectrum distribution with the passage of time by the particle filter, and the peak frequency having the highest likelihood of appearing in an appropriate time range is detected by the heartbeat signal. Presume that there is.

  The particle filter is one of the methods for estimating the parameters by changing the particle distribution according to the likelihood (= fitness). As an example, the Monte Carlo method is used to obtain an optimal solution. The particle filter algorithm consists of particle placement, movement prediction, weighting, and state estimation. In the particle arrangement, the particles are arranged in a predetermined range, and in the particle movement prediction, the particles are moved based on the equation of state. In weighting, weighting is performed by calculating the likelihood (applicability) of particles based on the current observation and particle state. That is, weighting is the stochastic likelihood maximization and particle distribution indicates the probability density. In state estimation, the current state is estimated based on the particle distribution.

  The heart rate / respiration rate detection unit 16 detects the peak frequency from the spectrum distribution of each independent signal separated from the face region by the particle filter, and the fluctuation of the peak frequency on the frequency value axis is constant after a certain period of time. Only detection processing for spectral distributions including independent signals that converge at positions within the range (spectral distributions where the peak frequency falls within a certain frequency value) continues, and detection processing for spectral distributions of other independent signals To stop.

Specifically, the heart rate / respiration rate detection unit 16 calculates the maximum value f _Dmax of the peak frequency difference between the n-th (current) sampling and the m-th sampling before a certain time by the following equation (6). Based on the minimum value f _Dmin , the component of the independent signal in which the frequency value of the peak frequency of the component is converged is determined.

f _Dmax = max (f _n−m , f _{n−m + 1} ,..., f _n )
f _Dmin = min (f _n−m , f _{n−m + 1} ,..., f _n )
| F _Dmax −f _Dmin | <F _th ...... Expression (6)
In equation (6),
max () is a function that outputs a large value in parentheses (),
min () is a function that outputs a small value in parentheses (),
F _th represents a threshold value.

  Thereby, the heart rate / respiration rate detection unit 16 can specify an independent component including a signal of interest (here, a heart rate signal) from among a plurality of independent components separated by the independent component analysis unit 14. In addition, the heart rate can be estimated by detecting the heartbeat signal from the spectral distribution of the specified independent component by the processing described later.

  FIG. 8 is a diagram for explaining the processing for detecting the frequency of the heartbeat signal by the particle filter.

  FIG. 8A is a diagram showing a spectrum distribution of a certain independent component obtained at the (n-1) th sampling. In the spectrum distribution shown in FIG. 8A, the heart rate / respiration rate detection unit 16 detects the peak frequency (frequency of the heartbeat signal) in the section of interest u, and the detected peak frequency is the frequency of the heartbeat signal. Estimated.

  FIG. 8B is a diagram illustrating a particle arrangement example of the particle filter. When the peak frequency is detected in the spectrum distribution shown in FIG. 8A, the heart rate / respiration rate detection unit 16 has already arranged the particles of the particle filter as shown in the upper part of FIG. 8B. Suppose there is.

  FIG. 8C shows a spectral distribution in the nth sampling. The heart rate / respiration rate detection unit 16 detects the peak frequency (frequency of the heartbeat signal) from the spectrum distribution in the nth sampling shown in FIG.

  As shown in the upper part of FIG. 8B, the heart rate / respiration rate detection unit 16 selects one particle of the particle filter arranged at the peak frequency position of the spectrum distribution in the (n-1) th sampling. , And determine a section of interest u centered on the position of the particle on the frequency axis (see FIG. 8A). Next, as shown in the lower part of FIG. 8B, the heart rate / respiration rate detection unit 16 determines a particle transition destination according to a normal random number.

  The heart rate / respiration rate detection unit 16 sets a cut-out section v centered on the position of the transition destination particle on the spectrum distribution of the nth sampling. Then, the heart rate / respiration rate detection unit 16 includes a section of interest u determined from the peak frequency in the spectral distribution in the (n−1) th sampling shown in FIG. 8 (A) and the nth th shown in FIG. 8 (C). Spectral waveforms included in the segmented section v determined from the peak frequency in the spectral distribution in sampling are extracted as shown in FIG. 8D, and the similarity between the extracted waveforms is calculated.

The extracted waveform similarity calculation method performs the same processing as the similarity calculation method executed by the independent signal sorting unit 15. The heart rate / respiration rate detection unit 16 uses the obtained similarity as the likelihood of the transition destination particle. The likelihood ^(a) Ruv of the ^a-th particle is calculated by the following equation (7).

The heart rate / respiration rate detection unit 16 repeats the same processing for all particles (number A), and obtains the likelihood ^(a) Ruv for all particles.

Then, the heart rate / respiration rate detection unit 16 normalizes the sum of the likelihoods of all the particles to be 1, and converts the likelihood ^(a) Ruv of each particle into a weight ^(a) w. The weighted average F _t is obtained from the position and weight on the frequency axis of each particle by the following equation (8).

In equation (8),
^(A) w is the weight of the a-th particle,
^(A) f is the frequency (heart rate) of the transition destination of the a-th particle,
f _min is the lower limit of the heart rate,
f _max represents the upper limit of the heart rate.

Heart rate and respiratory rate detection unit 16, the average F _t with weights determined by the equation (8), the estimated value of the frequency of the heartbeat signal at the n-th (current) sampling.

  Finally, the heart rate / respiration rate detection unit 16 determines the particle arrangement of the particle filter used at the (n + 1) th (next) sampling. Using the particle weight and the total particle number A, the particle number is given by the following equation (9).

^(A) n = N · ^(a) w ...... Formula (9)
In equation (9), the heart rate / respiration rate detection unit 16 increases the number of particles using the above equation in descending order of the likelihood of particles. Alternatively, the heart rate / respiration rate detection unit 16 repeats this process until the total number of particles becomes A. If the number of obtained particles is A or less, the particles are detected until the number becomes A. generate.

  By the above particle filter processing, at the time of the (n + 1) th (next) sampling, many particles of the particle filter are arranged near the current peak frequency (frequency of the heartbeat signal). Therefore, even in the next sampling, the particle filter particles transition from the vicinity of the heartbeat signal frequency where a large number of particles are present, so that the heart rate / respiration rate detection unit 16 can track the frequency of interest with high accuracy. it can.

  The likelihood initial value function storage unit 17 is a storage unit that holds the likelihood initial value function of the particle filter of the heart rate / respiration rate detection unit 16 in association with the user to be detected or the imaging time zone of the image data. is there. In this example, the likelihood initial value function storage unit 17 is, for example, a first storage unit or a second storage unit, and is implemented as a likelihood initial value function table.

  The likelihood function setting unit 18 selects a likelihood initial value function from the likelihood initial value function storage unit 17 based on the user or the imaging time zone captured in the image data to be detected and uses it as a particle filter. It is a means for setting.

  When the likelihood initial value function is set for each user by the likelihood initial value function storage unit 17, the identification information of the user and the feature information of the face area are stored in association with each other in advance. It shall be.

  The likelihood initial value function storage unit 17 is provided, and the detection accuracy when the peak frequency of the heartbeat signal is specified is improved by giving the particle filter a likelihood initial value function corresponding to the user to be detected or the imaging time zone. Can be made.

  FIG. 9 is a diagram showing a concept of particle arrangement of the particle filter by the likelihood initial value function.

  FIG. 9A is a diagram showing the concept of particle arrangement of the particle filter when the likelihood initial value function is not particularly specified.

  Since the position where the peak frequency appears in the spectrum distribution is unknown at the start of processing, the heart rate / respiration rate detection unit 16 uses the particle frequency as the initial state of the particle filter processing as shown in FIG. Distribute uniformly on the axis.

  FIG. 9B is a diagram conceptually showing the likelihood initial value function storage unit 17 when setting the likelihood initial value function for each user.

  The likelihood initial value function storage unit 17 stores user identification information and the initial value of the likelihood function set for the user in association with each other.

When the likelihood initial value function is set for each user to be detected, the face recognition processing unit 12
A face area is detected from the input image data 2, and a user (identification information) as a detection target is specified from the face area. The likelihood function setting unit 18 selects a likelihood initial value function corresponding to the identified user identification information from the likelihood initial value function storage unit 17 and determines a likelihood function to be given to the particle filter.

  FIG. 9C is a diagram conceptually illustrating the likelihood initial value function storage unit 17 in the case where the likelihood initial value function is set for each user and for each time period.

  The likelihood initial value function storage unit 17 shown in FIG. 9C stores the initial value of the likelihood function in association with the user identification information and the time zone. Here, it is divided into three time zones of morning, afternoon and night, but is not limited to this example, and may be divided into finer time zones.

  When the likelihood initial value function is set for each user to be detected and each time zone when the image data is captured, the face recognition processing unit 12 identifies the user (identification) based on the face area detected from the image data 2. Information) is specified, and the likelihood function setting unit 18 determines the time zone as follows.

  The likelihood function setting unit 18 determines the time zone based on the time during measurement when the optical wavelength component acquisition unit 11 is the one in which the image data 2 is measured in real time by a camera or the like. When the data 2 is acquired by the media, the time zone is determined based on the imaging time added to the image data 2, and the initial likelihood corresponding to the user identification information and the time zone is determined. Select a value function.

  FIG. 10 is a process flow diagram of the heart rate estimation process of the heart rate / respiration rate detection apparatus 1.

  The optical wavelength component acquisition unit 11 inputs image data 2 captured by a camera or the like (step S1). Next, the face recognition processing unit 12 detects a face area (rectangular area) from the sampling frame of the input image data 2 by face recognition processing (step S2). Then, the RGB average value calculation unit 13 calculates the average value of each of the RGB components in the detected face area (step S3).

  The independent component analysis unit 14 calculates the three independent signals (components) by applying independent component analysis (ICA) to the average value of the RGB components (step S4). Next, the independent signal sorting unit 15 calculates the spectrum distribution of the three independent signals obtained by the independent component analysis (step S5). Further, the independent signal sorting unit 15 obtains the similarity of each spectral distribution between the three independent signals of the pre-sampling (the (n-1) th sampling) and the current sampling (the nth sampling). Are paired together, and the spectrum distribution of the independent signal is segmented (step S6).

  The heart rate / respiration rate detection unit 16 tracks the peak frequency with a particle filter for each segmented spectrum distribution (step S7), and identifies the peak frequency frequency value converged (step S8). Further, the heart rate / respiration rate detection unit 16 estimates and outputs the heart rate based on the frequency value of the peak frequency in the spectrum distribution of the specified independent signal (step S9).

  FIG. 11 is a process flow diagram of the likelihood initial value function setting process of the heart rate / respiration rate detection apparatus 1.

  The face recognition processing unit 12 acquires a user ID (identification information) based on the face area (rectangular area) detected from the sampling frame of the input image data 2 (step S10). The likelihood function setting unit 18 reads the likelihood initial value function of the corresponding user ID from the likelihood initial value function storage unit 17 (step S11), and as an initial state of the particle filter of the heart rate / respiration rate detection unit 16, The read likelihood initial value function is set (step S12).

  In the above-described embodiment, the processing when the heart rate / respiration rate detecting device 1 detects the heart rate has been described. However, the heart rate / respiratory rate detecting device 1 performs respiration by the same processing as the heart rate detection processing. It is also possible to detect the number.

  The heart rate / respiration rate detection device 1 can be realized by a computer system including hardware having a CPU and a memory and a software program, or dedicated hardware. That is, the heart rate / respiration rate detection device 1 includes a calculation device (CPU), a temporary storage device (DRAM, flash memory, etc.) and a permanent storage device (HDD, flash memory, etc.), and inputs / outputs data to / from the outside. It can be implemented by a computer that performs.

  The heart rate / respiration rate detection apparatus 1 can also be implemented by a program executable by the computer. In this case, a program describing the processing contents of the functions that the heart rate / respiration rate detecting device 1 should have is provided. When the computer executes the provided program, the processing function of the heart rate / respiration rate detecting apparatus 1 described above is realized on the computer. The computer can also read a program directly from a portable recording medium and execute processing according to the program. Furthermore, the program can be recorded on a computer-readable recording medium.

  As described above, according to the disclosed heart rate / respiration rate detection apparatus 1, the following effects are obtained.

  (1) A peak frequency caused by a heartbeat signal or a respiration signal can be identified from a plurality of signals and a spectrum distribution of each signal, and a heart rate and a respiration rate can be accurately detected.

  (2) Since the image data 2 can be acquired by a small camera capable of taking a color image and the data can be acquired in a non-contact manner to the detection target person, the apparatus can be configured at low cost.

DESCRIPTION OF SYMBOLS 1 Heart rate / respiration rate detection device 11 Optical wavelength component acquisition unit 12 Face recognition processing unit 13 RGB average value calculation unit 14 Independent component analysis unit 15 Independent signal sorting unit 16 Heart rate / respiration rate detection unit 17 Likelihood initial value function storage Part 18 Likelihood function setting part

Claims (9)

A device for detecting heart rate or respiratory rate from image data,
An optical wavelength component acquisition unit that acquires time-series data of a plurality of optical wavelength components from image data obtained by imaging a detection target person;
An average value calculation unit for calculating an average value of each light wavelength component based on the time-series data of the light wavelength component;
An independent component analyzer that obtains a plurality of independent signals by applying independent component analysis to the calculated average value;
An independent signal sorting unit that rearranges and classifies the plurality of obtained independent signals so that characteristics of each independent signal are consistently maintained;
From among a plurality of peak frequencies included in the spectrum distribution of the segmented independent signal, an independent signal including a peak frequency at which a fluctuation of a frequency value falls within a certain range after a certain period of time is specified, and a heartbeat signal Or a heart rate / respiration rate detection unit that provides an independent signal including a peak frequency resulting from the respiration signal. The heart rate / respiration rate detecting unit estimates a heart rate or a respiration rate of the detection target person based on a frequency value of a peak frequency included in a spectrum distribution of the specified independent signal. The heart rate / respiration rate detection device according to claim 1. The independent signal sorting unit obtains the waveform similarity of the spectrum distribution of the independent signal acquired before and after the time series, and pairs the spectrum distributions having a high waveform similarity to classify the obtained independent signals. The heart rate / respiration rate detection device according to claim 1 or 2, wherein the heart rate / respiration rate detection device is provided. 2. The waveform included in a range of a predetermined frequency value of a spectrum distribution when the heart rate / respiration rate detection unit specifies a peak frequency caused by a heart rate signal or a respiration signal. The heart rate / respiration rate detecting device according to any one of claims 3 to 4. The heart rate / respiration rate detection unit tracks fluctuations of a plurality of peak frequencies included in the spectral distribution of the segmented independent signal by a particle filter, and estimates a waveform overlap rate in the spectral distribution in the particle filter. The heart rate / respiration rate detection device according to claim 1, wherein the heart rate / respiration rate detection device is a degree. A first storage unit that stores a likelihood initial value function in association with a user;
The heart rate according to claim 4 or 5, wherein the heart rate / respiration rate detection unit selects an initial likelihood function of a user corresponding to the detection target person from the first storage unit.・ Respiratory rate detection device. A second storage unit for storing a likelihood initial value function in association with an imaging time zone of the image data;
The heart rate / respiration rate detection unit selects a likelihood initial value function corresponding to an imaging time zone of the image data from the second storage unit. 7. The heart rate / respiration rate detection device according to the item. A method for detecting heart rate or respiratory rate from image data,
Computer
Obtain time-series data of multiple light wavelength components from the image data of the person to be detected,
Calculate the average value of each light wavelength component based on the time-series data of the light wavelength component,
Applying independent component analysis to the calculated average value to obtain a plurality of independent signals,
The plurality of independent signals obtained are rearranged and partitioned so that the characteristics of each independent signal are consistently maintained,
From among a plurality of peak frequencies included in the spectrum distribution of the segmented independent signal, an independent signal including a peak frequency at which a fluctuation of a frequency value falls within a certain range after a certain period of time is specified, and a heartbeat signal Alternatively, the heart rate / respiration rate detection method is characterized in that an independent signal including a peak frequency resulting from a respiration signal is used. A program for detecting heart rate or respiratory rate from image data,
Computer
Obtain time-series data of multiple light wavelength components from the image data of the person to be detected,
Calculate the average value of each light wavelength component based on the time-series data of the light wavelength component,
Applying independent component analysis to the calculated average value to obtain a plurality of independent signals,
The plurality of independent signals obtained are rearranged and partitioned so that the characteristics of each independent signal are consistently maintained,
From among a plurality of peak frequencies included in the spectrum distribution of the segmented independent signal, an independent signal including a peak frequency at which a fluctuation of a frequency value falls within a certain range after a certain period of time is specified, and a heartbeat signal Or a heart rate / respiration rate detection program for executing processing to make an independent signal including a peak frequency caused by a respiration signal.