JP6197451B2 - Information processing apparatus and information processing method - Google Patents

Description

  The present invention relates to an information processing apparatus and an information processing method.

  2. Description of the Related Art Conventionally, in order to determine a person's arousal level, there is a technique for measuring the sympathetic nerve and parasympathetic nerve activity levels by acquiring a heartbeat interval as biological information of a subject and performing a fluctuation analysis of the heartbeat interval. The heartbeat interval can be calculated using an R wave that is a wave having the largest amplitude in one pulsation. The heartbeat interval can be represented by, for example, the time interval between two adjacent R waves of the heartbeat, and is expressed as RRI (R−R Interval). In the case of a pulse wave, the heartbeat interval is expressed as AAI (Amplitude-Amplitude Interval). Hereinafter, RRI and AAI are used interchangeably in terms of heartbeat intervals.

  For example, this technique has been proposed to be used for a driver of a vehicle as a heartbeat detection device. When the heartbeat detection device tries to acquire the driver's heartbeat to calculate the RRI, it is generated as noise in the electrocardiogram due to the influence of the vibration of the vehicle and is acquired as a signal containing noise. If the calculated RRI is an abnormal value, the heartbeat detecting device calculates the R wave from the heartbeat again, assuming that the R wave used for calculating the RRI is noise. The heartbeat detection device performs recalculation of RRI based on the recalculated R wave, and determines the arousal level by performing fluctuation analysis of RRI. The abnormal value of RRI can be, for example, beyond a predetermined percentage range from the reference value, or a range of values corresponding to the upper limit value and the lower limit value of the heart rate.

JP 2010-142456 A

  However, fine body movements caused by arousal efforts or sudden fluctuations in autonomic nerve activity cause unusual RRI fluctuations. In such a case, since the arousal level is determined in a state including these noises, the probability that the arousal level estimation is misreported increases.

  One aspect of the present invention is to provide an information processing apparatus and an information processing method capable of accurately estimating the arousal level.

  In one embodiment, the information processing apparatus includes an analysis unit, a monitoring unit, and an estimation unit. The analysis unit of the information processing apparatus analyzes heartbeat interval information in the frequency domain using an autoregressive model. The monitoring unit of the information processing apparatus analyzes the heartbeat interval information in the frequency domain using Fourier transform, and monitors whether the peak value of the analysis result of the analysis is within a predetermined range. The estimation unit of the information processing apparatus estimates the arousal level of the subject based on the analysis result of the analysis unit and the analysis result of the monitoring unit.

  The arousal level can be estimated with high accuracy.

FIG. 1 is a block diagram illustrating an example of the configuration of the terminal device according to the embodiment. FIG. 2 is an explanatory diagram showing an example of a heartbeat signal. FIG. 3 is an explanatory diagram showing an example of a normal pulse wave. FIG. 4 is an explanatory diagram showing an example of an abnormal pulse wave. FIG. 5 is an explanatory diagram for explaining processing for calculating a heartbeat interval. FIG. 6 is an explanatory diagram showing an example of heartbeat interval data. FIG. 7 is an explanatory diagram for explaining respiratory sinus arrhythmia. FIG. 8 is an explanatory diagram illustrating the physiological mechanism of respiratory sinus arrhythmia. FIG. 9 is an explanatory diagram showing an example of spectral density data. FIG. 10 is an explanatory diagram showing the maximum frequency in time series. FIG. 11 is an explanatory diagram showing the maximum spectral density in time series. FIG. 12 is an explanatory diagram showing an example in the case where there is a peak in the RSA region by FFT. FIG. 13 is an explanatory diagram showing an example when there is no peak in the RSA region by FFT. FIG. 14 is an explanatory diagram illustrating an example of a scale. FIG. 15 is a flowchart illustrating an example of the arousal level estimation process of the terminal device. FIG. 16 is a flowchart illustrating an example of time domain processing. FIG. 17 is an explanatory diagram showing an example of the AAI drop rate. FIG. 18 is a flowchart illustrating an example of the first frequency domain process. FIG. 19 is an explanatory diagram showing an example of a frequency having the maximum PSD by FFT. FIG. 20 is a flowchart illustrating an example of the second frequency domain process. FIG. 21 is an explanatory diagram illustrating an example of an AR model prediction error. FIG. 22 is an explanatory diagram illustrating an example of a computer that executes an information processing program.

  Hereinafter, embodiments of an information processing apparatus and an information processing method disclosed in the present application will be described in detail based on the drawings. The disclosed technology is not limited by the present embodiment. Further, the following embodiments may be appropriately combined within a consistent range.

  FIG. 1 is a block diagram illustrating an example of the configuration of the terminal device according to the embodiment. The terminal device 100 illustrated in FIG. 1 includes a detection unit 110, a calculation unit 120, an analysis unit 130, a monitoring unit 140, a scale storage unit 150, an estimation unit 160, and an adjustment unit 170. As the terminal device 100, for example, a smartphone, a mobile phone, a PHS (Personal Handy phone System), a PDA (Personal Digital Assistant, or Personal Data Assistance) can be used.

  The detection unit 110 detects a heartbeat signal of the subject. For example, the detection unit 110 applies a voltage to the electrode that contacts the subject, and acquires the heartbeat signal of the subject from the potential difference between the electrodes. The electrode used by the detection unit 110 corresponds to, for example, an electrode embedded in a chest belt type or wristwatch type small device (both hands are required). If the pulse wave is acquired, it is optical, and there are a wristwatch type, a wristband type (reflection type), an ear clip type (reflection type, transmission type), and the like.

  FIG. 2 is a diagram illustrating an example of a heartbeat signal detected by the detection unit. The horizontal axis in FIG. 2 indicates the passage of time, and the vertical axis indicates the electrocardiogram strength. As shown in FIG. 2, the electrocardiogram signal of a healthy person generally has four waveforms and is called a P wave, QRS wave, T wave, and U wave in time series order. In particular, the QRS wave is detected as an acute peak, and includes a Q wave that is a peak start point, an R wave that is a peak apex, and an S wave that is a peak end point. Each waveform from P wave to U wave corresponds to one heart beat. RRI2a calculated as an interval from the R wave to the next R wave corresponds to a heartbeat interval indicating a time interval of heartbeats. The detection unit 110 outputs the detected heartbeat signal data to the calculation unit 120 as heartbeat signal data.

  Moreover, the detection part 110 may measure blood flow, such as a test subject's earlobe, with light, and may make it acquire a pulse wave, for example. FIG. 3 is an explanatory diagram showing an example of a normal pulse wave. When detecting the pulse wave, the detection unit 110 calculates the interval from the peak of the amplitude to the peak of the next amplitude as AAI. The calculated AAI can be used as a heartbeat signal in the same manner as RRI. In the following description, RRI is used, but AAI may be used. FIG. 4 is an explanatory diagram showing an example of an abnormal pulse wave. Here, as shown in FIG. 4, when the pulse wave waveform is distorted and the AAI cannot be calculated correctly, the detection unit 110 calculates AAI by complementing the previous and subsequent AAIs.

  The calculation unit 120 calculates the heartbeat interval (RRI) and the fluctuation value (difference) of the heartbeat interval from the heartbeat signal of the subject. Then, the calculation unit 120 outputs the calculated RRI and fluctuation value of the heartbeat interval to the analysis unit 130 and the monitoring unit 140 as heartbeat interval information. Below, the process of the calculation part 120 is demonstrated in detail.

  For example, the calculation unit 120 calculates a heartbeat interval from the heartbeat signal data input by the detection unit 110. FIG. 5 is an explanatory diagram for explaining processing for calculating a heartbeat interval. The horizontal axis in FIG. 5 indicates the passage of time, and the vertical axis indicates the electrocardiogram strength. As illustrated in FIG. 5, the calculation unit 120 detects an amplitude peak at which the amplitude of the heartbeat signal is greater than or equal to a threshold value as an R wave. Then, every time an R wave is detected, the calculation unit 120 calculates the heartbeat interval 3a from the detected appearance time of each R wave. The method for detecting the amplitude peak is not limited to the method described above. For example, the calculation unit 120 may use a method using a zero-cross point where the differential coefficient of the heartbeat signal changes from positive to negative, a method of detecting an amplitude peak by performing pattern matching, and the like.

  For example, the calculation unit 120 generates heartbeat interval data indicating fluctuations of the calculated heartbeat interval over time. FIG. 6 is a diagram illustrating an example of heartbeat interval data. The horizontal axis in FIG. 6 indicates the passage of time, and the vertical axis indicates the heartbeat interval. As illustrated in FIG. 6, for example, the calculation unit 120 associates the calculated heartbeat interval with the R-wave detection time to generate heartbeat interval data. In addition, when receiving an instruction from the monitoring unit 140 to complement the heart rate interval (RRI), the calculation unit 120 complements the heart rate interval.

  For example, the calculation unit 120 calculates a fluctuation value of the heartbeat interval from the generated heartbeat interval data. Note that the fluctuation value of the heartbeat interval calculated by the calculation unit 120 corresponds to, for example, an RSA (Respiratory Sinus Arrhythmia) value.

  Respiratory sinus arrhythmia (RSA) will be described with reference to FIGS. FIG. 7 is an explanatory diagram for explaining respiratory sinus arrhythmia. The horizontal axis of FIG. 7 shows the passage of time, and the vertical axis shows the strength of the electrocardiogram. RSA is a physiological arrhythmia that has tachycardia during inspiration and a bradycardia during expiration. That is, as shown in FIG. 7, the RSA is detected as a change in the heartbeat interval that shortens the heartbeat interval at the time of inspiration 5a to become the heartbeat interval 5b, and extends at the time of expiration 5c to become the heartbeat interval 5d. Is done. The difference between the heart rate interval 5d and the heart rate interval 5b indicates an RSA value, which is an index for quantitatively evaluating the degree of RSA.

  It is known that the RSA value changes between awake and non-wake. The RSA value increases when the state of the subject changes from the awake state to the non-awake state. The RSA value changes because it is controlled by various physiological mechanisms as disclosed in non-patent literature (aging change of biological rhythm, Junichiro Hayano, 95 CLINICIAN, 94 No. 429). FIG. 8 is an explanatory diagram illustrating the physiological mechanism of respiratory sinus arrhythmia. As shown in FIG. 8, RSA 9a appears according to the activity level of cardiac vagus nerve activity 9b. The activity level of the cardiac vagus nerve activity 9b is subjected to three kinds of control: interference 9c from the respiratory center, excitatory stimulus 9d by baroreceptor and chemoreceptor reflex, and central vagus excitatory stimulus 9e. Of these, the interference 9c from the respiratory center includes an inhibitory stimulus during inspiration and an excitatory stimulus during expiration. At the time of awakening, the RSA 9a is not easily controlled by the excitatory stimulus 9d by the baroreceptor and chemoreceptor reflex and the central vagus nerve excitatory stimulus 9e, and is strongly controlled only by the interference 9c from the respiratory center.

  Returning to the description of FIG. 1, the analysis unit 130 calculates the spectral density for each frequency by performing frequency analysis on the heartbeat interval information input from the calculation unit 120. The analysis unit 130 calculates the spectral density using, for example, an autoregressive (AR) model. As disclosed in non-patent literature (Shunsuke Sato, Akira Yoshikawa, Toru Kiryu, “Basics of Biological Signal Processing”, Corona, Inc.), the AR model is a linear representation of past time-series data. It is a model expressed as a sum. The AR model has a feature that a clear maximum point can be obtained even with a small number of data compared to the Fourier transform.

によって表される。なお、理想的には、ｅ（ｓ）は、ホワイトノイズである。 The p-order AR model of the time series x (s) uses the AR coefficient a (m) and the error term e (s), which are weights for past values,

Represented by Ideally, e (s) is white noise.

をｋ次のＡＲ係数とすると、スペクトル密度Ｐ_ＡＲ（ｆ）は、

によって表される。解析部１３０は、式（３）及び心拍間隔情報に基づいて、スペクトル密度を算出する。なお、スペクトル密度の算出方法は、上述の方法に限定されるものではない。 P is the identification order, f _s is the sampling frequency, ε _p is the identification error,

Is the k-th order AR coefficient, the spectral density P _AR (f) is

Represented by The analysis unit 130 calculates the spectral density based on the equation (3) and the heartbeat interval information. Note that the method for calculating the spectral density is not limited to the above-described method.

  Each time the analysis unit 130 calculates the spectral density, it generates spectral density data indicating the spectral density for each frequency. FIG. 9 is an explanatory diagram showing an example of spectral density data. In FIG. 9, the horizontal axis indicates the frequency, and the vertical axis indicates the spectral density. For example, the spectral density component that appears in the region near 0.05 to 0.15 Hz is a low frequency (LF) component that reflects the active state of the sympathetic nerve. For example, the spectral density component that appears in the region near 0.15 to 0.4 Hz is a high frequency (HF) component that reflects the activity state of the parasympathetic nerve. As a component representative of the LF region, there is MWSA (Mayer Wave Sinus Arrhythmia) known to be related to blood pressure fluctuation. In addition, as a component representative of the HF region, there is respiratory sinus arrhythmia (RSA).

となる周波数ｆを極大点の周波数として算出し、この極大点の周波数を式（３）に代入することによって極大点のスペクトル密度を算出する。図９に示す例では、解析部１３０は、２つの極大点５１、５２を取得する。なお、極大点の周波数を「極大周波数」と表記し、極大点のスペクトル密度を「極大スペクトル密度」と表記する。 The analysis unit 130 acquires a local maximum point where the spectral density of the spectral density data is a local maximum. For example, the analysis unit 130

Is calculated as the frequency of the local maximum point, and the spectral density of the local maximum point is calculated by substituting the frequency of the local maximum point into Equation (3). In the example illustrated in FIG. 9, the analysis unit 130 acquires two maximum points 51 and 52. The frequency at the maximum point is expressed as “maximum frequency”, and the spectral density at the maximum point is expressed as “maximum spectral density”.

  The analysis unit 130 selects one maximum point included in the HF component from the acquired maximum points 51 and 52. As shown in FIG. 9, when one maximal point 52 is included in the HF component, the analysis unit 130 selects the corresponding maximal point 52. When the HF component includes a plurality of maximum points, the analysis unit 130 selects the maximum point having the lowest frequency. This is because the subject's sleepiness can be more accurately determined by selecting a local point on the low frequency side among the local maximum points included in the HF component. When the local maximum point is not included in the HF component, the analysis unit 130 selects one local maximum point having the lowest frequency from the region on the higher frequency side than 0.4 Hz.

  The analysis unit 130 calculates the maximum frequency and the maximum spectral density at the acquired maximum point. FIG. 10 is an explanatory diagram showing the maximum frequency in time series. The horizontal axis in FIG. 10 indicates the passage of time, and the vertical axis indicates the frequency. FIG. 11 is an explanatory diagram showing the maximum spectral density in time series. The horizontal axis of FIG. 11 shows the passage of time, and the vertical axis shows the spectral density. When the analysis unit 130 calculates the spectral density data at intervals of 10 seconds, the interval between the points in the time series direction shown in FIGS. 10 and 11 is an interval of 10 seconds. As shown in FIGS. 10 and 11, the analysis unit 130 calculates the maximum frequency and the maximum spectral density at regular time intervals. The analysis unit 130 outputs the calculated maximum frequency and maximum spectral density to the estimation unit 160 and the adjustment unit 170. Further, the analysis unit 130 outputs the prediction error of the AR model to the monitoring unit 140.

  Returning to the description of FIG. 1, the monitoring unit 140 calculates a spectral density for each frequency by performing frequency analysis on the heartbeat interval information input from the calculation unit 120. The monitoring unit 140 calculates the spectral density using, for example, FFT (Fast Fourier Transform).

  The monitoring unit 140 generates spectral density data indicating the spectral density for each frequency every time the spectral density is calculated. FIG. 12 is an explanatory diagram showing an example in the case where there is a peak in the RSA region by FFT. In FIG. 12, the horizontal axis indicates the frequency, and the vertical axis indicates the spectral density. The monitoring unit 140 acquires a local maximum point where the spectral density of the spectral density data is a local maximum. In the example illustrated in FIG. 12, the monitoring unit 140 acquires the maximum point 53 included in the HF component. The monitoring unit 140 calculates the maximum frequency and the maximum spectral density at the acquired maximum point 53. The monitoring unit 140 calculates the maximum frequency and the maximum spectral density at regular intervals. The monitoring unit 140 outputs the calculated maximum frequency and maximum spectral density to the estimation unit 160 and the adjustment unit 170.

  The monitoring unit 140 monitors whether or not the maximum frequency that is the peak value of the analysis result is a predetermined range, that is, a lower limit value of the predetermined frequency range. The monitoring unit 140 can set the HF region (0.15 Hz to 0.40 Hz) as the predetermined frequency range, for example. That is, the lower limit value of the predetermined frequency range is 0.15 Hz.

  Here, a case where the maximum frequency is the lower limit value of the predetermined frequency range will be described. FIG. 13 is an explanatory diagram showing an example when there is no peak in the RSA region by FFT. In the example of FIG. 13, there is no conspicuous peak in the HF region of the spectral density data, and the peak 54 is 0.15 Hz which is the lower limit value of the HF region. The monitoring unit 140 monitors whether or not the spectral density data is continuously attached to the lower limit value of the HF region for a predetermined time. The frequency spectrum has a 1 / f fluctuation characteristic, and the fact that the peak of the spectral density data is the lower limit of the frequency range in FFT indicates that there is a pseudo peak in the AR model.

  The determination of the pseudo peak will be described with reference to FIGS. It is assumed that the spectral density data shown in FIG. 9 is analyzed by the analysis unit 130 with respect to certain heartbeat interval information using an AR model. Here, the monitoring unit 140 analyzes the same heartbeat interval information using FFT. As shown in FIG. 12, when there is a local maximum point (peak) 53 other than the lower limit value of the frequency range in the analysis result, the monitoring unit 140 displays the local maximum point (peak) 52 in FIG. 9 as a desired heart rate interval (RRI). ) And the analysis result of the analysis unit 130 is determined to be valid.

  As shown in FIG. 13, when the monitoring unit 140 has a maximum point (peak) 54 in the lower limit value of the frequency range in the analysis result, the maximum point (peak) 52 in FIG. 9 is a pseudo peak based on noise. Is determined. The monitoring unit 140 calculates a pseudo peak rate within a predetermined time, that is, a rate at which the peak is the lower limit value of the frequency range. The monitoring unit 140 determines that the analysis result of the analysis unit 130 is invalid when the rate that is the calculated lower limit value exceeds a predetermined rate, for example, 50% in 10 seconds. When the analysis result of the analysis unit 130 is invalid, the monitoring unit 140 outputs invalid information to the estimation unit 160.

  In addition, the monitoring unit 140 monitors whether the deviation rate of each interval of the heartbeat interval information and the AR model prediction error exceed a predetermined threshold. The monitoring unit 140 receives heartbeat interval information from the calculation unit 120, and receives an AR model prediction error from the analysis unit 130. The monitoring unit 140 calculates a deviation rate of each interval of the heartbeat interval information. The monitoring unit 140 calculates a difference between heartbeat intervals in order to calculate a deviation rate. For example, if the certain heartbeat interval (RRI) is 950 ms and the other RRI is 1000 ms, the difference is 50 ms. When the monitoring unit 140 calculates the deviation rate from the difference, 50/950 = 0.053 = 5.3%. Here, the deviation rate calculation timing can be, for example, 10 seconds. The RRI between the deviation rate calculation timings is averaged, and the deviation rate with respect to the average value of the newly input RRI is calculated. Also good. The monitoring unit 140 monitors whether the deviation rate exceeds a predetermined threshold set in advance, for example, ± 20%. The monitoring unit 140 outputs invalid information to the estimation unit 160 when the deviation rate exceeds a predetermined threshold. In addition, when the adjacent heartbeat interval (RRI) exceeds a predetermined threshold, that is, when the deviation rate between adjacent heartbeat intervals exceeds a predetermined threshold, the monitoring unit 140 complements the calculation unit 120 with RRI. Instruct.

  The monitoring unit 140 accumulates the AR model prediction error in a predetermined period. The prediction error of the AR model is the e (s) term in the equation (1). If the AR model does not exist, the e (s) term changes abruptly. it can. The monitoring unit 140 accumulates the prediction error for a predetermined period, for example, 10 seconds, and monitors whether the accumulated prediction error exceeds a predetermined threshold. The monitoring unit 140 outputs invalid information to the estimation unit 160 when the accumulated prediction error exceeds a predetermined threshold. Note that the predetermined threshold value and the calculation timing in the deviation rate and the AR model prediction error can be adjusted based on the adjustment information input from the adjustment unit 170. For example, the predetermined threshold value of the deviation rate is lowered from ± 20% to ± 15%, and the calculation timing can be adjusted to be shortened from the 10 second interval to the 5 second interval.

  Returning to the description of FIG. 1, the scale storage unit 150 stores a scale for estimating the awakening level by the estimation unit 160. The scale generates a range in which the frequency and the spectral density can be changed as an index of the degree of arousal by using the maximum frequency and the maximum spectral density at the time of awakening and at the time of awakening in advance.

  FIG. 14 is an explanatory diagram illustrating an example of a scale. The horizontal axis in FIG. 14 indicates the frequency, and the vertical axis indicates the spectral density. In the example shown in FIG. 14, the scale 16a is set so that there is no sleepiness in the upper right and sleepiness is stronger in the lower left as shown in the sleepiness direction 16b. In this case, the scale 16a is divided into five areas from the upper right to the lower left, and five levels of sleepiness levels are assigned to each of the five areas. That is, the drowsiness level determined by the scale 16a increases in drowsiness from level 1 to level 5, and the arousal level decreases. As shown in FIG. 14, the scale storage unit 150 stores the normalized scale 16a. The scale data stored in the scale storage unit 150 is, for example, data including an expression indicating the boundary of each region in the scale and a sleepiness level value. Although FIG. 14 illustrates the case where the widths of the normalized scales 16a are equally spaced, the present invention is not limited to this. For example, the width of each region of the normalized scale 16a may be adjusted so as to become narrower as the drowsiness level becomes higher. In addition, the scale data is not limited to the above-described configuration, and may be configured by, for example, the frequency and spectral density of wake-up data and the frequency and spectral density of non-wake-up data.

  Returning to the description of FIG. 1, the estimation unit 160 estimates the arousal level of the subject. The estimation unit 160 receives the maximum frequency and the maximum spectral density from the analysis unit 130 and the monitoring unit 140, respectively. The estimation unit 160 receives invalid information from the monitoring unit 140. When invalid information is input from the monitoring unit 140, the estimation unit 160 determines whether the invalid information is based on frequency domain processing. If the invalid information is based on frequency domain processing, the estimation unit 160 outputs setting information to the adjustment unit 170 so as to adjust the setting value. Here, the set value is a set value of a deviation rate, a calculation timing of the deviation rate, and a prediction error of the AR model.

  The estimation unit 160 estimates the arousal level of the subject by comparing the maximum frequency and the maximum spectral density input from the analysis unit 130 with the index of the arousal level stored in the scale storage unit 150. For example, the estimation unit 160 receives input of identification information from the subject, and reads a scale corresponding to the identification information from the scale storage unit 150. The estimation unit 160 estimates the arousal level of the subject by comparing the maximum point calculated by the analysis unit 130 with the read scale. Specifically, for example, the estimation unit 160 determines a region including the calculated maximum point by substituting the maximum frequency and maximum spectral density of the maximum point into an expression representing each region of the scale. The estimation unit 160 estimates the sleepiness level of the subject according to the region determined to include the maximum point. And the estimation part 160 outputs an estimation result. The estimation unit 160 uses, for example, a monitor or a speaker (not shown) to notify the subject or a person around the subject of information indicating that the subject's arousal level has decreased as an estimation result. Note that the method for accepting identification information is not limited to the above-described method. For example, the estimation unit 160 may use a method of acquiring identification information from a camera image obtained by photographing the current subject, a method of determining using a region characteristic of an individual among heartbeat signals, and the like.

The adjustment unit 170 receives the maximum frequency and the maximum spectral density from the analysis unit 130 and the monitoring unit 140. When the setting information is input from the estimation unit 160, the adjustment unit 170 adjusts the heart rate interval deviation rate and the predetermined threshold of the AR model prediction error based on the input maximum frequency and maximum spectral density. Is generated. For example, the adjustment unit 170 generates adjustment information that lowers a predetermined threshold value from ± 20% to ± 15% for the heart rate interval deviation rate, and shortens the deviation rate calculation timing from the 10-second interval to the 5-second interval. Generate adjustment information. In addition, for example, the adjustment unit 170 generates adjustment information for reducing a predetermined threshold value from 5 × 10 ⁻⁷ to 4 × 10 ⁻⁷ for the prediction error of the AR model. The adjustment unit 170 outputs the generated adjustment information to the monitoring unit 140.

  Next, the operation of the terminal device 100 according to the embodiment will be described.

FIG. 15 is a flowchart illustrating an example of the arousal level estimation process of the terminal device. First, the terminal device 100 sets predetermined thresholds for the interval deviation rate of heart rate interval information, the calculation timing, and the prediction error of the AR model as an initial setting (step S1). For example, the predetermined threshold is set such that the deviation rate is ± 20%, the calculation timing is 10 seconds, and the AR model prediction error is 5 × 10 ⁻⁷ .

  When the initial setting is completed, the terminal device 100 starts acquiring heart rate data (step S2). The detection unit 110 outputs the acquired heartbeat signal data to the calculation unit 120. The calculation unit 120 calculates a heartbeat interval (RRI) and a fluctuation value of the heartbeat interval from the heartbeat signal of the subject (step S3). The calculation unit 120 outputs the RRI and the calculated fluctuation value of the heartbeat interval to the analysis unit 130 and the monitoring unit 140 as heartbeat interval information.

  The monitoring unit 140 performs time domain processing based on the input heartbeat interval information (step S4). FIG. 16 is a flowchart illustrating an example of time domain processing. The monitoring unit 140 calculates a difference between adjacent heartbeat intervals (RRI) based on the RRI of the heartbeat interval information (step S41). The monitoring unit 140 may use a heartbeat interval variation value of the heartbeat interval information as the RRI difference. The monitoring unit 140 calculates a deviation rate from the difference between adjacent heartbeat intervals within a predetermined period, that is, between calculation timings (step S42). That is, the monitoring unit 140 calculates a deviation rate based on a difference between a certain heartbeat interval and the previous heartbeat interval, and when a new heartbeat interval is input, a new heartbeat interval and a certain heartbeat interval are calculated. A deviation rate is calculated based on the difference. For this reason, the monitoring unit 140 calculates a deviation rate for each heartbeat interval.

  FIG. 17 is an explanatory diagram showing an example of the AAI drop rate. FIG. 17 is a graph showing a change of a deviation rate (drop rate) with time when AAI is used as a heartbeat interval. In the example of FIG. 17, the deviation rate is almost 0% until 650 seconds, and is several percent from 650 seconds to 900 seconds. Further, the predetermined threshold 55 is set to 20%, and the deviation rate is equal to or less than the predetermined threshold 55.

  Returning to the description of FIG. 16, the monitoring unit 140 determines whether or not the deviation rate is within a specified value, that is, within a predetermined threshold (step S43). If the deviation rate is within the specified value (step S43: Yes), the monitoring unit 140 ends the time domain process and returns to the original process. If the deviation rate is not within the specified value (No at Step S43), the monitoring unit 140 outputs out-of-service, that is, invalid information to the estimation unit 160, and returns to the original process (Step S44).

  Returning to the description of FIG. 15, the monitoring unit 140 executes the first frequency domain process based on the input heartbeat interval information (step S5). FIG. 18 is a flowchart illustrating an example of the first frequency domain process. The monitoring unit 140 analyzes the heartbeat interval information using FFT and calculates the spectral density (step S51). The monitoring unit 140 generates spectral density data indicating the spectral density for each frequency every time the spectral density is calculated.

  The monitoring unit 140 calculates the rate of the pseudo peak of the spectral density (PSD) within the predetermined time, that is, the rate at which the peak of the spectral density is the lower limit value of the frequency range (step S52). FIG. 19 is an explanatory diagram showing an example of a frequency having the maximum PSD by FFT. FIG. 19 is a graph showing changes over time of frequencies having spectral density peaks by FFT. In the example of FIG. 19, the peak frequency is approximately 0.3 Hz, but is approximately 0.18 Hz around 300 seconds and 500 seconds. For the time when the spectral density peak is about 0.18 Hz, for example, if the lower limit value 56 of the frequency range is 0.18 Hz, the lower limit value 56 of the frequency range is stuck. For example, when the time when the spectral density peak is 0.18 Hz is 1 second and the predetermined time is 10 seconds, the monitoring unit 140 has a rate at which the spectral density peak within the predetermined time is the lower limit value of the frequency range of 10%. And calculate.

  Returning to the description of FIG. 18, the monitoring unit 140 determines whether or not the rate at which the peak of the spectral density within the calculated predetermined time is the lower limit value of the frequency range is within a specified value (step S53). If the rate that is the calculated lower limit value is not within the specified value (No at Step S53), the monitoring unit 140 determines that the analysis result of the analysis unit 130 is invalid, and sends out-of-service, that is, invalid information to the estimation unit 160. Output and return to the original process (step S54). If the rate that is the calculated lower limit value is within the specified value (step S53: Yes), the monitoring unit 140 ends the first frequency domain process and returns to the original process.

  Returning to the description of FIG. 15, the analysis unit 130 performs the second frequency domain process based on the input heartbeat interval information (step S <b> 6). FIG. 20 is a flowchart illustrating an example of the second frequency domain process. The analysis unit 130 analyzes the heartbeat interval information using the AR model and calculates the spectral density (step S61). Each time the analysis unit 130 calculates the spectral density, it generates spectral density data indicating the spectral density for each frequency. In addition, the analysis unit 130 acquires a local maximum point at which the spectral density of the spectral density data is maximum, and calculates a local maximum frequency and a local spectral density at the local maximum point acquired at regular intervals. The analysis unit 130 outputs the calculated maximum frequency and maximum spectral density to the estimation unit 160 and the adjustment unit 170. Further, the analysis unit 130 outputs the prediction error of the AR model to the monitoring unit 140.

The monitoring unit 140 accumulates the input AR model prediction error for a predetermined period, for example, 10 seconds (step S62). FIG. 21 is an explanatory diagram illustrating an example of an AR model prediction error. FIG. 21 is a graph showing a change in prediction error of the AR model over time. In the example of FIG. 21, when the predetermined threshold 57 of the prediction error, that is, the specified value is 5 × 10 ⁻⁷ , the prediction error exceeds the predetermined threshold after about 750 seconds.

  Returning to the description of FIG. 20, the monitoring unit 140 monitors whether the accumulated prediction error of the AR model exceeds a predetermined threshold. That is, the monitoring unit 140 determines whether or not the accumulated AR model prediction error is within a specified value (step S63). When the prediction error of the accumulated AR model is not within the specified value (No at Step S63), the monitoring unit 140 determines that the analysis result of the analysis unit 130 is invalid, and estimates the out-of-service, that is, invalid information, estimation unit 160. And return to the original process (step S64). When the prediction error of the accumulated AR model is within the specified value (step S63: Yes), the monitoring unit 140 ends the second frequency domain process and returns to the original process.

  Returning to the description of FIG. 15, the estimation unit 160 determines whether or not out-of-service, that is, invalid information is input from the monitoring unit 140 (step S <b> 7). When invalid information is input (step S7: affirmative), the estimation unit 160 determines whether the invalid information (out of service) is based on the first frequency domain process or the second frequency domain process. Determination is made (step S8). When the invalid information is based on the first frequency domain process or the second frequency domain process (step S8: affirmative), the estimation unit 160 is set to adjust the setting value to the adjustment unit 170. Output information. Here, the setting information is information for instructing to adjust the deviation rate of the heartbeat interval and a predetermined threshold of the prediction error of the AR model.

  When the setting information is input from the estimation unit 160, the adjustment unit 170, based on the maximum frequency and the maximum spectral density input from the analysis unit 130 and the monitoring unit 140, determines the heart rate interval deviation rate and the AR model prediction error. Adjustment information for adjusting a predetermined threshold value is generated. The adjustment unit 170 outputs the generated adjustment information to the monitoring unit 140. Based on the adjustment information, the monitoring unit 140 adjusts a predetermined threshold value and calculation timing in the deviation rate and the AR model prediction error (step S9). Moreover, when the setting unit 160 outputs the setting information, the estimation unit 160 notifies the subject that it is out-of-service instead of the estimation result (step S10), and returns to step S2.

  If the invalid information is not based on the first frequency domain process or the second frequency domain process (Step S8: No), the estimation unit 160 notifies the subject that it is out-of-service instead of the estimation result. (Step S10), the process returns to Step S2. Even after notifying the subject that the terminal device 100 is out-of-service, the terminal device 100 repeats the processing from step S2 to step S10. Can be estimated.

  When invalid information is not input (Step S7: No), the estimation unit 160 compares the maximum frequency and maximum spectral density input from the analysis unit 130 with the wakefulness index stored in the scale storage unit 150. Thus, the awakening level of the subject is estimated (step S11). The estimation unit 160 uses, for example, a monitor or a speaker (not shown) to notify the subject or a person around the subject of information indicating that the subject's arousal level has decreased as an estimation result (step S12). The estimation unit 160 determines whether or not the estimation of the arousal level is finished (step S13). When the estimation of the arousal level is not completed (No at Step S13), the estimation unit 160 returns to Step S2 and continues the arousal level estimation process. When the estimation of the arousal level is completed (step S13: affirmative), the estimation unit 160 ends the arousal level estimation process.

  As described above, the terminal device 100 analyzes the heart rate interval information in the frequency domain using the autoregressive model, analyzes the heart rate interval information in the frequency domain using Fourier transform, and the peak value of the analysis result of the analysis is predetermined. Monitor whether it is in range. The terminal device 100 estimates the arousal level of the subject based on the analysis result using the autoregressive model and the analysis result using Fourier transform. As a result, the arousal level can be estimated with high accuracy.

  Further, the terminal device 100 uses the frequency range as the predetermined range, monitors whether the peak value is at the lower limit value of the analyzed frequency range, and when the peak value is at the lower limit value of the analyzed frequency range, Invalidate the analysis results using the autoregressive model. As a result, it is possible to prevent misjudgment due to a pseudo peak of the autoregressive model and improve the estimation accuracy of the arousal level.

  Further, the terminal device 100 further calculates a deviation rate of each interval of the heartbeat interval information, and monitors whether one or more of the deviation rate and the prediction error of the autoregressive model exceed a predetermined threshold. The terminal device 100 invalidates the analysis result using the autoregressive model when at least one of the deviation rate and the prediction error of the autoregressive model exceeds a predetermined threshold. As a result, it is possible to improve the reliability of the arousal level by stopping the estimation of the arousal level when the signal such as a pulse wave or the autoregressive model does not match the actual heartbeat interval. Moreover, the troublesomeness of the driver of a vehicle can be reduced by reducing misinformation.

  Further, the terminal device 100 further reduces the predetermined threshold and shortens the timing for calculating the deviation rate based on one or more of the analysis result using the autoregressive model and the analysis result using the Fourier transform. Adjust to do one or more of the things. As a result, it is possible to reduce the influence of noise by making the conditions for analysis stricter. Moreover, the troublesomeness of the driver of a vehicle can be reduced by reducing misinformation.

  Moreover, in the said Example, although the terminal device 100 was illustrated as an information processing apparatus, it is not limited to this. For example, a personal computer or a workstation may be used as the information processing apparatus. Further, for example, when the information processing device is mounted on a vehicle, a vehicle control device, a car navigation device, or an operation recorder (digital tachograph) may be used.

  Of the processes described in the above embodiment, all or part of the processes described as being automatically performed can be performed manually, or all or all of the processes described as being performed manually can be performed. A part can be automatically performed by a known method. For example, the series of processes of the terminal device 100 illustrated in FIG. 15 may be executed in response to receiving an instruction from the driver after the system of the vehicle on which the terminal device 100 is mounted is activated.

  In addition, each component of each part illustrated does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each part is not limited to the one shown in the figure, and all or a part thereof may be functionally or physically distributed / integrated in arbitrary units according to various loads and usage conditions. Can be configured. For example, the monitoring unit 140 may be configured by separately performing analysis processing by FFT and processing for monitoring heartbeat intervals, analysis results, and the like.

  FIG. 22 is an explanatory diagram illustrating an example of a computer that executes an information processing program according to the present embodiment. As shown in FIG. 22, a computer 300 includes an input device 301 that receives data input from a user, a monitor 302, a medium reading device 303 that reads a program and the like from a storage medium, and an interface that exchanges data with other devices. Device 304. The computer 300 also includes a CPU 305 that executes various arithmetic processes, a RAM 306 that temporarily stores various information, and a hard disk device 307. Each device 301 to 307 is connected to a bus 308. In addition, the computer 300 is connected via a heartbeat sensor 310 that detects a heartbeat signal of a subject, a speaker 320, and an interface device 304.

  The hard disk device 307 stores various programs having functions similar to the processing units of the calculation unit 120, the analysis unit 130, the monitoring unit 140, the estimation unit 160, and the adjustment unit 170 illustrated in FIG. Further, the hard disk device 307 stores the scale of the subject as the scale storage unit 150 in association with identification information for identifying the subject.

  The CPU 305 reads out various programs from the hard disk device 307, develops them in the RAM 306, and executes them, whereby the various programs function as various processes. That is, various programs function as processes similar to the processing units of the calculation unit 120, the analysis unit 130, the monitoring unit 140, the estimation unit 160, and the adjustment unit 170.

  Note that the various programs described above are not necessarily stored in the hard disk device 307. For example, the computer 300 may read and execute a program stored in a computer-readable recording medium. As the computer-readable recording medium, for example, a portable recording medium such as a CD-ROM, a DVD disk, and a USB memory, a semiconductor memory such as a flash memory, a hard disk drive, and the like are supported. Further, the program may be stored in a device connected to a public line, the Internet, a local area network (LAN), a wide area network (WAN), etc., and the computer 300 may read and execute the program from these devices. good.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Additional remark 1) The analysis part which analyzes heartbeat interval information in a frequency domain using an autoregressive model,
A monitoring unit that analyzes the heartbeat interval information in a frequency domain using Fourier transform and monitors whether or not a peak value of an analysis result of the analysis is within a predetermined range;
Based on the analysis result of the analysis unit and the analysis result of the monitoring unit, an estimation unit for estimating the arousal level of the subject,
An information processing apparatus comprising:

(Supplementary Note 2) The monitoring unit uses a frequency range as the predetermined range, and monitors whether the peak value is a lower limit value of the analyzed frequency range,
The information processing apparatus according to appendix 1, wherein the estimation unit invalidates an analysis result of the analysis unit when the peak value is at a lower limit value of the analyzed frequency range.

(Supplementary Note 3) The monitoring unit further calculates a deviation rate of each interval of the heartbeat interval information, and monitors whether one or more of the deviation rate and the prediction error of the autoregressive model exceed a predetermined threshold. ,
The estimation unit invalidates the analysis result of the analysis unit when one or more of the deviation rate and the prediction error of the autoregressive model exceed a predetermined threshold. Information processing device.

(Supplementary Note 4) Further, one of lowering the predetermined threshold and shortening the calculation timing of the deviation rate based on one or more of the analysis result of the analysis unit and the analysis result of the monitoring unit The information processing apparatus according to attachment 3, further comprising: an adjustment unit that adjusts to perform at least two.

(Appendix 5) The computer
Analyzing heart rate interval information in the frequency domain using an autoregressive model,
Analyzing the heartbeat interval information in the frequency domain using Fourier transform, monitoring whether the peak value of the analysis result of the analysis is within a predetermined range,
An information processing method comprising: executing a process of estimating a subject's arousal level based on an analysis result using the autoregressive model and an analysis result using the Fourier transform.

(Appendix 6)
Analyzing heart rate interval information in the frequency domain using an autoregressive model,
Analyzing the heartbeat interval information in the frequency domain using Fourier transform, monitoring whether the peak value of the analysis result of the analysis is within a predetermined range,
An information processing program for executing a process of estimating a subject's arousal level based on an analysis result using the autoregressive model and an analysis result using the Fourier transform.

DESCRIPTION OF SYMBOLS 100 Terminal device 110 Detection part 120 Calculation part 130 Analysis part 140 Monitoring part 150 Scale memory | storage part 160 Estimation part 170 Adjustment part

Claims (5)

An analysis unit that analyzes heartbeat interval information in the frequency domain using an autoregressive model;
A monitoring unit that analyzes the heartbeat interval information in a frequency domain using Fourier transform and monitors whether a peak value of an analysis result of the analysis is a predetermined value in a predetermined frequency range;
When the monitoring result of the monitoring unit is not the predetermined value, based on the analysis result of the analysis unit and the analysis result of the monitoring unit, an estimation unit that estimates the arousal level of the subject;
An information processing apparatus comprising: The monitoring unit monitors whether before Symbol peak value is the lower limit of the predetermined frequency range analyzed,
The information processing apparatus according to claim 1, wherein the estimation unit invalidates an analysis result of the analysis unit when the peak value is at a lower limit value of the predetermined frequency range in which the analysis is performed. . The monitoring unit further calculates a deviation rate of each interval of the heartbeat interval information, and monitors whether one or more of the deviation rate and the prediction error of the autoregressive model exceed a predetermined threshold,
The said estimation part invalidates the analysis result of the said analysis part, when one or more among the said deviation rate and the prediction error of the said autoregressive model exceeds a predetermined threshold value, The analysis part of Claim 2 characterized by the above-mentioned. Information processing device. Further, based on one or more of the analysis result of the analysis unit and the analysis result of the monitoring unit, one or more of lowering the predetermined threshold and shortening the calculation timing of the deviation rate is performed. The information processing apparatus according to claim 3, further comprising: an adjustment unit that adjusts as described above. Computer
Analyzing heart rate interval information in the frequency domain using an autoregressive model,
Analyzing the heart rate interval information in the frequency domain using Fourier transform, monitoring whether the peak value of the analysis result of the analysis is a predetermined value in a predetermined frequency range,
Information processing characterized in that, when a monitoring result is not the predetermined value, a process of estimating a subject's arousal level is executed based on an analysis result using the autoregressive model and an analysis result using the Fourier transform Method.

Images