JP2014012042A - Drowsiness determination method, device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow driver drowsiness to be determined highly accurately in a drowsiness determination method, device, and program.SOLUTION: A drowsiness determination method for determining drowsiness of a subject on the basis of a drowsiness scale is constituted so that a computer executes the following steps of: obtaining heartbeat interval data by arranging an RRI (R-R Interval) in time series on the basis of a heartbeat waveform of the subject in a rest state with stable breathing measured by a sensor; subjecting the heartbeat interval data to frequency-analysis to determine the frequency analysis result of heartbeat fluctuation that includes power spectral density (PSD) for a frequency at a certain time and total power (TP) of autonomous nerve; setting the frequency analysis result as an initial state; and determining a drowsiness scale in which the origin of a drowsiness position and the origin of an awakening position are set, on the basis of an estimated drowsiness position and an estimated awakening position contained in the initial state, and storing the determine scale in a storage medium.

Description

  The present invention relates to a drowsiness determination method, apparatus, and program.

  One of the causes of traffic accidents is a human error such as a sloppy driving by a driver. One of the causes such as casual driving is snoozing driving, and a reduction in traffic accidents can be expected by detecting the driver's sleepiness. The detection of drowsiness is desired not only for drivers of ordinary vehicles but also for operators who perform driving or steering operations on devices such as heavy machinery, trains, ships, and aircraft. In this specification, an operator who performs a driving operation or a steering operation of these devices is referred to as a “driver” for convenience.

  A drowsiness determination method using heart rate fluctuation has been proposed. First, a heartbeat signal is obtained by measuring a driver's heartbeat using a sensor, and heartbeat interval data in which intervals of peak values called R waves of the heartbeat signal are time-series is obtained. The heartbeat interval data is subjected to frequency analysis to determine a power spectral density (PSD) with respect to the frequency at a certain time. In this PSD, it has been reported that the influence of sympathetic nerves and parasympathetic nerves appears in the low frequency component, and the influence of parasympathetic nerves appears in the high frequency component. In this proposed method, drowsiness is determined from the ratio of low frequency components to high frequency components. However, in this proposed method, since the drowsiness of different drivers is determined using a common threshold, it is difficult to determine drowsiness with high accuracy due to the great influence of individual differences among drivers.

  On the other hand, a sleepiness determination method using a sleepiness transition graph specialized for each driver has been proposed. The sleepiness transition graph is a graph in which the movement range of sleepiness and awakening is determined in advance. In this proposed method, the movement range of sleepiness and awakening is learned in advance. First, the sleepiness transition graph is created by learning the movement range of sleepiness and awakening when the target driver drives the vehicle several times. Next, when the target driver gets on the vehicle after learning, the driver sets the awake state assuming that the driver is in the awake state, and estimates the position of the sleepy state from the awake state using the sleepiness transition graph . However, with this proposed method, learning is necessary when the target driver actually drives the vehicle, so an accurate sleepiness transition graph must be created unless the driver actually drives the vehicle to some extent. I can't. For example, even if the driver determines drowsiness using a sleepiness transition graph created based on one driving for a relatively short time, it is difficult to determine drowsiness with high accuracy. Further, even if the driver has created a sleepiness transition graph, it is assumed that the driver is in an awake state when he gets on the vehicle. There was a problem that drowsiness could not be judged correctly when riding on the car.

JP 2008-264138 A JP 2008-125801 A JP 2009-39167 A International Publication WO2008 / 065724

  With the conventional sleepiness determination method, it is difficult to determine the driver's sleepiness with high accuracy.

  Accordingly, an object of the present invention is to provide a drowsiness determination method, apparatus, and program capable of determining a driver's drowsiness with high accuracy.

  According to one aspect of the present invention, a sleepiness determination method for determining sleepiness of a subject based on a sleepiness scale, wherein RRI (RR Interval) is calculated based on a heartbeat waveform of a resting subject with stable breathing measured by a sensor. Time-series heartbeat interval data is obtained, the heartbeat interval data is subjected to frequency analysis, and a frequency analysis result of heartbeat fluctuation including power spectrum density (PSD) and total power (TP) of the autonomic nerve at a certain time is obtained. Obtaining the frequency analysis result as an initial state, and based on the estimated drowsiness position and the estimated arousal position included in the initial state, the origin of the drowsiness position and the origin of the awakening position are set. A drowsiness determination method in which a computer executes a process of determining a drowsiness scale and storing it in a storage medium is provided.

  According to the disclosed sleepiness determination method, apparatus, and program, it is possible to determine the driver's sleepiness with high accuracy.

It is a flowchart explaining an example of the initial setting process in one Example. It is a figure which shows an example of a heartbeat waveform. It is a figure which shows the heartbeat waveform of FIG. 2 about several RRI. It is a figure which shows an example of the heartbeat interval data corresponding to the heartbeat waveform of FIG. It is a figure which shows an example of PSD. It is a figure explaining the heartbeat interval at the time of inhalation and expiration. It is a figure explaining the RRI series at the time of inhalation and expiration. It is a figure which shows the relationship between RSA and a RRI series when PSD similar to FIG. 5 is calculated | required in a resting state. It is the figure which plotted the relationship between the width of RSA and frequency. It is the figure which plotted the relationship between the amplitude of RSA and PSD. It is the figure which plotted the relationship between the highest frequency and the lowest frequency where the HF component of the frequency analysis result becomes maximum, It is the figure which plotted the relationship between the maximum PSD and the minimum PSD at the frequency where the HF component of the frequency analysis result becomes maximum. It is a figure which shows an example of a sleepiness scale. It is a figure explaining the time transition of PSD calculated | required in the resting state. It is a figure explaining the determination method of a sleepiness scale. It is a figure explaining simulation driving | operation. It is a figure explaining an example at the time of simulation operation, and PSD. It is a figure explaining an example of the detection output of a pulse wave. It is a figure explaining an example of PSD. It is a figure which shows an example of the correlation of PSD and TP. It is a block diagram which shows an example of a function structure of the sleepiness determination apparatus in one Example. It is a block diagram which shows an example of the hardware constitutions of the drowsiness determination apparatus in one Example. It is a flowchart explaining an example of the estimation process of a drowsiness position. It is a flowchart explaining an example of the estimation process of the position of awakening. It is a flowchart explaining an example of a drowsiness determination process. It is a figure explaining an example of a sensor. It is a figure explaining the other example of a sensor.

  The disclosed sleepiness determination method, apparatus, and program determine sleepiness of a subject based on a sleepiness scale. Based on the heartbeat waveform of the subject in a stable state of respiration, which is measured by the sensor, heartbeat interval data obtained by time-series RRI (RR Interval) is obtained, the heartbeat interval data is subjected to frequency analysis, and the power spectrum for the frequency at a certain time. Obtain the frequency analysis result of heart rate fluctuation including density (PSD) and autonomic nerve total power (TP), set the frequency analysis result as the initial state, and estimate the position of the estimated sleepiness included in the initial state and the estimated Based on the awakening position, a sleepiness scale in which the origin of the drowsiness position and the origin of the awakening position are set is determined.

  Embodiments of the disclosed drowsiness determination method, apparatus, and program will be described below with reference to the drawings.

  The human heart rate when drowsiness (ie, sleepiness) can be approximated by the heart rate when resting (ie, resting state). A resting state refers to a state in which human respiration is stable with relatively little disturbance. However, when the heart rate interval data in the resting state is to be substituted for the heart rate interval data in the sleepy state, unlike the heart rate information, the heart rate interval data in the resting state and the power spectral density (PSD: Power Spectral Density) values are There is variation compared to heartbeat interval data and PSD values in the sleepy state. For this reason, if the heartbeat interval data in the resting state is directly used as the heartbeat interval data in the drowsiness state, sleepiness may be erroneously determined.

  As will be described below, the inventor uses a heartbeat waveform measured in a resting state, so that the frequency at which the high-frequency component of the frequency analysis result is maximum (or extreme value) is the high-frequency component in the sleepy state. It has been found that the frequency becomes the maximum (or extreme value) or less, and the PSD of the frequency analysis result is more than the PSD in the sleepy state.

  The present inventor has also found that there is a correlation between the PSD value and the total power (TP) of the autonomic nerve, as will be described below.

  FIG. 1 is a flowchart illustrating an example of an initial setting process in one embodiment. In FIG. 1, in step S <b> 1, the heart rate of a target driver is measured using a sensor in a resting state to obtain a heart rate waveform as shown in FIG. 2, for example, and a peak called an R wave of the heartbeat waveform as shown in FIG. Heartbeat interval data as shown in FIG. 4 in which the interval of values (RRI: RR Interval) is time-series is obtained. The heart rate interval data may be referred to as heart rate variability (HRV) data. FIG. 2 is a diagram illustrating an example of a heartbeat waveform, in which the vertical axis indicates electrocardiogram strength in arbitrary units (au), and the horizontal axis indicates time in arbitrary units (au). 3 is a diagram showing the heartbeat waveform of FIG. 2 for a plurality of RRIs, where the vertical axis indicates the electrocardiogram strength in arbitrary units, and the horizontal axis indicates time in arbitrary units (au). FIG. 4 is a diagram illustrating an example of heartbeat interval data corresponding to the heartbeat waveform of FIG. 3. The vertical axis indicates the heartbeat interval in arbitrary units, and the horizontal axis indicates time in arbitrary units.

  Since step S1 obtains a heartbeat waveform in a resting state, the driver's heartbeat may be measured, for example, in a relatively dark and quiet place or in a quiet place with the driver's eyes closed. . The driver's heart rate may be measured, for example, in a dedicated place for initial setting, and the dedicated place may be, for example, a room in a store that sells vehicles. In addition, the driver's heart rate may be measured in a parked vehicle in an environment suitable for the initial setting.

In step S2, the heart rate interval data is subjected to frequency analysis (or spectrum analysis), and for example, as shown in FIG. 5, the power spectral density (PSD: Power Spectral Density) and the total power (TP: Obtain the frequency analysis result of heart rate fluctuation including Total Power) and set the frequency analysis result as the initial state. The frequency analysis of the heartbeat interval data itself can be performed by a known frequency analysis method, and the frequency analysis method is not particularly limited. FIG. 5 is a diagram illustrating an example of a PSD, where the vertical axis indicates PSD (ms ² / Hz) and the horizontal axis indicates frequency (Hz). TP is represented by TP = LF + HF (or LF + HF + VLF (0.05−0.15 Hz)), and the power of the power of the low frequency (LF) component and the high frequency (HF) component in the PSD. It is sum. In this example, the LF component is a frequency component of 0.05 Hz to 0.15 Hz, and the HF component is a frequency component of 0.15 Hz to 0.4 Hz. The inventor uses the heartbeat waveform measured in a resting state, so that the frequency at which the HF component of the frequency analysis result is maximized is equal to or less than the frequency at which the HF component in the sleepy state is maximized, and the PSD of the frequency analysis result is It has been found that the amplitude is equal to or greater than the PSD amplitude in the case of sleepiness. In PSD, it has been reported that sympathetic and parasympathetic effects appear in the LF component, and parasympathetic effects appear in the HF component. In FIG. 5, the LF component includes a blood pressure fluctuation component (MWSA: Mayer Wave related Sinus Arrhythmia), and the HF component includes a respiratory sinus arrhythmia (RSA).

  In step S3, based on the set initial state, at least one of the position of the driver's sleepiness and the position of the awakening in the sleepiness scale (or awakening scale) for determining sleepiness (or awakening) is determined. presume. The sleepiness scale will be described later. The drowsiness position in the drowsiness scale is the threshold or range on the drowsiness scale that determines that the driver is in a drowsiness state, and the awakening position in the drowsiness scale is determined that the driver is in a waking state The threshold or range on the sleepiness scale to be played. Both the position of sleepiness and the position of arousal in the sleepiness scale may be estimated separately, but as described later, the position of awakening can be estimated from the position of sleepiness. Since the position can be estimated, at least one of the drowsiness position and the awakening position may be estimated. When estimating at least one of the position of sleepiness and the position of arousal in the sleepiness scale based on the initial state, attention is paid to respiratory sinus arrhythmia (RSA). RSA is an interference from the respiratory center (ie, respiratory components including inhibitory stimuli during inspiration and excitatory stimuli during expiration), excitatory stimuli from baroreceptor and chemoreceptor reflexes (blood pressure fluctuations), and It is known to depend on three factors: central vagal excitatory stimulation (stimulation from the brain).

  FIG. 6 is a diagram for explaining heartbeat intervals during inspiration and expiration. In FIG. 6, the vertical axis indicates electrocardiogram strength in arbitrary units, and the horizontal axis indicates time in arbitrary units. As can be seen from FIG. 6, the heartbeat interval (or RRI) W1 tends to be shorter during inspiration, and the heartbeat interval (or RRI) W2 tends to be longer during expiration. FIG. 7 is a diagram illustrating an RRI sequence during inspiration and expiration. In FIG. 7, the vertical axis indicates the heartbeat interval in arbitrary units, and the horizontal axis indicates time in arbitrary units. The amplitude of RSA corresponds to the difference between the heartbeat interval (or RRI) W1 during inspiration and the heartbeat interval (or RRI) W2 during expiration.

  In the resting state, the influence of the respiratory component on the RSA becomes dominant. In this embodiment, the RSA is extracted using this characteristic. Specifically, the position of sleepiness in the sleepiness scale is estimated by extracting the frequency (that is, the RSA feature point) at which the HF component of the frequency analysis result of heartbeat fluctuation is maximized.

  The inventor has a relatively high correlation between resting respiratory changes (that is, the period and amplitude of the RRI sequence) and RSA (that is, the frequency analysis result including PSD and the frequency at which the HF component is maximized). I found. FIG. 8 is a diagram showing the relationship between the RSA and the RRI sequence when a PSD similar to that in FIG. 5 is obtained for the subject in a resting state similar to step S1. In FIG. 8, the vertical axis indicates the heartbeat interval, and the horizontal axis indicates time. In FIG. 8, the data surrounded by a circle corresponds to the frequency at which the HF component in FIG. 5 becomes maximum (that is, the feature point of RSA), the amplitude of RSA is, for example, 50 ms, and the width of RSA is, for example, 3. 3 seconds (s). FIG. 9 is a diagram plotting the relationship between the RSA width and frequency, and FIG. 10 is a diagram plotting the relationship between the RSA amplitude and PSD. As can be seen from FIGS. 9 and 10, in the case of a resting subject, respiratory analysis (that is, the period and amplitude of the RRI sequence), RSA (that is, the frequency at which the HF component is maximized, and frequency analysis including PSD) It can be confirmed that the correlation with (result) is relatively high. In other words, it can be confirmed that the respiratory fluctuation corresponds to a frequency (characteristic point of RSA) at which PSD becomes a maximum.

The correlation between the maximum frequency F _{max at} which the HF component of the frequency analysis result is maximized and the minimum frequency F _min is as shown in FIG. 11, for example, and the maximum PSD _max and minimum PSD _min at the frequency at which the HF component of the frequency analysis result is maximized. The correlation is as shown in FIG. FIG. 11 is a graph plotting the relationship between the maximum frequency F _max and the minimum frequency F _min at which the HF component of the frequency analysis result is maximum, and FIG. 12 is the maximum at the frequency at which the HF component of the frequency analysis result is maximum. It is the figure which plotted the relationship between PSD _max and minimum PSD _min . In the example of FIG. 11, the correlation coefficient between the maximum frequency F _max and the minimum frequency F _min is about 0.81, and in the example of FIG. 12, the correlation coefficient between the maximum PSD _max and the minimum PSD _min is about 0.88. Therefore, if the correlation between the maximum frequency F _max and the minimum frequency F _{min in} the frequency analysis result and the correlation between the maximum PSD _max and the minimum PSD _min are used, the minimum frequency F _min is determined when the maximum frequency F _max is determined, and conversely the minimum frequency _{F min} is the highest frequency _{F max} that is determined can be confirmed and determined. Similarly, it can be confirmed that when the maximum PSD _max is determined, the minimum PSD _min is determined, and conversely, when the minimum PSD _min is determined, the maximum PSD _max is determined.

  Next, a method for estimating the position of sleepiness will be described. When the position of sleepiness is estimated in step S3, it may be performed in the same place or room as in step S1 or in a vehicle parked in an environment suitable for estimation. The drowsiness position corresponds to, for example, the lower left position surrounded by the solid line of the sleepiness scale shown in FIG. 13, and the awakening position corresponds to the upper right position surrounded by the solid line of the sleepiness scale. FIG. 13 is a diagram illustrating an example of the sleepiness scale. In FIG. 13, the vertical axis indicates the arousal level Y (or the arousal level axis Y) in arbitrary units, and the horizontal axis indicates the arousal level X (or the arousal level axis X) in arbitrary units. Further, in FIG. 13, a state in which both wakefulness level axes X and Y are high indicates the state of the most wakefulness (level 1), and a state in which both of the wakefulness level axes X and Y are low indicates the sleepy state (level 5). . The closer to level 5 indicating sleepiness, the closer to sleepiness from the awake state. The sleepiness scale as shown in FIG. 13, for example, is a frequency at which the HF component of the PSD shifts to a lower frequency as a result of examining how the frequency position at which the HF component of the PSD shifts is maximum. When the PSD corresponding to the position increases, the sleepiness level is generated based on the increase. Creation of such a sleepiness scale itself is well known, and for example, a technique described in Patent Document 4 may be used.

  In the sleepiness scale, in particular, it is desirable to accurately set the position of sleepiness (level 5 indicating sleepiness in FIG. 13). However, it takes time and effort to actually measure the position of different sleepiness for each driver. Take it. On the other hand, with the conventional technology, it is difficult to accurately estimate the position of sleepiness that is different for each driver. On the other hand, in this embodiment, by using the heartbeat waveform measured in the resting state, the frequency at which the high frequency component of the frequency analysis result is maximized is equal to or lower than the frequency at which the high frequency component in the sleepy state is maximized. Since the position of the driver's sleepiness is estimated by utilizing that the resulting PSD is equal to or higher than the PSD in the sleepiness state, the position of the driver's sleepiness can be estimated with high accuracy.

  FIG. 14 is a diagram for explaining the time transition of PSD obtained in a resting state, where the vertical axis indicates PSD in arbitrary units, and the horizontal axis indicates time in arbitrary units. FIG. 15 is a diagram for explaining a method for determining a drowsiness scale, where the vertical axis indicates the arousal level Y (or the arousal level axis Y) in arbitrary units, and the horizontal axis indicates the arousal level X (or the arousal level axis X). Shown in arbitrary units. As can be seen from FIG. 14, it can be confirmed that the PSD amplitude obtained in the resting state I is greater than or equal to the PSD amplitude obtained in the sleepy state II. In the example shown in FIG. 14, the PSD amplitude obtained in the rest state I is larger than the PSD amplitude obtained in the sleepiness state II.

The frequency at which the HF component of the PSD obtained in the resting state is maximized is set to the sleepiness position (or the origin of the sleepiness position) L5 of the sleepiness scale shown in FIG. 15, that is, the lower left position. As described with reference to FIGS. 9 to 12, if the correlation between the maximum frequency F _max and the minimum frequency F _{min and} the correlation between the maximum PSD _max and the minimum PSD _min in the frequency analysis result are used, the sleepiness scale awakens from the sleepiness position L5. The position L1, that is, the upper right position can be estimated indirectly.

  Next, a method for estimating the position of awakening will be described. The awakening position corresponds to, for example, the upper right position of the sleepiness scale shown in FIG. In this embodiment, the awakening position is estimated by simulating the awakening state during the actual driving of the driver in the simulated driving state.

  In the following explanation, an example of a simulated driving that simulates driving, in which the driver uses his own vision, makes a judgment based on the result of using the vision, and operates the operation unit of the device according to the judgment result. Show. For example, when the device is a vehicle, the operation unit includes a handle, a brake, an accelerator, and the like, and the operation unit can be operated by at least one of the driver's hand and foot. The simulated driving state can be generated by computer simulation or the like without the driver actually driving the vehicle. For example, a number as shown in FIG. 16 (“87” in this example) is displayed on the display unit of the personal computer, and the driver uses his / her vision by looking at the number displayed on the display unit. , Generate a state to make some judgment based on the result of utilizing vision by recognizing what the displayed number is, and input the number recognized by the driver from the keyboard of the personal computer Thus, it is possible to generate a state of operating the operation unit of the apparatus according to the determination result. In this way, in the simulated driving state, it is only necessary to give the driver the same load as during actual driving, so the simulation may be performed in the vehicle or in the place or environment where step S1 is performed. . When the simulation is performed in the vehicle, a computer of a navigation system mounted on the vehicle can be used instead of the personal computer, and numbers and the like may be input using a numeric keypad.

  FIG. 17 is a diagram for explaining an example of the frequency and PSD at which the HF component of the frequency analysis result at the time of the simulation operation becomes maximum. In FIG. 17, the numbers indicate the numbers displayed on the display unit, the frequency indicates the frequency at which the HF component is maximized in arbitrary units, and PSD indicates the logarithmic value of the value expressed in arbitrary units. As a result of investigating how the frequency position where the HF component of the PSD is maximized, for example, as a result of such a simulation, the frequency position where the HF component of the PSD is maximized is obtained. The drowsiness level decreases when the corresponding PSD decreases.

The frequency at which the HF component of the PSD that has shifted to a high frequency in such a simulation is maximized is set to the awake position (or the origin of the awake position) L1 of the sleepiness scale shown in FIG. 15, that is, the upper right position. As described with reference to FIGS. 9 to 12, if the correlation between the maximum frequency F _max and the minimum frequency F _{min and} the correlation between the maximum PSD _max and the minimum PSD _min in the frequency analysis result are used, the sleepiness scale sleepiness from the awakening position L1 is used. The position L5, that is, the lower left position can be estimated indirectly.

  When the position of awakening is estimated in step S3 (when it is not an indirect estimation using the correlation from the estimated position of sleepiness), the load is applied to the driver in the same load as during actual driving of the vehicle (ie, , During simulated operation). In this case, step S3 may be performed in the same place or room as step S1 described above, or may be performed in a parked vehicle in an environment suitable for estimation. The estimation in step S3 may be performed in a load state where the driver is actually driving the vehicle. However, the driver actually drives the vehicle in steps S1 to S3 and steps S4 and S5 described later. By performing before, it becomes possible to perform sleepiness determination based on the determined sleepiness scale from the start of actual driving.

  In step S4, based on the estimated drowsiness position and the estimated arousal position, a drowsiness scale in which the origin of the drowsiness position and the origin of the awakening position are set is determined.

  When setting the origin of the drowsiness position and the origin of the awakening position, the frequency at which the HF component of the frequency analysis result is maximized is relatively stable as shown in FIG. 18, but the HF component of the frequency analysis result is As can be seen from FIG. 19, the PSD at the maximum frequency may change greatly depending on the physical condition of the driver at that time. FIG. 18 is a diagram illustrating an example of an observation output when a near-infrared light, which is a pulse wave depending on a heartbeat, is applied to a finger and the reflected or transmitted light is optically acquired. The output (reflected light or transmitted light) is shown, and the horizontal axis shows time. FIG. 19 is a diagram illustrating an example of a PSD, where the vertical axis indicates PSD in arbitrary units, and the horizontal axis indicates frequency in arbitrary units. Therefore, if necessary, step S5 described below may be performed so that the PSD at the frequency at which the HF component of the frequency analysis result is maximized may be made relatively stable.

As shown in FIG. 20, the present inventor has found that there is a correlation between the PSD value and the TP of the autonomic nerve. FIG. 20 is a diagram showing an example of the correlation between PSD and TP, in which y = −421.93x + 26.389, R ² = 0.9144. The vertical axis represents TP in arbitrary units, and the horizontal axis represents PSD. Shown in arbitrary units. Here, R ² is the square of the correlation coefficient, and is used as a measure of the goodness of fit of the regression equation obtained from the sample value. In step S5, the sleepiness scale determined in step S4 is updated by calibrating (ie, normalizing) the PSD value obtained in the resting state. Specifically, in step S5, the PSD value obtained in the resting state is calibrated with the TP when the PSD is obtained, and the calibrated PSD value is set as the PSD value in the sleepy state. That is, step S5 is executed as necessary, and the sleepiness scale is updated by calibrating the PSD based on the TP when the PSD is obtained. The relationship between TP and PSD may be acquired by performing the process of estimating the position of the driver's drowsiness a plurality of times, and calibration for calculating PSD from TP may be performed. The relationship between TP and PSD is acquired by performing the process of estimating the position of awakening during the simulated driving (ie, the load state) of the driver a plurality of times, and the PSD during the simulated driving (ie, the load state) is obtained as TP. You may make it perform the calibration calculated from. When step S5 is executed, step S4 corresponds to provisional determination of the sleepiness scale.

  The TP of the autonomic nerve is, for example, the sum TP = LF + HF of the LF component (0.05 Hz to 0.15 Hz) and the HF component (0.15 Hz to 0.4 Hz) obtained by frequency analysis of the RRI time series. It may be a value of a heart rate variability coefficient (CVRR: Coefficient of Variance of RR interval) (that is, a value obtained by dividing the standard deviation of the heart rate by the average value). According to the results of experiments conducted by the present inventor, it is confirmed that there is a relatively high correlation between the value of PSD and the value of TP (LF component, HF component, etc.) of the autonomic nerve, as can be seen from FIG. The PSD value could be stabilized by calibration (ie, normalization) with the TP value.

  FIG. 21 is a block diagram illustrating an example of a functional configuration of a drowsiness determination device according to an embodiment. The drowsiness determination device 1 illustrated in FIG. 21 includes an RRI acquisition unit 11, a frequency analysis unit 12, an initial state setting unit 13, a drowsiness scale setting unit 14, and a drowsiness scale update unit 15. Each of the sensor 101, the storage unit 16, and the drowsiness level determination unit 17 that can electrically or optically measure the movement of the driver's heart forms sleepiness even if it forms part of the sleepiness determination device 1. The determination apparatus 1 may be externally connected. The functions of the RRI acquisition unit 11, the frequency analysis unit 12, the initial state setting unit 13, the drowsiness scale setting unit 14, the drowsiness scale update unit 15, and the drowsiness determination unit 17 are a processor such as a CPU (Central Processing Unit) as described later. Can be realized.

  The RRI acquisition unit 11 obtains heartbeat interval data in which RRI is time-series based on the driver's heartbeat waveform measured by the sensor 101. The frequency analysis unit 12 performs frequency analysis on the heartbeat interval data, and obtains a frequency analysis result of heartbeat fluctuation including PSD and autonomic nerve TP for the frequency at a certain time. The initial state setting unit 13 sets the frequency analysis result as an initial state. The RRI acquisition unit 11 executes the process of step S1 of FIG. 1, the frequency analysis unit 12 executes the process of step S2 of FIG. 1, and the initial setting unit 13 executes the process of step S3 of FIG.

  The drowsiness scale setting unit 14 sets a drowsiness scale in which the origin of the drowsiness position and the origin of the awakening position are set based on the set initial state, that is, the estimated drowsiness position and the estimated awakening position. decide. The sleepiness scale determined by the sleepiness scale setting unit 14 can be stored in the storage unit 16. The drowsiness scale update unit 15 obtains the TP during the simulated driving of the driver, calibrates the PSD based on the obtained TP, and updates the drowsiness scale. The sleepiness scale updated by the sleepiness scale update unit 15 can be stored in the storage unit 16. The sleepiness scale setting unit 14 executes the process of step S4 in FIG. 1, and the sleepiness scale update unit 15 executes the process of step S5 in FIG. 1 as necessary.

  After determining the sleepiness scale by the sleepiness scale setting unit 14 or after updating the sleepiness scale by the sleepiness scale update unit 15, the sleepiness determination unit 17 uses the sleepiness scale stored in the storage unit 16 to output the frequency from the frequency analysis unit 12. By analyzing the analysis result, it may be determined whether or not the driver is in a drowsiness state, and the determination result may be output. The processing of the drowsiness determination unit 17 may be started, for example, when the driver actually starts driving the vehicle. In this case, the vehicle-side processing includes the initial state setting unit 13, the drowsiness scale setting unit 14, and the sleepiness. The scale update unit 15 can be omitted. If the driver is in a drowsiness state, the drowsiness determination unit 17 may output at least one alarm from among, for example, display, sound, and vibration by a known method to alert the driver.

  For example, when the process of storing the drowsiness scale in the storage unit 16 is performed at a vehicle store, the storage unit 16 storing the drowsiness scale is connected so as to be accessible by the drowsiness determination unit 17 of the vehicle, so that the driver can actually The drowsiness determination process can be started when driving of the vehicle is started.

  FIG. 22 is a block diagram illustrating an example of a hardware configuration of the drowsiness determination device in one embodiment. The drowsiness determination device 1 illustrated in FIG. 22 includes a CPU 31, a storage device 32, an input device 33, a display device 34, a medium reading device 35, an interface (I / F) 36, and a sensor 101 connected by a bus 37. . The sensor 101 may be externally connected to the drowsiness determination device 1, and the sensor 101 can be connected to the I / F 36, for example. The connection between the CPU 31 and other parts of the drowsiness determination device 1 is not limited to the bus connection shown in FIG.

  The CPU 31 is an example of a processor that controls the entire sleepiness determination apparatus 1, and includes an RRI acquisition unit 11, a frequency analysis unit 12, an initial state setting unit 13, a sleepiness scale setting unit 14, a sleepiness scale update unit 15 illustrated in FIG. And the function of the sleepiness determination part 17 is realizable. The storage device 32 stores various data including a program executed by the CPU 31 and intermediate data of operations executed by the CPU 31. The storage unit 31 can be formed of a computer-readable storage medium including a magnetic recording medium, an optical recording medium, a magneto-optical recording medium, a semiconductor storage device, and the like. The program includes a program that causes the CPU 31 to execute a procedure of sleepiness determination processing of the sleepiness determination device 1. The storage device 32 may store a sleepiness scale.

  The input device 33 is used by the driver to input various instructions, data, and the like to the drowsiness determination device 1 and can be formed by, for example, a keyboard or a numeric keypad. The display device 34 is used to display various menus, messages, alarms, and the like to the driver, and can be formed of, for example, a liquid crystal display (LCD) panel. The input device 33 and the display device 34 may be formed by a touch panel having the functions of both the devices 33 and 34, for example.

  The medium reader 35 has a function of reading data from a computer-readable storage medium loaded in the drowsiness determination apparatus 1, and a storage medium for storing a drowsiness scale may be loaded.

  The I / F 36 forms an interface between the drowsiness determination device 1 and an external device. The I / F 36 has, for example, a USB (Universal Serial Bus) and may be connected to a storage medium that stores a drowsiness scale. When a storage medium such as a USB memory for storing the sleepiness scale is connected to the I / F 36, the medium reader 35 can be omitted. Further, the I / F 36 may have a wireless interface function that enables wireless communication between the drowsiness determination device 1 and an external device.

  The CPU 31 may be shared with another device such as a navigation system in addition to the sleepiness determination device 1. In this case, the storage device 32, the input device 33, the display device 34, the medium reading device 35, and the I / F 36 may also be shared with another device such as a navigation system. By sharing a part of the drowsiness determination device 1 with another device, resources (or resources) can be effectively used, and the cost of the drowsiness determination device 1 can be reduced.

  Next, estimation processing in the case of separately estimating both the position of sleepiness and the position of awakening in the sleepiness scale will be described with reference to FIGS.

  FIG. 23 is a flowchart for explaining an example of the sleepiness position estimation process executed by the CPU 31. In FIG. 23, in step S11, the heartbeat waveform of the driver is measured by the sensor 101 in a resting state before the driver actually drives the vehicle. In step S12, heartbeat interval data in which RRI is time-series based on the heartbeat waveform is obtained. In step S13, frequency analysis of heartbeat interval data is performed, and the frequency analysis result is set as an initial state. In step S14, the position of the driver's sleepiness in the sleepiness scale for determining sleepiness is estimated by the method as described above based on the set initial state, and the process ends.

  FIG. 24 is a flowchart for explaining an example of the awakening position estimation process executed by the CPU 31. In FIG. 24, in step S31, the heartbeat waveform of the driver is measured by the sensor 101 in a simulated driving state before the driver actually drives the vehicle. In step S22, heartbeat interval data in which RRI is time-series based on the heartbeat waveform is obtained. In step S23, frequency analysis of heartbeat interval data is performed, and the frequency analysis result is set as an initial state. In step S24, the position of the driver's awakening in the sleepiness scale for determining sleepiness is estimated by the method as described above based on the set initial state, and the process ends.

  FIG. 25 is a flowchart for explaining an example of sleepiness determination processing executed by the CPU 31. In FIG. 25, in step S31, the driver gets on the vehicle to be driven and, for example, a storage medium storing a drowsiness determination scale is connected to the I / F 36 of the drowsiness determination device 1 on the vehicle side. The driver starts driving the vehicle at an arbitrary timing after getting on the vehicle. In step S32, the sleepiness determination scale is read from the storage medium. In step S33, the heartbeat waveform of the driver is measured by the sensor 101, and the frequency analysis result of the heartbeat interval data obtained by time-series RRI based on the heartbeat waveform is within level 5 indicating the sleepiness state of the sleepiness scale of FIG. If the level is within level 5, drowsiness is determined and an alarm is output.

  In step S34, it is determined whether or not the sleepiness scale needs to be updated. For example, if the position of drowsiness or awakening estimated from the frequency analysis result is deviated from the drowsiness position or awakening position of the sleepiness scale by a threshold or more, the determination result in step S34 is YES. If the decision result in the step S34 is YES, a TP at the time of actual driving of the driver is obtained in a step S35, and the PSD of the frequency analysis result is calibrated based on the obtained TP to update the sleepiness scale. In this case, since PSD calibration based on TP during actual driving is performed, the drowsiness scale can be further optimized compared to PSD calibration based on TP during simulated driving. In step S35, the updated sleepiness scale is stored in the storage medium. The determination result in step S34 is NO, or after step S35, in step S36, it is determined by a known method whether the driver has got off the vehicle. For example, if the seat is equipped with a sensor that detects the seating of a person in order to urge the user to wear a seat belt, if the sensor no longer detects the seating of the person, the driver gets out of the vehicle. It can be detected. If the determination result of step S36 is NO, the process returns to step S33, and if the determination result is YES, the process ends.

  A pulse may be used instead of the heartbeat. That is, the sensor 101 is not particularly limited as long as it can measure data representing the heartbeat waveform of the subject, and for example, an electrocardiograph, a pulse wave meter, a heart sound sensor, a pulse sensor, or the like may be used.

  FIG. 26 is a diagram illustrating an example of a sensor. In the example of FIG. 26, the sensor 101 is built in the vehicle handle 200 and detects the pulse from the hand of the driver holding the handle 200.

  FIG. 27 is a diagram illustrating another example of the sensor. In the example of FIG. 27, the sensor 101 is worn on the earlobe of the driver 300 and detects a pulse from the earlobe. The output of the sensor 101 is connected to the drowsiness determination device 1 through the wire 102.

  When the I / F 36 has a wireless interface function that enables wireless communication between the drowsiness determination device 1 and an external device, the sensor 101 may be incorporated in the external device having a wireless communication function. In this case, the external device can be formed, for example, in a wristwatch shape, and the pulse detected by the sensor 101 can be input to the drowsiness determination device 1 via the wireless communication function of the external device and the wireless interface function of the I / F 36.

  Thus, in order to determine sleepiness with high accuracy, a different sleepiness scale for each driver can be set in advance (that is, before the driver actually starts driving). Can perform sleepiness determination using the set sleepiness scale immediately. Further, if PSD calibration based on TP during actual driving is performed after driving is started, the drowsiness scale can be further optimized as compared with PSD calibration based on TP during simulated driving.

  Needless to say, drowsiness detection can be performed not only for a driver of a general vehicle but also for an operator of a device requiring a driving operation or a steering operation such as a heavy machine, a train, a ship, and an aircraft.

  By the way, when the sleepiness scale is determined in advance at, for example, a vehicle dealer, the sleepiness determination device 1 on the dealer side can omit the function of the sleepiness determination unit 17 shown in FIG. When the function is omitted, the sleepiness determination device 1 has the functions of a sleepiness scale creation device, a sleepiness scale setting device, or a sleepiness scale determination device. In this case, the functions of the initial state setting unit 13 and the sleepiness scale setting unit 14 can be omitted in the drowsiness determination device 1 on the vehicle side. Furthermore, if it is not necessary to update the sleepiness scale in the drowsiness determination device 1 on the vehicle side, the function of the sleepiness scale update unit 15 can be omitted. When the store-side drowsiness determination device 1 (for example, the drowsiness scale creation device) stores the storage medium storing the drowsiness scale in the vehicle-side drowsiness determination device 1, the vehicle-side drowsiness determination device 1 has a relatively simple configuration. The driver's sleepiness can be determined with high accuracy.

The following supplementary notes are further disclosed with respect to the embodiments including the above examples.
(Appendix 1)
A sleepiness determination method for determining a subject's sleepiness based on a sleepiness scale,
Based on the heartbeat waveform of a resting subject whose breathing is stable measured by the sensor, RRI (RR Interval) is time-sequentially determined to obtain heartbeat interval data.
Frequency analysis of the heartbeat interval data to obtain a frequency analysis result of heartbeat fluctuation including a power spectral density (PSD) and a total power (TP) of the autonomic nerve with respect to a frequency at a certain time,
Set the frequency analysis result as an initial state,
Processing for determining a drowsiness scale in which the origin of the drowsiness position and the origin of the awakening position are set based on the estimated drowsiness position and the estimated awakening position included in the initial state and storing them in a storage medium A drowsiness determination method characterized in that the computer executes.
(Appendix 2)
The computer further executes a process of obtaining a TP when a load is applied to the subject, calibrating the PSD based on the obtained TP, updating the sleepiness scale, and storing it in the storage medium. The drowsiness determination method according to Supplementary Note 1, wherein:
(Appendix 3)
A process of analyzing the frequency analysis result during actual driving of the subject using the sleepiness scale stored in the storage medium to determine whether the subject is in a sleepy state and outputting the determination result. The sleepiness determination method according to appendix 1 or 2, further comprising:
(Appendix 4)
When the position of sleepiness or arousal estimated from the frequency analysis result during actual driving of the subject is more than a threshold value from the position of sleepiness or awakening of the sleepiness scale, the TP during the actual driving of the subject is obtained. The sleepiness determination method according to appendix 3, wherein the computer further executes a process of calibrating the PSD based on the obtained TP, updating the sleepiness scale, and storing the updated sleepiness scale in the storage medium.
(Appendix 5)
A drowsiness determination device that determines a subject's drowsiness based on a drowsiness scale,
An RRI acquisition unit for obtaining heart rate interval data in which RRI (RR Interval) is time-series based on a heartbeat waveform of a subject in a stable state of respiration measured by a sensor;
A frequency analysis unit for frequency-analyzing the heartbeat interval data to obtain a frequency analysis result of heartbeat fluctuation, including a power spectral density (PSD) for a frequency at a certain time and a total power (TP) of an autonomic nerve;
An initial state setting unit for setting the frequency analysis result as an initial state;
Based on the estimated drowsiness position and the estimated arousal position included in the initial state, a drowsiness scale in which the origin of the drowsiness position and the origin of the awakening position are set is determined and stored in the storage medium A drowsiness determination device comprising a scale setting unit.
(Appendix 6)
A sleepiness scale update unit that obtains a TP when a load is applied to the subject, calibrates the PSD based on the obtained TP, updates the sleepiness scale, and stores the sleepiness scale in the storage medium; The drowsiness determination device according to appendix 5.
(Appendix 7)
A sleepiness determination unit that analyzes a frequency analysis result during actual driving of the subject using the sleepiness scale stored in the storage medium to determine whether the subject is in a sleepy state and outputs a determination result; The drowsiness determination device according to appendix 5 or 6, further comprising:
(Appendix 8)
The drowsiness determination unit, when the position of sleepiness or wakefulness estimated from the frequency analysis result during actual driving of the subject deviates from the sleepiness position or wakefulness position of the sleepiness scale by a threshold value or more, The sleepiness determination apparatus according to appendix 7, wherein the TP during driving is obtained, the PSD is calibrated based on the obtained TP, the sleepiness scale is updated, and the sleepiness scale is stored in the storage medium.
(Appendix 9)
A program for causing a computer to execute processing for determining sleepiness of a subject based on a sleepiness scale,
A procedure for obtaining heart rate interval data in which RRI (RR Interval) is time-series based on a heart rate waveform of a subject in a stable state of respiration measured by a sensor;
A step of performing frequency analysis of the heartbeat interval data to obtain a frequency analysis result of heartbeat fluctuation including a power spectral density (PSD) and a total power (TP) of an autonomic nerve with respect to a frequency at a certain time;
A procedure for setting the frequency analysis result as an initial state;
A procedure for determining a drowsiness scale in which the origin of the drowsiness position and the origin of the awakening position are set based on the estimated drowsiness position and the estimated awakening position included in the initial state and storing them in a storage medium That causes the computer to execute the program.
(Appendix 10)
Obtaining the TP when a load is applied to the subject, calibrating the PSD based on the obtained TP, updating the drowsiness scale, and storing the storage medium in the storage medium. The program according to appendix 9.
(Appendix 11)
Analyzing the frequency analysis result of the subject during actual driving using the sleepiness scale to determine whether or not the subject is in a sleepiness state, and causing the computer to further execute a procedure of outputting the determination result. The program according to appendix 9 or 10.
(Appendix 12)
When the position of sleepiness or arousal estimated from the frequency analysis result during actual driving of the subject is more than a threshold value from the position of sleepiness or awakening of the sleepiness scale, the TP during the actual driving of the subject is obtained. The sleepiness determination method according to appendix 11, further comprising: causing the computer to further execute a procedure of calibrating the PSD based on the obtained TP, updating the sleepiness scale, and storing the updated sleepiness scale in the storage medium.

  Although the disclosed drowsiness determination method, apparatus, and program have been described in the above embodiments, the present invention is not limited to the above embodiments, and various modifications and improvements can be made within the scope of the present invention. Needless to say.

DESCRIPTION OF SYMBOLS 1 Sleepiness determination apparatus 11 RRI acquisition part 12 Frequency analysis part 13 Initial state setting part 14 Sleepiness scale setting part 15 Sleepiness scale update part 16 Storage part 17 Sleepiness determination part 31 CPU
32 Storage device 35 Reading device 36 I / F
101 sensor

Claims (5)

A program for causing a computer to execute processing for determining sleepiness of a subject based on a sleepiness scale,
A procedure for obtaining heart rate interval data in which RRI (RR Interval) is time-series based on a heart rate waveform of a subject in a stable state of respiration measured by a sensor;
A step of performing frequency analysis of the heartbeat interval data to obtain a frequency analysis result of heartbeat fluctuation including a power spectral density (PSD) and a total power (TP) of an autonomic nerve with respect to a frequency at a certain time;
A procedure for setting the frequency analysis result as an initial state;
A procedure for determining a drowsiness scale in which the origin of the drowsiness position and the origin of the awakening position are set based on the estimated drowsiness position and the estimated awakening position included in the initial state and storing them in a storage medium That causes the computer to execute the program. Obtaining the TP when a load is applied to the subject, calibrating the PSD based on the obtained TP, updating the drowsiness scale, and storing the storage medium in the storage medium. The program according to claim 1. Analyzing the frequency analysis result of the subject during actual driving using the sleepiness scale to determine whether or not the subject is in a sleepiness state, and causing the computer to further execute a procedure of outputting the determination result. The program according to claim 1 or 2. A sleepiness determination method for determining a subject's sleepiness based on a sleepiness scale,
Based on the heartbeat waveform of a resting subject whose breathing is stable measured by the sensor, RRI (RR Interval) is time-sequentially determined to obtain heartbeat interval data.
Frequency analysis of the heartbeat interval data to obtain a frequency analysis result of heartbeat fluctuation including a power spectral density (PSD) and a total power (TP) of the autonomic nerve with respect to a frequency at a certain time,
Set the frequency analysis result as an initial state,
Processing for determining a drowsiness scale in which the origin of the drowsiness position and the origin of the awakening position are set based on the estimated drowsiness position and the estimated awakening position included in the initial state and storing them in a storage medium A drowsiness determination method characterized in that the computer executes. A drowsiness determination device that determines a subject's drowsiness based on a drowsiness scale,
An RRI acquisition unit for obtaining heart rate interval data in which RRI (RR Interval) is time-series based on a heartbeat waveform of a subject in a stable state of respiration measured by a sensor;
A frequency analysis unit for frequency-analyzing the heartbeat interval data to obtain a frequency analysis result of heartbeat fluctuation, including a power spectral density (PSD) for a frequency at a certain time and a total power (TP) of an autonomic nerve;
An initial state setting unit for setting the frequency analysis result as an initial state;
Based on the estimated drowsiness position and the estimated arousal position included in the initial state, a drowsiness scale in which the origin of the drowsiness position and the origin of the awakening position are set is determined and stored in the storage medium A drowsiness determination device comprising a scale setting unit.

Images