JP2009172292A - Estimating device for human state - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an estimating device for a human state for estimating a state of a human with high accuracy. <P>SOLUTION: The estimating device 1 for a human state for estimating the state of a subject includes state variation obtaining means 3, 4, 7 for obtaining a state variation of the subject, a state estimating means 7 for estimating the state of the subject from the state variation, a stable state determining means 7 for determining whether the state of the subject is stable or not, and a correcting means 7 for correcting the state variation of the subject or the state of the subject when the state of the subject is determined to be stable. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a human state estimating apparatus that estimates a human state with high accuracy.

Various apparatuses for estimating a human state such as an arousal state, a psychological state, and a fatigue state have been proposed for use for driver assistance. For example, in the human state estimation device described in Patent Document 1, a stimulus is given to the subject, the current state change amount of the subject is obtained from the physiological response to the stimulus, and the current state change amount is set to the previous awakening level. In consideration of this, the current awakening level is estimated.
Japanese Patent Application No. 2006-259204

  When the subject is relaxed and the state is stable, even if a stimulus is given, the physiological reaction or the like is not changed, and the state change amount is also not changed. For this reason, in the device described in Patent Document 1, since the awakening level is estimated based on the relative change in the state change amount, the estimated awakening level is not changed when the subject is in a stable state. As a result, if an error is included in the estimated wakefulness level, the error remains as a stationary error in the wakefulness level until the state of the subject changes.

  Then, this invention makes it a subject to provide the human state estimation apparatus which estimates a human state with high precision.

  The human state estimation device according to the present invention is a human state estimation device that estimates a subject's state, and is a state change acquisition unit that acquires the state change amount of the subject and a state acquired by the state change amount acquisition unit State estimation means for estimating the state of the subject based on the amount of change, stable state determination means for determining whether or not the subject's state is in a stable state, and when the stable state determination means determines that the state is stable, And correction means for correcting the quantity or the condition of the subject.

  In this human state estimation device, the state change amount of the subject is acquired by the state change amount acquisition means, and the state of the subject is estimated based on the state change amount by the state estimation means. The state change amount is an index indicating a change in the state of the subject obtained from the physiological reaction or behavioral response of the subject, and is, for example, an SPR value (and thus an AR value) described in the embodiment. The state of the subject is, for example, an arousal state (sleepiness state), a psychological state (impression, irritation, boredom, etc.), a fatigue state, concentration, attention, and stress. In particular, in the human state estimation device, it is determined whether or not the state of the subject is a stable state by the stable state determination means. When a stable state is reached, there may be no change in the detected state change amount or the estimated state of the subject, and a stationary error may remain. Therefore, in the human state estimation device, when the subject is in a stable state, the state change amount of the subject (and thus the state of the subject) or the state of the subject is corrected by the correcting means. As described above, in the human state estimation device, when the subject becomes stable, the state change amount or the state is corrected so that the steady state error can be removed or reduced, and the human state is estimated with high accuracy. Can do.

  The human state estimation apparatus of the present invention includes state change amount density acquisition means for acquiring the state change amount density from the state change amount of the subject acquired by the state change amount acquisition means, and the correction means is the stable state determination means. If it is determined that the state is stable, the state of the subject is estimated based on the state change density of the subject acquired by the state change density acquisition means, and is corrected using the state of the subject estimated based on the state change density. It is good also as a structure.

  In this human state estimation device, the state change amount density acquisition means acquires the state change amount density (for example, the SPR frequency described in the embodiment) from the state change amount of the subject. Then, in the human state estimation device, when the subject is in a stable state, the state of the subject is estimated based on the state change density of the subject, and correction is performed based on the estimated state of the subject. As described above, in the human state estimation device, the state change amount density when the subject is in a stable state can be used to perform absolute estimation of the state of the subject. Since the correction is performed based on the absolute estimated value, it is possible to remove or reduce the steady-state error included in the estimated sleepiness level.

  The present invention can estimate a human state with high accuracy by correcting a state change amount or a state when the subject is in a stable state.

  Embodiments of a human state estimation device according to the present invention will be described below with reference to the drawings.

  In the present embodiment, the human state estimation device according to the present invention is applied to a state estimation system that is mounted on a vehicle and estimates a driver's drowsiness level (wake state). In the state estimation system according to the present invention, a reference stimulus is given to the driver, and the driver's physiological response to the reference stimulus is an electrodermal activity (EDA [Electro Dermal Activity]) (especially, skin potential response (SPR [Skin Potential Response])) is detected, the AR value is calculated based on the SPR value (corresponding to the state change amount) by the AR [Active Reference] method, and the sleepiness level (corresponding to the state) is estimated based on the AR value. In the state estimation system according to the present invention, when the drowsiness level is a level that causes even a little trouble in driving, an arousal stimulus is given to alert the driver according to the drowsiness level. In the AR method, a reference stimulus (vibration, etc.) is given to a person at regular intervals, and the state of the person is determined based on a relative change for each section of the person's physiological response obtained for the stimulus ( This is a method for estimating an arousal state or the like.

  With reference to FIGS. 1-10, the state estimation system 1 which concerns on this Embodiment is demonstrated. FIG. 1 is a configuration diagram of a state estimation system according to the present embodiment. FIG. 2 is an example of temporal changes in the estimation start timing, reference stimulus, wakefulness stimulus, SPR value, and AR value according to the present embodiment. FIG. 3 is a diagram showing a correspondence relationship between the drowsiness level (wake level) and the AR value according to the present embodiment. FIG. 4 is an example of a time change of the SPR value with respect to the reference stimulus according to the present embodiment. FIG. 5 is an example of a time change of the SPR frequency according to the present embodiment. FIG. 6 is an example of the time change of the sleepiness level estimated by the sleepiness estimation model when the stable state is reached, the sleepiness level by subjective evaluation, and the AR value. FIG. 7 is an example of a time change of the AR value, the AR ′ value, and the ARR value according to the present embodiment. FIG. 8 is an example of a correspondence relationship between the SPR frequency and the sleepiness level in the stable state. FIG. 9 is an example of a calibration interval for performing absolute estimation of the sleepiness level based on the SPR frequency according to the present embodiment. FIG. 10 is an example of a time change of the sleepiness level estimated value by the sleepiness estimation model in the actual vehicle test, the sleepiness level correction value when the correction process is performed, and the sleepiness level sensory evaluation value.

  The state estimation system 1 estimates the current sleepiness level based on the AR value obtained this time and the sleepiness level estimated last time by using the sleepiness estimation model. In particular, in the state estimation system 1, it is determined whether or not the driver is in a stable state in order to improve estimation accuracy, and in the case of the stable state, the SPR is used to reduce (remove) the steady state error of the estimated sleepiness level. The sleepiness level is corrected based on the absolute estimated sleepiness level obtained from the frequency (corresponding to the state change amount density).

  Here, before describing a specific configuration of the state estimation system 1, a sleepiness level estimation method and a sleepiness level correction method using the sleepiness estimation model in the state estimation system 1 will be described. In addition, about a sleepiness estimation model, although the 1st-3rd sleepiness estimation model is demonstrated, any model is applicable.

First, a sleepiness level estimation method using a sleepiness estimation model will be described. As shown in FIG. 2, when the state estimation is started, a reference stimulus is given to the driver at timings T ₁ , T ₂ ,... At a certain period (for example, every 30 seconds). Each time a reference stimulus is applied, EDA (especially, skin potential activity (SPA)) is detected as a physiological index for the reference stimulus, and a direct current component (skin potential level (SPL [Skin Potential Activity) is detected from the detected SPA. Level])) is removed, and the component obtained by removing the direct current component (SPL) from the SPA is the component of the SPR, and further, the predetermined processing section of the SPR component (spr (t)) according to the equation (1). The time integral value (SPR (t)) at (the processing time T immediately after the reference stimulus is applied) is calculated, which is the SPR value for the reference stimulus, as shown in FIG. S _{1, S} 2, by ..., shows an example of the detected SPR value when given the reference stimulus at each timing _{T 1, T} 2,.. The processing time T is set in consideration of the response characteristic of the physiological index (SPR) to the stimulus, etc. As the SPR value, in addition to the time integral value, the maximum value of the SPR component in a predetermined processing interval The difference between the minimum values (absolute maximum value: Peak-to-Peak in the processing section) or the average value of the predetermined processing section of the SPR component may be used.

Every time the SPR value is detected, the AR value (AR (t)) of the SPR is calculated by the equation (2) using the SPR (t) detected this time and the SPR (t-1) detected last time. The AR value is a normalized value of the relative change in the SPR value, and is a value based on 0. When the AR value is 0, there is no relative change. The AR value correlates with the drowsiness level, and it can be estimated that the drowsiness is weaker (the arousal state is higher) as it is larger than 0, and that the drowsiness is stronger (the awake state is lower) as it is smaller than 0. In addition, since the relative change is normalized, the AR value is updated sequentially and is a slow reaction index.

In Equation (2), AR (t) is the AR value for the current reference stimulus, SPR (t) is the SPR value for the current reference stimulus, and SPR (t−1) is the SPR value for the previous reference stimulus. It is. The time interval between t-1 and t is a reference stimulus application interval. FIG. 2 shows an example of the AR value obtained from the preceding and following SPR values by broken lines A ₂ , A ₃ ,. In this example, AR (2) is smaller than 0 and the awake state is low, and AR (3) is larger than 0 and the awake state is high.

When sleepiness is strong (when the state of wakefulness is low), in order to increase the driver's state of wakefulness, a wakeful stimulus is given to the driver at timing _T3S immediately after it is determined that sleepiness is strong. Each time the arousal stimulus is applied, the SPR value and the AR value are obtained in the same manner as when the reference stimulus is applied. In FIG. 2, when given the SPR value detected when given arousal stimulated with timing T _3S, arousal stimulated with timing T _3S by the dashed line A _3s to end point a white circle with a solid line S _3s to endpoint white circles An example of the detected AR value is shown in FIG.

A sleepiness estimation model for estimating the sleepiness level based on the AR value is considered using the correlation between the AR value and the sleepiness level. The sleepiness estimation model is a linear model in which the AR value obtained this time is added to the sleepiness level estimated last time. Equation (3) shows the first sleepiness estimation model that is the basis of the sleepiness estimation model.

  In Equation (3), DL (t) is the current sleepiness level, AR (t) is the AR value for the current reference stimulus, and DL (t−1) is the previous sleepiness level. In the present embodiment, the sleepiness level (wakefulness level) is expressed in the range of 6 to 1 (D0 to D5), 6 (D0) is the weakest sleepiness (the highest awakening state), and 1 (D5) is sleepiness. Is the strongest (lowest wakefulness). On the other hand, the AR value is a value based on 0, the range of the value varies depending on the situation, and the absolute amount is unknown. Therefore, the sleepiness level and the range of the AR value are different, and parameters of different ranges are used in the sleepiness estimation model. Therefore, it is necessary to adjust the sleepiness level and the range of the AR value before estimating the state.

In the present embodiment, as shown in FIG. 3, a calibration interval (several minutes) for adjusting the range is obtained after stable running (skillful running) immediately after starting running (several tens of seconds to several minutes). Degree) and then state estimation is performed. First, when traveling is started, a reference stimulus is given at timings T ₁ , T ₂ ,. Each time a reference stimulus is applied, an SPR value is obtained, and an AR value is further obtained. Here, since it can be assumed that the driver is in a high arousal state immediately after the start of traveling, it is assumed that the drowsiness level (wake level) is 6 to 5 (D0 to D1) in the calibration section. Then, the fluctuation range is obtained from the time change AC of the AR value in the calibration section. Since 6-5 sleepiness level corresponding to the fluctuation range _6-5, fluctuation range _{5 to 1} (= variation range _6-5 × the AR value of 5 to 1 of drowsiness level according to the variation range _6-5 4) to determine the range of AR values corresponding to sleepiness levels of 5-1. In the example of FIG. 3, the AR value is 0.2 to _{−0.8 in} the calibration section, and the fluctuation range 6 to ₅ is 1.0. Accordingly, the fluctuation range 5 to 1 of the AR value of the sleepiness level 5 _{to 1} is 4.0, and the range of the AR value corresponding to the sleepiness level 5 to 1 is −0.8 to −4.8. The range of AR values corresponding to the entire level is 0.2 to -4.8. Note that the range correspondence relationship between the sleepiness level and the AR value varies depending on various factors such as the driver's condition, traveling conditions, weather, and the like, and thus the range adjustment needs to be performed for each traveling.

  In addition, since the sleepiness estimation model uses the previous sleepiness level, an initial value of the sleepiness level is required. Therefore, immediately after the start of traveling, it is assumed that the driver is in a high arousal state, and the initial value DL (0) of the drowsiness level is set to 6 (D0).

  When the SPR (SPR amplitude) is small, the SPR greatly fluctuates even under the influence of slight noise (for example, road noise, wind noise, passenger's behavior) due to the external environment, compared to when it is large. When the SPR fluctuates greatly, the AR value obtained from the relative change of the preceding and following SPR values also fluctuates greatly. In the first sleepiness estimation model, since the AR value is taken into account as it is, if the AR value greatly fluctuates due to the influence of noise or the like as described above, the estimated drowsiness level DL (t) also fluctuates greatly due to the direct influence. . Further, when the driver is sensitive to the stimulus, the SPR fluctuates greatly because the driver reacts excessively even under the influence of a little noise. Also in this case, similarly to the above, the estimated sleepiness level DL (t) is directly affected and greatly fluctuates. Furthermore, since the first drowsiness estimation model is a linear model, a steady error occurs. Therefore, if an error is included in the previous sleepiness level DL (t−1), the error is accumulated and the steady-state error is also increased.

In the first drowsiness estimation model, since the degree of reflection of the AR value to the estimated drowsiness level DL (t) is always maximized, the estimation accuracy may decrease due to the influence of the noise described above. Therefore, in order to be able to adjust the degree of reflection of the AR value, a second drowsiness estimation model was constructed in which the degree of consideration of the AR value is variable according to the gain K, as shown in Equation (4).

  In Equation (4), K is a gain for determining the degree of reflection of the AR value for the current reference stimulus. The gain K is a value from 0 to 1, and is a three-stage value in the present embodiment. The gain K is set based on the SPR frequency. The SPR frequency (den (t)) is a frequency per unit time of the SPR, and is an index indicating the degree of fluctuation of the SPR component. A method of obtaining the SPR frequency will be described with reference to FIG. First, every time the reference stimulus is applied, the time when the mean time (mean value) + 3 × σ (standard deviation value) is larger than the mean-3 × σ is counted in the processing time T of the SPR component, The count value is set as the SPR frequency. As mean and σ, values obtained as the average value and the standard deviation value of the SPR component in the processing time T of the SPR component when the previous reference stimulus is applied are used. The greater the SPR frequency, the greater and faster the SPR variation. By the way, when the amplitude of the previous SPR component is small as a whole, the standard deviation value is small. Therefore, even if the driver reacts due to a slight noise, the next SPR frequency increases. Moreover, since the SPR frequency directly represents the degree of fluctuation of the SPR component, it is a fast reaction index.

  When the SPR frequency is high, the state is a sympathetic activation state, the behavior state is low sleepiness (high arousal state), the activity level is high, the psychological state is high tension, and the state is excited. In such a case, since it is sensitive as a response to the stimulus, it reacts to a slight noise (highly likely to pick up noise), and the AR value may fluctuate greatly. In this case, since the influence of noise on the sleepiness level is increased, the gain K is decreased and the reflection degree of the AR value is decreased. On the other hand, when the SPR frequency is low, drowsiness is high as the action state (the awake state is low), the activity level is low, the tension state is low as the psychological state, and the state is relaxed. In such a case, since the response to the stimulus is insensitive, the response to noise is also slow. Therefore, the gain K is increased and the reflection degree of the AR value is increased.

  A method for setting the gain K based on the SPR frequency will be described with reference to FIG. Here, two thresholds, th1 and th2, are prepared for the SPR frequency. The thresholds th1 and th2 are set in consideration of the response characteristics of the physiological index (SPR) to the stimulus, and th2> th1. When the SPR frequency is less than th1, the gain K = A is set. When the SPR frequency is greater than or equal to th1 and less than th2, the gain K = B is set. When the SPR frequency is greater than or equal to th2, the gain K = C is set. A, B, and C are set by experiment etc., and are values from 0 to 1, and A> B> C. In the example shown in FIG. 5, the gain K = A when the SPR frequency is F2 and F5, the gain K = B when the SPR frequency is F4, and the gain when the SPR frequency is F1 and F3. The gain K = C.

As described above, a wakeful stimulus is given to the driver when sleepiness becomes strong, but in the second sleepiness estimation model, the driver's response to the wakefulness stimulus is not reflected in the estimated sleepiness level. Therefore, in order to reflect the effect of the arousal stimulus, a third drowsiness estimation model was constructed in which a term that takes into account the AR value for the arousal stimulus was added, as shown in Equation (5).

In Equation (5), AR (t _s ) is an AR value for the applied wakefulness stimulus, and τ is a gain for determining the degree of reflection of the AR value for the wakefulness stimulus. The gain τ is a value from 0 to 1 (a value smaller than 1), and is a two-stage value in the present embodiment. The gain τ is also set based on the SPR frequency.

  When an arousal stimulus is given to the driver, if the SPR frequency is high, it indicates that the arousal state is increased by the arousal stimulus (that is, the effect of the arousal stimulus is present), and the driver's It shows that the response to the stimulus is fast. In this case, it is necessary to largely reflect the effect of the arousal stimulus on the state estimation. In addition, it is necessary to quickly reflect the change in the driver's fast state in the state estimation so that the state estimation is not delayed with respect to the change in the driver's state. Therefore, when the SPR frequency for the wakeful stimulus is high, the gain τ is increased, and the degree of reflection of the AR value for the wakeful stimulus is increased.

  A method for setting the gain τ based on the SPR frequency will be described with reference to FIG. Here, one of th3 is prepared as a threshold for the SPR frequency. The threshold th3 is set in consideration of the response characteristic of a physiological index (SPR) to a stimulus, and th3> th2> th1. When the SPR frequency is less than th3, the gain τ = α is set. When the SPR frequency is greater than or equal to th3, the gain τ = β is set. α and β are set by experiment or the like, and are values from 0 to 1, where α <β. In the example shown in FIG. 5, when the SPR frequency is F1, F2, F4, and F5, the gain τ = α, and when the SPR frequency is F3, the gain τ = β.

  Next, a method for correcting the drowsiness level will be described. When a person relaxes and becomes stable, even if a stimulus is given, there is no change in physiological responses such as skin potential response (SPR). For this reason, since the above sleepiness estimation model uses the AR value, which is a relative change in the SPR value, as shown in FIG. 6, when the subject (for example, the driver) is in a stable state, the AR value does not change. (AR value in broken-line rectangle SC), there is no change in the sleepiness level LE estimated by the sleepiness estimation model. In addition, since the sleepiness estimation model takes into account the sleepiness level estimated last time, if an error is included in the estimated sleepiness level, the error remains as a stationary error.

  FIG. 6 also shows the sleepiness level LS subjectively determined by the subject. When the subject becomes stable, the sleepiness level LE estimated by the sleepiness estimation model is compared with the sleepiness level LS subjectively evaluated by the subject. There is a difference SE between them. Thus, the estimated drowsiness level LE deviates from the drowsiness level LS felt by the subject by a substantially constant amount, and a steady error occurs. Therefore, when the driver is in a stable state (when there is no change in the AR value), it is necessary to correct the sleepiness level estimated by the sleepiness estimation model in order to reduce (remove) the steady state error.

In order to determine the change of the AR value, every time the AR value is calculated, the AR ′ value is calculated by the equation (6) using the AR value (see FIG. 7). The AR ′ value is a value obtained by adding 1 to the AR value, and is a value of 0 or more. Furthermore, using the past six AR ′ values including the current AR ′ value, the ARR value is calculated by Equation (7) (see FIG. 7). The ARR value is an index indicating a change in the AR value. When the value is 1, there is no change in the AR value. Incidentally, since the EDA reaction has a very good time resolution, it is easy to pick up noise. Therefore, when obtaining the ARR value, the ARR value is calculated using six AR values (including the AR value used when the ARR value was calculated in the past).

  Each time the ARR value is calculated, it is determined whether or not the condition of 0.8 <ARR (t) <1.2 is satisfied. When this condition is satisfied, it is determined that the AR value has not changed (that is, the driver is in a stable state), and the sleepiness level is corrected. In the case of the example shown in FIG. 7, ARR (t−4),..., ARR (t) satisfies the above conditions, and the driver's state is stable at this time.

  FIG. 8 shows the SPR frequency at each sleepiness level (sensory evaluation value) when the subject is in a stable state, and the average value of the SPR frequency is represented by solid lines AL1 and AL2. In the example of FIG. 8, two subjects are shown, but the two subjects show the same tendency as can be seen from the solid lines AL1 and AL2. As for this tendency, when the arousal is high, the SPR frequency is high, and when the arousal is low, the SPR frequency is low, and the absolute amount of the SPR frequency is almost the same. Based on the relationship between the SPR frequency and the drowsiness level in the stable state, when the relationship between the SPR frequency and the drowsiness level until the level of arousal is lower is predicted, prediction lines PL1 and PL2 as indicated by a one-dot chain line are obtained. It is done. The prediction line PL1 and the prediction line PL2 are similar prediction lines. Furthermore, even when the same experiment was performed on many other subjects, the relationship between the SPR frequency and the drowsiness level showed a similar tendency, and a similar prediction line was obtained.

As described above, when a person is in a stable state, the relationship between the SPR frequency and the drowsiness level (wake level) is common to all people and is proportional. Therefore, using this relationship, the sleepiness level is absolutely estimated from the SPR frequency. Formula (8) shows a formula for calculating the absolute estimated sleepiness level DLR value using the SPR frequency obtained from the experimental results of many subjects as a variable. In Expression (8), DLR (t) is the absolute relative value of the current sleepiness level, and den (t) is the SPR frequency calculated this time.

  As described above, since the EDA reaction has a considerably good time resolution, it is easy to pick up noise. Therefore, when the sleepiness level is absolute estimated using only the local SPR frequency, the accuracy of the absolute estimated value may be reduced. Therefore, a calibration interval having a certain time width is set, an average value of a plurality of SPR frequencies in the calibration interval is obtained, and an absolute estimated sleepiness level DLR value is obtained from the average value of the plurality of SPR frequencies. Thus, the absolute estimated sleepiness level DLR value is calculated using the SPR frequency obtained by smoothing a large number of SPR frequencies in the calibration section.

In the present embodiment, as shown in FIG. 9, the calibration interval is set for 5 minutes, and three overlapping SPR frequency measurement intervals (1), (2), and (3) are performed. Then, average values of the six SPR frequencies calculated every 30 seconds in each section are respectively obtained, and absolute estimated sleepiness is obtained using the average values of the three sections (1), (2), and (3). Ask for a level. Specifically, by using six SPR frequency of _t 0 ~t ₁ of the SPR frequency measurement section (1) calculating an average value den ₁ by the equation (9), equation (10) by the _t 2 ~t ₃ The average value den ₂ is calculated using the six SPR frequencies in the SPR frequency measurement section (2), and the average value is calculated using the six SPR frequencies in the SPR frequency measurement section (3) from t ₄ to t _{5 according} to the equation (11). The value den ₃ is calculated. And then. The absolute estimated drowsiness level DLR (t) is calculated using den ₁ , den _{2, and} den ₃ of the three SPR frequency measurement sections according to Expression (12).

  In addition, as described above, the initial value DL (0) of the drowsiness level is not fixed to 6 (D0), but in the calibration section immediately after the start of the traveling, the SPR frequency is calculated by the equations (9) to (12). The absolute estimated sleepiness level DLR (0) may be calculated from the DLR (0), and the DLR (0) may be used as the initial value DL (0).

Then, a correction value of the sleepiness level DL value is calculated by using the absolute estimated sleepiness level DLR value obtained from the SPR frequency and the sleepiness level DL value obtained from the sleepiness estimation model according to the equation (13).

  FIG. 10 shows an example of the result of the actual vehicle test. In this actual vehicle test, a reference stimulus is given to the driver at regular intervals, and when the drowsiness becomes strong, a wakeful stimulus such as a sound is given to the driver, and a drowsiness estimation model (for example, a third drowsiness estimation model) When the driver is in a stable state, the sleepiness level corrected by the absolute estimated sleepiness level obtained from the SPR frequency is obtained, and the sensory evaluation value of the sleepiness level is also obtained. The sensory evaluation value is obtained by judging from an objective evaluation based on a change in facial expression based on an image of the driver's face. In FIG. 10, the solid line LP indicates the temporal change of the sensory evaluation value, the alternate long and short dash line LE indicates the temporal change of the sleepiness level estimated value by the sleepiness estimation model, and the broken line LR indicates the sleepiness level correction value by the correction process. The time change is shown. As can be seen from this result, the estimated value of the sleepiness level by the sleepiness estimation model deviates from the sensory evaluation value and includes a steady-state error. However, the drowsiness level correction value follows the sensory evaluation value as compared with the drowsiness level correction value, and the steady-state error is reduced (removed).

  Now, a specific configuration of the state estimation system 1 will be described. Here, a case where the third sleepiness estimation model is used will be described. As shown in FIG. 1, the state estimation system 1 includes a vibration generator 2, an electrodermal activity sensor 3, an amplifier 4, a speaker 5, a display 6, and an ECU [Electronic Control Unit] 7. In the present embodiment, the processes in the electrical skin activity sensor 3, the amplifier 4 and the ECU 7 correspond to the state change amount acquisition means described in the claims, and each process in the ECU 7 is a state described in the claims. It corresponds to an estimation unit, a stable state determination unit, a correction unit, and a state change amount density acquisition unit.

  The vibration generator 2 is a device that generates vibration, and gives the driver a fine vibration for a reference stimulus and a vibration for an arousal stimulus. The vibration generator 2 is built in several places on the seat of the driver's seat. The position and the number of the built-in parts are arbitrary. For example, a total of six places are built in each of the left and right sides of the driver's back, waist, and thigh. The vibration generator 2 has variable parameters such as the intensity of vibration to be generated, frequency, and generation time. When the vibration generation device 2 receives the vibration generation signal from the ECU 7, the vibration generation device 2 generates vibration according to the parameter indicated by the vibration generation signal.

  In order to distinguish all vibrations generated by the vibration generating device 2 from vibrations generated in the vehicle, parameters of strength and frequency that are clearly different from those generated in the vehicle are set. In the slight vibration for the reference stimulus, a parameter having a small strength is set so as not to make the driver feel uncomfortable. Since the vibration for arousal stimulation needs to improve the driver's arousal state, a parameter with a large strength is set.

  The electrodermal activity sensor 3 is a sensor that detects EDA (particularly, SPA). The electrodermal activity sensor 3 is attached to each of the left and right sides of the steering wheel with which the driver's palm contacts in order to detect the driver's mental sweating. When the electrodermal activity sensor 3 detects SPA, the detection signal is transmitted to the amplifier 4. The amplifier 4 amplifies the detection signal and transmits the amplified detection signal to the ECU 7. By the way, when a person is awake, a person tends to sweat easily with a stimulus. When a person is awake, a person tends to sweat less easily.

  The speaker 5 and the display 6 are used in common with each system in the vehicle. In the state estimation system 1, the speaker 5 and the display 6 are used to give an arousal stimulus to the driver. When the speaker 5 receives a sound signal from the ECU 7, the speaker 5 outputs a sound according to the sound signal. When receiving the image signal from the ECU 7, the display 6 displays an image according to the image signal.

  The ECU 7 includes a CPU [Central Processing Unit], a ROM [Read Only Memory], a RAM [Random Access Memory], and the like, and comprehensively controls the state estimation system 1. The ECU 7 periodically transmits a vibration generation signal for generating a fine vibration for reference stimulation. Then, the ECU 7 receives a signal obtained by amplifying the detection signal of the electrical skin activity sensor 3 by the amplifier 4 and estimates the driver's sleepiness level by the third sleepiness estimation model based on the signal. Further, the ECU 7 determines whether or not the driver is in a stable state, and corrects the estimated sleepiness level based on the absolute estimated sleepiness level obtained from the SPR frequency in the stable state. Then, in the ECU 7, when the drowsiness level is a level that hinders driving, the awakening stimulus is given to the driver using the speaker 5, the display 6, and the vibration generator 2.

  The ECU 7 transmits a vibration generation signal indicating a fine parameter for reference stimulation to the vibration generator 2 every time a certain period elapses. The fixed period is an interval for acquiring the drowsiness level of the driver and may be an arbitrary time.

  The ECU 7 receives an amplification detection signal from the amplifier 4 every time a fine vibration for reference stimulation is generated. Then, the ECU 7 filters the SPA of the amplification detection signal, removes the SPL component from the SPA, and extracts the SPR component. Then, the ECU 7 calculates and buffers the current SPR value from the time series data of the SPR component according to the equation (1). Further, the ECU 7 counts the number of times that the SPR component processing time T comes out of the range of mean + 3 × σ to mean−3 × σ, and buffers it as the current SPR frequency. Further, the ECU 7 calculates and buffers an average value (mean) and a standard deviation value (σ) of the SPR component in the processing time T for use next time.

  When the SPR value and the SPR frequency this time are obtained, the ECU 7 determines whether the SPR frequency is less than th1, th1 or more and less than th2, or th2 or more. The ECU 7 sets A as the gain K when the SPR frequency is less than th1, sets B as the gain K when it is equal to or greater than th1 and less than th2, and sets C as the gain K when equal to or greater than th2. Set. Further, the ECU 7 calculates and buffers the current AR value using the current SPR value and the previous SPR value according to the equation (2).

  Further, the ECU 7 receives an amplification detection signal from the amplifier 4 when a vibration for arousal stimulation or the like is generated. Then, the ECU 7 extracts the SPR component as in the case of the reference stimulus, obtains and buffers the SPR value and the SPR frequency at the processing time Ts of the SPR component for the arousal stimulus. When the SPR value and the SPR frequency for the arousal stimulus are obtained, the ECU 7 determines whether the SPR frequency is less than th3 or more than th3. Then, the ECU 7 sets α as the gain τ when the SPR frequency is less than th3, and sets β as the gain τ when it is equal to or greater than th3. Further, the ECU 7 calculates and buffers the AR value for the wakefulness stimulus using the SPR value for the wakefulness stimulus and the SPR value for the previous reference stimulus by Expression (2).

  Then, the ECU 7 calculates the current sleepiness level by using the current AR value, the gain K, the AR value for the wakefulness stimulus, the gain τ, and the previous sleepiness level by the third sleepiness estimation model of Expression (5). Buffer. Note that if the arousal stimulus is not applied between the previous and previous reference stimuli, the AR value for the arousal stimulus is set to 0, and the current sleepiness level is calculated.

  In addition, when the current AR value is obtained, the ECU 7 calculates the current AR ′ value using the current AR value and buffering it according to the equation (6). Then, the ECU 7 reads the past five AR 'values, and calculates the current ARR value by using the six AR' values including the current AR 'value according to the equation (7). Further, the ECU 7 determines whether or not 0.8 <current ARR value <1.2 is satisfied. When the current ARR value is 0.8 or less or 1.2 or more, the ECU 7 determines the sleepiness level estimated by the third sleepiness estimation model as the current sleepiness level.

When 0.8 <current ARR value <1.2 (when the driver is in a stable state), the ECU 7 sets a calibration section retroactively from the present time, and sets all the SPR frequencies included in the calibration section. read out. Then, the ECU 7 calculates the average value den ₁ using the six SPR frequencies in the SPR frequency measurement section (1) by the equation (9), and the six SPR frequencies in the SPR frequency measurement section (2) by the equation (10). Is used to calculate the average value den ₂ , and the average value den ₃ is calculated using the six SPR frequencies in the SPR frequency measurement section (3) using equation (11). Further, the ECU 7 calculates the current absolute estimated sleepiness level using the _three average values den ₁ , den ₂ , and den _{3 according} to the equation (12).

  Then, the ECU 7 determines whether or not there is a difference between the sleepiness level estimated by the third sleepiness estimation model and the absolute estimated sleepiness level calculated from the SPR frequency. If there is a difference, the ECU 7 uses the absolute sleepiness level calculated from the sleepiness level estimated by the third sleepiness estimation model and the SPR frequency in the calibration section according to the equation (13), and this sleepiness level (correction value). Is calculated. When there is no difference, the ECU 7 sets the sleepiness level estimated by the third sleepiness estimation model as the current sleepiness level.

  Every time the sleepiness level this time is obtained, the ECU 7 determines whether or not the sleepiness level is equal to or less than the arousal stimulus imparting level. The arousal stimulus imparting level is a threshold value for determining whether or not the level of the arousal state is low enough to cause the driver to interfere with driving even a little (a level where sleepiness is strong). For example, 4 (D2 ) Value is set. In the ECU 7, when it is determined that the sleepiness level this time is equal to or less than the arousal stimulus imparting level, a voice message and an image for alerting the driver are generated according to the sleepiness level, and a voice signal including the voice data is generated by the speaker. 5 and an image signal composed of the image data is transmitted to the display 6. Further, the ECU 7 sets a vibration generation signal of a parameter for generating a relatively strong vibration for improving the arousal state for the driver according to the sleepiness level, and transmits the vibration generation signal to the vibration generator 2. At this time, as the drowsiness level decreases from 4 (increases drowsiness), image data, sound data, and vibration parameters are generated so as to give an image, sound, and vibration in a stepwise manner. At this time, all of the image, sound, and vibration may be given, or one or two of them may be given according to the sleepiness level.

  The operation of the state estimation system 1 will be described with reference to FIG. In particular, the correction process in the ECU 7 will be described with reference to the flowchart of FIG. FIG. 11 is a flowchart showing the flow of correction processing in the ECU of FIG.

  When the vehicle starts running, the state estimation system 1 performs range adjustment between the drowsiness level and the AR value by the above-described method in the calibration section after the stable running. After the range adjustment, the state estimation system 1 moves to estimation of the drowsiness level. In this calibration section, the sleepiness level may be absolute estimated from the SPR frequency, and the absolute estimated sleepiness level may be set as an initial value.

  The electrodermal activity sensor 3 detects sweating from the palm of the driver and transmits the detection signal to the amplifier 4. The amplifier 4 amplifies the detection signal from the electrical skin activity sensor 3 and transmits the amplified detection signal to the ECU 7.

  In the ECU 7, a vibration generation signal for generating a fine vibration for reference stimulation is set and transmitted to the vibration generator 2 at regular intervals. When this vibration generation signal is received, the vibration generator 2 generates a fine vibration for reference stimulation.

  Each time the reference stimulus is applied, the ECU 7 analyzes the SPA at the processing time T immediately after the reference stimulus is applied based on the amplification detection signal from the amplifier 4, and the SPR value (SPR (SPR ( t)), and the SPR frequency (den (t)) is obtained based on the average value and standard deviation value of the previous SPR component. Then, the ECU 7 calculates the current AR value (AR (t)) from the SPR value (SPR (t)) for the current reference stimulus and the SPR value (SPR (t−1)) for the previous reference stimulus by the equation (2). ) Is calculated. Further, the ECU 7 sets the gain K based on the SPR frequency (den (t)) for the current reference stimulus. At this time, the ECU 7 obtains the average value and standard deviation value of the SPR component for the next time.

In particular, when the wakefulness stimulus is applied, the ECU 7 analyzes the SPA at the processing time Ts immediately after the wakefulness stimulus is applied based on the amplification detection signal from the amplifier 4, and the SPR value (SPR (tPR (t _s )), and the SPR frequency (den (t _s )) is obtained based on the average value and standard deviation value of the previous SPR component. Then, the ECU 7, the equation (2), SPR values for awakening stimuli _{(SPR (t} s)) and AR value for the waking stimulus from SPR values for the previous reference stimuli (SPR (t-1)) (AR (t s )) Is calculated. In addition, the ECU 7 sets the gain τ based on the SPR frequency (den (t _s )) for the arousal stimulus.

Each time the AR value for the current reference stimulus is obtained, the ECU 7 calculates the AR value (AR (t)) for the current reference stimulus and the AR value (AR (t _s )) (0 when no arousal stimulus is applied), gain K, gain τ, and previous sleepiness level (DL (t−1)) are used to calculate the current sleepiness level (DL (t)) To do. Here, the current sleepiness level in which the degree of addition of the AR value is adjusted according to the SPR frequency is obtained. In particular, when the arousal stimulus is applied immediately before, the effect of the awakening stimulus is also added to the current sleepiness level. Reflected.

  Further, every time the AR value for the current reference stimulus is obtained, the ECU 7 obtains the current AR ′ value (AR ′ (t)) from the AR value (AR (t)) according to the equation (6) (S1). Further, the ECU 7 calculates the current ARR value (ARR (t)) from the six AR ′ values (AR ′ (t−5),..., AR ′ (t)) including the current time using the equation (7). Is obtained (S1). Then, the ECU 7 determines whether or not the current ARR value (ARR (t)) is larger than 0.8 and smaller than 1.2 (S2).

  When it is determined in S2 that the current ARR value (ARR (t)) is 0.8 or less or 1.2 or more, it is determined that the driver is not in a stable state, and the ECU 7 obtains the current drowsiness estimation model. The sleepiness level (DL (t)) is directly used as the current sleepiness level (DL (t)).

When it is determined in S2 that the current ARR value (ARR (t)) is larger than 0.8 and smaller than 1.2, the driver determines that the state is stable, and the ECU 7 performs calibration retroactively from the present time. Set the interval. Then, in the ECU 7, from the SPR frequency (den (t)) in each of the SPR frequency measurement sections (1), (2), and (3) according to the expressions (9), (10), and (11). Average values den ₁ , den ₂ , den ₃ of SPR frequencies in each section are calculated (S 3). Further, the ECU 7 calculates an absolute estimated sleepiness level (DLR (t)) from the average values den ₁ , den ₂ , den ₃ of the SPR frequency according to the equation (12) (S3).

  The ECU 7 determines whether the absolute value of the difference between the absolute estimated sleepiness level (DLR (t)) obtained from the SPR frequency and the sleepiness level (DL (t)) obtained from the third sleepiness estimation model is greater than zero. (S4). When the absolute value of the difference is determined to be 0 in S4, it is determined that the steadyness error is not included in the sleepiness level (DL (t)) obtained by the third sleepiness estimation model, and the ECU 7 estimates the third sleepiness. The current sleepiness level (DL (t)) obtained from the model is directly used as the current sleepiness level (DL (t)). On the other hand, if it is determined in S4 that the absolute value of the difference is greater than 0, it is determined that the drowsiness level (DL (t)) obtained by the third drowsiness estimation model includes a steady state error. From the absolute estimated sleepiness level (DLR (t)) obtained from the SPR frequency and the sleepiness level (DL (t)) obtained from the third sleepiness estimation model, the sleepiness level (correction value) (DL ( t)) is calculated (S5).

  Each time the current sleepiness level is obtained, the ECU 7 determines whether or not the current sleepiness level (DL (t)) is equal to or lower than the arousal stimulus imparting level. And when it is determined that the sleepiness level (DL (t)) of this time is equal to or less than the arousal stimulus imparting level (when the arousal state is low enough to interfere with driving even a little), in order to impart the arousal stimulus to the driver, The ECU 7 generates a voice message and an image for alerting the driver according to the drowsiness level (DL (t)), transmits a voice signal composed of the voice data to the speaker 5 and uses the image data. An image signal is transmitted to the display 6, and a vibration generation signal for improving the arousal state is set for the driver according to the drowsiness level (DL (t)) and transmitted to the vibration generator 2. The ECU 7 repeats the above processing at regular intervals.

  When the audio signal is received, the speaker 5 outputs a warning message according to the audio signal. When the image signal is received, the display 6 displays a warning image according to the image signal. Further, when the vibration generation signal is received, the vibration generator 2 generates a strong vibration. These arousal stimuli increase the driver's arousal state and increase their attention to driving.

  According to the state estimation system 1, when the driver is in a stable state, the sleepiness level estimated by the sleepiness estimation model is corrected using the sleepiness level that is absolute estimated from the SPR frequency, so that the relative change in the SPR value is detected. The steady state error included in the estimated sleepiness level can be reduced (removed), and the driver's sleepiness level can be estimated with high accuracy. At this time, the state estimation system 1 can estimate the absolute value of the sleepiness level with high accuracy by using the SPR frequency when the driver is in a stable state.

  Further, in the state estimation system 1, since the calculation is performed using a plurality of overlapping data when calculating the absolute estimated sleepiness level and the ARR value, the calculated value is robust against noise. Moreover, in the state estimation system 1, since the stable state of the driver is determined based on the change in the AR value, the stable state of the driver can be determined with high accuracy.

  As mentioned above, although embodiment which concerns on this invention was described, this invention is implemented in various forms, without being limited to the said embodiment.

  For example, in this embodiment, the present invention is applied to a device that is mounted on a vehicle and estimates a drowsiness level (wake level) as a state of a vehicle driver. However, a driver of another vehicle, a monitor of various plants, an employee at night It may be used to estimate the state of various people such as a person, and other states such as psychological state (impression, irritability, boredom), fatigue state, concentration, attention, stress, etc. You may apply to the apparatus to estimate.

  In this embodiment, vibration generated from the seat is used as a reference stimulus to be given to a person. However, various reference stimuli can be used, for example, by generating other physical stimuli such as sound and light. Alternatively, vibration, light, sound, etc. that are constantly emitted from the device may be used. Moreover, the physical stimulus converted from the stimulus received from an external environment may be sufficient, for example, the sound heard by a driver | operator may be detected and the driver | operator may be given the mechanical vibration converted from the detected sound. The stimulus received from the external environment is, for example, road noise from a tire. Further, in the present embodiment, the reference stimulus is given at a constant cycle. However, the reference stimulus may be given at various timings. For example, the reference stimulus may be given at a random cycle, or the state of the driver is estimated. A reference stimulus may be applied when needed (eg, before each event is expected to occur). This event is an environment in which the driver is placed during driving, for example, traffic congestion that induces the driver's sleepiness, traveling on a highway.

  In addition, various stimulus factors can be considered as the stimulus factors of the reference stimulus. For example, there are visual, tactile, auditory, warm and cold, olfactory psychology, visceral (abdominal pain, etc.) stimuli as environmental factors, visual, auditory, psychological stimuli as traffic factors, and visual, auditory, psychological stimuli as passenger factors . Human senses include superficial sensations (such as tactile sensation, pain sensation, and temperature sensation), depth sensations (such as pressure sensation, positional sensation, and vibration sensation), and cortical sensations (such as two-point discrimination and three-dimensional discrimination). The somatosensory stimulation given to the driver (subject) as mentioned above can be actively used as a reference stimulus. However, in order to improve the estimation accuracy, the following conditions are preferable. Conditions include periodic stimuli (fixed intervals), unified stimulus intensity and type, and dynamics rather than static stimuli. In consideration of such conditions, any of the types of stimuli listed in the factors is possible, but a type of stimulus that can be recognized as a different type from the stimulus that is normally received is necessary. Furthermore, since the receptive sensory organs differ depending on the type of stimulus, and the sensitivity of each receptive sensory device is different, it is necessary to understand what kind of stimulus it is. For example, assuming wind as a reference stimulus that gives vision, specifically, air is blown from the air conditioner duct toward the eyes, and the number of blinks, the time to close the eyes, the reaction time, etc. are measured, and the state is determined from the measured values. A method for judging the above can be considered. The same effect can be obtained by applying the AR method based on this.

  In the present embodiment, dermatoelectric activity (EDA), particularly skin potential response (SPR), which is one of physiological indices as the state change amount, is utilized, but other various physiological indices can be utilized, for example, Physiological index (eg, blink) using electrooculogram (EOG [Electrooculogram]), physiological index (α wave, β wave, etc.) based on electroencephalogram (EEG [Electroencephalogram]), physiological index (heart rate) based on electrocardiogram (ECG [Electrocardiogram]) ), Physiological indices such as myoelectricity (EMG [Electromyogram]) may be used, and among skin electrical activities, skin resistance level (SRL), skin resistance response (SRR), skin potential level (SPL), etc. Other physiological indices may be used. Furthermore, the state may be estimated by combining a plurality of physiological indices instead of estimating the state by using only one physiological index.

  In the present embodiment, the state of a person is estimated using a physiological index as a state change amount. However, the state can be estimated even with a state change amount other than the physiological index. (Handle operation, brake operation, accelerator operation, etc.) and vehicle status (yaw rate, vehicle speed, etc.).

  Further, in this embodiment, when the drowsiness level becomes a level that hinders driving, it is configured to be alerted by image display, audio output, vibration, etc., but other means such as cold air, alarm buzzer, smell, etc. It may be configured to alert the driver, or when the state becomes a level that hinders driving, the control timing of the driving support system (for example, pre-crash safety system, adaptive cruise control system, lane keeping system) The vehicle may be controlled to improve safety by changing the control threshold. Moreover, it is good also as a structure which detects a person's state and outputs the detected person's state.

  In the present embodiment, the first to third sleepiness estimation models for estimating the sleepiness level (human state) have been shown, but various other methods can be applied as methods for estimating the human state. It is. A method of estimating the state of a person without applying a reference stimulus may be used.

  In the present embodiment, the sleepiness level (subject's state) is corrected when the driver is in a stable state, but the subject state change amount such as the SPR value is corrected when the subject is in a stable state. It is good.

  In the present embodiment, the average value of the sleepiness level obtained from the sleepiness estimation model and the absolute estimated sleepiness level obtained from the SPR frequency when the driver is in a stable state is used as the sleepiness level correction value. The absolute estimated sleepiness level obtained from the frequency may be used as the sleepiness level correction value as it is.

  In the present embodiment, the absolute estimated sleepiness level is obtained from the SPR frequency and the sleepiness level is corrected using the absolute estimated sleepiness level. However, the sleepiness level may be corrected by other methods.

  In the present embodiment, the AR ′ value is obtained from the AR value, the ARR value is obtained from a plurality of AR ′ values, and the driver's stable state is determined based on the ARR value. A stable state may be determined.

  In the present embodiment, the calibration interval is set to obtain the absolute estimated sleepiness level from the SPR frequency, and the absolute estimated sleepiness level is calculated from a plurality of SPR frequencies included in the calibration interval. If is sufficiently low, the absolute estimated sleepiness level may be calculated from only the current SPR frequency.

  Further, in the present embodiment, the primary expression common to all persons for calculating the absolute estimated sleepiness level is shown, but a primary expression learned for each individual may be set.

  In the present embodiment, the absolute estimated sleepiness level is calculated by a linear expression using the SPR frequency as a variable, but the absolute estimated sleepiness level may be obtained by other methods. For example, a lookup table showing the relationship between the sleepiness level and the SPR frequency as shown in FIG. 12 is prepared, and the sleepiness level corresponding to the SPR frequency is extracted with reference to this lookup table (each threshold value). May be. The lookup table may be common to all people or may be individual.

It is a block diagram of the state estimation system which concerns on this Embodiment. It is an example of the time change of the estimation start timing which concerns on this Embodiment, a reference stimulus, an arousal stimulus, an SPR value, and an AR value. It is a figure which shows the correspondence of the drowsiness level (wakefulness level) and AR value which concern on this Embodiment. It is an example of the time change of the SPR value with respect to the reference stimulus which concerns on this Embodiment. It is an example of the time change of SPR frequency which concerns on this Embodiment. It is an example of the time change of the sleepiness level estimated by the sleepiness estimation model when it became a stable state, the sleepiness level by subjective evaluation, and AR value. It is an example of the time change of AR value, AR 'value, and ARR value which concern on this Embodiment. It is an example of the correspondence between SPR frequency and drowsiness level in a stable state, (a) is subject A, and (b) is subject B. It is an example of the calibration area for performing the absolute estimation of the drowsiness level by the SPR frequency which concerns on this Embodiment. It is an example of the time change of the sleepiness level estimated value by the sleepiness estimation model in a real vehicle test, the sleepiness level correction value at the time of performing a correction process, and a sleepiness level sensory evaluation value. It is a flowchart which shows the flow of the correction process in ECU of FIG. It is an example of the look-up table for performing the absolute estimation of the sleepiness level by SPR frequency.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... State estimation system, 2 ... Vibration generator, 3 ... Electrical skin activity sensor, 4 ... Amplifier, 5 ... Speaker, 6 ... Display, 7 ... ECU

Claims (2)

A human state estimation device for estimating the state of a subject,
State change acquisition means for acquiring the state change of the subject;
State estimation means for estimating the state of the subject based on the state change amount acquired by the state change amount acquisition means;
Stable state determination means for determining whether or not the state of the subject is a stable state;
A human state estimation device comprising: a correction unit that corrects the state change amount of the subject or the state of the subject when the stable state determination unit determines that the state is stable. A state change amount density acquisition means for acquiring a state change amount density from the state change amount of the subject acquired by the state change amount acquisition means;
When the correction means determines that the stable state is determined by the stable state determination means, the correction means estimates the state of the subject based on the state change density of the subject acquired by the state change density acquisition means, and determines the state change amount density. The human state estimation device according to claim 1, wherein correction is performed using the state of the subject estimated based on the state.

Images