JP3509839B2 - Arousal level estimation device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is suitable for activating driver's attention by focusing on the blink of a driver and easily detecting the driver's arousal level and, by extension, reducing the awakening level and certainly issuing a warning. Awakening degree estimation device.

[0002]

[Related Background Art] Recently, various systems have been developed for estimating driver's arousal level based on various information, and issuing a warning when a decrease in the arousal level is detected to call attention to driving. Has been done. One of the methods to estimate this type of arousal
For example, a driver's blink is used as an index for evaluation. For example, Japanese Patent Laid-Open No. 61-175129 discloses a method of counting the number of blinks per unit time to determine a decrease in arousal level. There is. However, when the number of blinks per unit time is used as an index for arousal level evaluation, there is a problem in that an error due to individual differences in blinking is likely to occur and the estimation accuracy cannot be improved.

Therefore, the present applicant previously filed Japanese Patent Application No. 8-913.
No. 24, and a paper [52.
A method of estimating a decrease in arousal level by focusing on the blink time of the driver was proposed as a method of evaluating arousal level during car driving (9632415)]. The awakening level estimation method focusing on this blinking time is based on the frequency distribution of the blinking time, and is a threshold value (blinking time) for determining a long blink from the standard blinking distribution time width and the central time of the distribution width. ) Is set, the ratio between the total number of blinks within a predetermined period and the number of occurrences of long blinks exceeding the above threshold value is obtained, and this ratio is evaluated to determine a decrease in arousal level.

According to such a method, there is an advantage that the individual difference such as blinking time and blinking frequency can be absorbed and the arousal level can be evaluated accurately.

[0005]

By the way, as an index for estimating the arousal level, not only the above-mentioned blink, but also the change of brain waves can be used. However, in consideration of estimating the arousal level of a driver mounted on a car, the brain wave measurement that requires sticking electrodes to the driver's head is not practical. Therefore, the degree of drowsiness (awakening degree)
Information about the behavior of the driver such as the frequency of forward or backward tilt of the driver's head, the frequency of movement of the line of sight, and the frequency of various driving operations such as steering wheel operation and turn signal operation (behavior) It is considered desirable to detect the characteristics) without contacting the driver.

However, by using the above-mentioned information about the behavior of the driver other than the blinking time or by using the information about the behavior of the driver together with the information about the blinking, how to make the driver's arousal degree easier Moreover, some problems remain in terms of estimating with high accuracy. In particular, there remains a problem in that the information about the behavior of the driver (behavioral characteristics) is used to estimate the awakening level without being affected by individual differences.

The present invention has been made in consideration of such circumstances, and its purpose is to wake up the driver by using not only the information on the driver's blink but also other behavioral characteristics of the driver. An object of the present invention is to provide an awakening degree estimation device that can estimate the degree easily, efficiently, and highly accurately.

[0008]

In order to achieve the above-mentioned object, a wakefulness estimating apparatus according to the present invention detects a blink of the driver from a face image of the driver captured by a camera, and detects the blink of the driver. Along with detecting a long blink of a predetermined time or more based on the blink time, with a long blink occurrence ratio calculation means for obtaining the long blink occurrence ratio from the total number of blinks and the number of long blinks within a predetermined evaluation period A behavior detection means for detecting a behavior indicating the behavioral characteristics of the driver, wherein the awakening degree determination means has a relationship between the occurrence ratio of the long blink and the behavioral characteristic information of the driver.
It is characterized in that so as to evaluate the alertness of the driver according to the prediction equation obtained by regression analysis using the sleepiness prediction model showing the.

Particularly, in detecting the long blink,
For example, a blink reference time [To] unique to the driver is obtained from the blink time of the driver during awakening, and the blink is longer than the time [Ts] obtained by increasing the reference time by a predetermined ratio [r%]. It is detected as a blink, and the occurrence ratio [Lrate] of a long blink with respect to the total number of blinks within a predetermined time is obtained. On the other hand, as the behavior information indicating the behavior characteristic of the driver, for example, the standard deviation [EMH-SD] of the horizontal movement of the eyeball indicating the movement of the driver's line of sight is obtained.

In the awakening degree determining means, for example, the awakening degree and the occurrence rate of long blink [Lrate] and the eyeball value obtained by regression analysis using a drowsiness prediction model .
Prediction formula showing the relationship with the standard deviation [EMH-SD] of horizontal movement Y = A + B [Lrate] + C [EMH-SD] (A, B and C are prediction coefficients) ], And the sleepiness prediction value is discriminated by a predetermined threshold value to evaluate the awakening degree.

That is, the arousal level is estimated according to the prediction formula shown in the multiple regression model (sleepiness prediction model) obtained by integrating the information about the blink and the information indicating other behavioral characteristics such as the standard deviation of the eye movement. By doing so, it is possible to detect the driver's arousal level, and thus the decrease in the arousal level, in a simple and highly accurate manner.

[0012]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the arousal level estimation device according to the present invention will be described below with reference to the drawings. Figure 1
2 conceptually shows the configuration of the embodiment apparatus mounted on the vehicle 1. In the figure, reference numeral 2 denotes a TV camera for imaging the face of the driver D, especially the eye region. Further, in the figure, 3 is a display (multi-information display device) for displaying various information as an image and presenting it to the driver D, and 4 is a speaker for outputting a voice message, an alarm sound or the like. These TV camera 2, display 3, and speaker 4 are incorporated in, for example, an instrument panel in front of the driver's seat.

The awakening degree estimation apparatus according to this embodiment is
The driver's blink is detected from the driver's face image captured by the V-camera 2 to estimate the driver's arousal level, a message is displayed via the display 3 when the awakening level is lowered, and the speaker 4 is also used. It plays a role to generate a warning from the driver and call attention to driving. This device is realized by, for example, an electronic control unit (ECU) mainly including a microprocessor, and is schematically configured as shown in FIG.

That is, this arousal level estimation device is shown in FIG.
As shown in the functional block configuration of FIG. 1, the image processing unit 1 displays a driver's face image captured and input by the TV camera 2.
The recognition processing is performed at 0, and the partial image of the eye region is extracted at a predetermined cycle, for example. The blink detection unit 11 detects the blink by detecting the opening and closing of the eyelid from the change of the image with time. The blink time calculation unit 12 uses the blink detection unit 11 described above.
Every time a blink is detected, the blinking time (blinking time) [t] indicated by the eye closing time from the start to the end of the eye closing is measured. Such a blink time detection process is repeatedly executed over a predetermined period under the control of a timer, for example.

The blinking reference time calculating unit 14 is strongly conscious of "starting (starting) driving," for example, at the beginning of driving, and it is considered that the driver D is sufficiently awake. Whenever possible, based on the blink time [t] obtained by the blink time calculation unit 12 over a predetermined period, the blink reference time [To] unique to the driver D at the time of awakening is obtained. There is. The reference time [To] is calculated as the average value of the blink time [t] in the predetermined period, for example, 5 minutes at the start of operation.

The long-blink time setting unit 15 sets a blink evaluation time [Ts] for detecting a long blink associated with a decrease in arousal level from among the blinks of the driver D. Reference time [T obtained by time calculation unit 14
The evaluation time [Ts] is defined as Ts = To + To · r / 100 by increasing [o] by a predetermined ratio [r (%)]. The above-mentioned ratios are given as 5%, 10%, 15%, 20%, for example, and are actually set to 10%, for example, based on the simulation result described later. That is, in the long-blinking time setting unit 15, when the arousal level decreases,
In general, in consideration of the fact that a long blink is likely to occur, the blink evaluation time [Ts] for detecting a long blink in order to use this long blink as a determination index for the decrease in arousal level.
Is set based on the blink reference time [To] peculiar to the driver D.

Although the blink evaluation time [Ts] is set on the basis of the blink reference time [To] here, as the applicant of the present invention previously proposed in Japanese Patent Application No. 8-91324,
Based on the frequency distribution of the blink time [t], the standard blink distribution time width and the central time of the distribution width are obtained, and the blink evaluation time is calculated according to the frequency distribution information (time width and central time) of these blinks. Of course, it is possible to set [Ts].

The long blink detection unit 16 detects the blink time [t] of the driver D sequentially detected by the blink time calculation unit 12 according to the blink evaluation time [Ts] set as described above. A blink that exceeds the evaluation time [Ts] is detected as a long blink. The long blink occurrence ratio calculation unit 17 determines, for example, the total number of blinks [Ntotal] evaluated by the long blink detection unit 16 and the long blink detection unit 1.
The number of long blinks required as a detection result by 6 [Nlo
ng] and are counted respectively. The occurrence ratio calculation unit 17 calculates the occurrence ratio [Lra of long blinks from the total number [Ntotal] of the blinks in the preset predetermined evaluation period and the number of long blinks [Nlong] in the blinks.
te (%)] is obtained as Lrate = 100.Nlong / Ntotal.

In the long blink occurrence ratio calculation unit 17, the number of long blinks [Nlong] and the standard number of other blinks [Nnormal] are respectively counted, and Lrate = 100.Nlong / (Nlong) Needless to say, the occurrence rate [Lrate] of a long blink may be obtained as + Nnormal).

On the other hand, the eye movement detecting section 21 detects the horizontal movement of the eyeball from the partial image showing the eye area of the driver D obtained by the image processing section 10. The movement of the eyeball in the horizontal direction is based on behavioral characteristics (behavior) of the driver D, such as confirmation of information on an adjacent lane from the attention state of the traveling lane or confirmation of information via a rearview mirror or a side mirror (gaze at the mirror). It is shown. Then, the standard deviation calculation unit 22 obtains the standard deviation [EMH-SD] of the movement from the information on the horizontal movement of the eyeball (the position of the pupil in the eye). The standard deviation of the horizontal movement of the eyeball is calculated, for example, as the average value of the standard deviation per unit time obtained in a step of 60 seconds for 5 minutes.

Therefore, the awakening degree predicting / calculating section 23 determines the awakening degree and the occurrence rate of the long blink [Lrate], which is obtained by multiple regression analysis using a drowsiness prediction model as described later.
(%)] And the standard deviation of eye movement [EMH-SD], which is a prediction formula Y = A + B.Lrate + C.EMH-SD (A,
B and C are for calculating the sleepiness prediction value [Y] based on the prediction coefficient. The above-mentioned prediction coefficients A, B, and C define, for example, the level of the arousal level in five stages as follows, and the rate of increase in setting the above-mentioned blink evaluation time [Ts] based on various simulation results [ r%] is set to 10% as described later, for example, A = 2.812, B = 0.032, C = −0.695. That is, the prediction formula is Y = 2.812 + 0.032 · Lrate−0.695 ·
Given as EMH-SD, the occurrence rate [Lrate (%)] of long blink during the blink of the driver D is calculated as described above, and the standard deviation [EMH-SD] of the eye movement at that time is calculated to obtain the above. The sleepiness prediction value [Y] is obtained by calculating the prediction formula.

By the way, a prediction expression y = a + b.Lrate (a, b is shown as follows) showing the relationship between the arousal level obtained by multiple regression analysis using a sleepiness prediction model and the occurrence rate [Lrate (%)] of the above-mentioned long blink. When the drowsiness prediction value [y] is calculated based on (prediction coefficient), the prediction coefficients a and b are given as a 1 = 1.238, b = 0.046, for example, under the same evaluation condition. That is, the prediction formula in this case is given as y = 1.238 + 0.046.Lrate.

The above-mentioned five levels of arousal level are, for example, level 1 ... totally sleepless (the line of sight moves quickly and frequently. The blink is stable and the movement is active.) Level 2 ... somewhat sleepy (line of sight Moves slowly. Lips open. Level 3 ... seems sleepy (blinks slowly and often. Mouth moves.) Level 4 seems quite sleepy (has conscious blinks, slow blinks and gaze movements) ..) Level 5 ... Set as very sleepy (closes eyelids. Head tilts back and forth). Therefore, when the drowsiness prediction values [Y] and [y] exceed [3], the awakening degree is low and it is estimated (determined) that the subject is sleepy as shown below.

Thus, the drowsiness prediction value [Y] obtained by the awakening degree prediction calculation unit 23 on the basis of the occurrence ratio [Lrate] of the long blink and the standard deviation [EMH-SD] related to the horizontal movement of the eyeball is It is provided to the awakening degree lowering determination unit 24 and used for the determination of the lowering of the awakening degree. When the decrease in arousal level is detected by the awakening level decrease determining unit 24, a warning is issued by using the display 3 and the speaker 4 described above, and the driver D is alerted to the driving attention. Will be seen.

The wakefulness lowering determination process shown in the above-mentioned functional block is actually executed under the microprocessor according to the control routine shown in FIG. That is, at the beginning of driving, for example, the driver D
Of the blinking time [t] of the eye (steps S1, S2, S
3) Then, the average of the blink times is calculated as the blink reference time [To] peculiar to the driver D (step S4).
Thereafter, one unit of measurement target period is set to 5 minutes, and the subsequent blink time [t] is measured (steps S5, S6,
S7).

Thereafter, the reference time [To] is set to 1 in accordance with the blink time [t] detected for 5 minutes each as described above.
The occurrence rate [Lrate (= Long10)] of the long blink detected under the blink evaluation time [Ts] set to be 0% longer is calculated (step S8). At the same time, the horizontal movement of the eyeball indicating the movement of the line of sight is detected, and the standard deviation [EMH-SD] of the horizontal movement is obtained based on the movement information (step S
9).

Then, the occurrence ratio of the long blink [Lrate (= Lo
ng10)] and the standard deviation [EMH-SD] of the eye movement are calculated respectively, the sleepiness prediction value [Y] is calculated according to the above-described prediction formula (step S10), and is calculated by this calculation, for example. The sleepiness prediction value (awakening degree) is displayed on the display 3 (step S11). This drowsiness prediction value (awakening degree) is displayed on the display, for example, by displaying the awakening degree in a bar graph or changing the display color of the information according to the levels set in five stages as described above.

Then, the sleepiness prediction value [Y] obtained as described above is evaluated (step S12). If the level (prediction value) exceeds [3], the driver D is awakened. A warning is issued to prompt the driver's attention (step S13). When the level of the arousal level is equal to or lower than [3], the process from step S5 described above is repeatedly executed, and the awakening level estimation process based on the blink information in the next 5 minutes is executed again.

The occurrence ratio of the long blink [Lra
te (= Long10)] and the standard deviation of horizontal movement of the eyeball [EMH
Calculation of the sleepiness prediction value [Y] based on -SD] and its evaluation will now be described in a little more detail. As the evaluation index of the arousal level, not only the above-mentioned blink, but also an electroencephalogram, an electrocardiogram,
It is conceivable to use changes over time in physiological indexes such as breathing, and changes in performance indexes such as steering operation characteristics indicated by the steering wheel angle. FIG. 4 shows the results of a simulation experiment showing the relationship between each of these indexes and the drowsiness of the driver D objectively evaluated by a third party at that time.

In FIG. 4, Cz (α / β) and Pz (α / β)
Is the ratio of the spectral power values of the α and β waves of the electroencephalogram obtained at the driver's D parietal Cz and Pz regions. Blink-No is the number of blinks obtained from the eye movement of the driver D, for example, 5 seconds, and Blink-Dur
Is the average blink time, HR is the heart rate of the driver D per minute, and Resp is the respiratory rate. Furthermore, Speed is the running speed of the vehicle, Steer is the average steering wheel angle, and Steer-SD is the deviation of the steering wheel angle. Sleepiness is a drowsiness of the driver D evaluated by a third party, and MWS is a subjective evaluation of drowsiness by the driver D himself.

As shown in the result of the simulation experiment illustrated in FIG. 4, the sleepiness of the driver D (Sleepine
ss) increases with the running time. Moreover, the drowsiness of the driver D depends on the number of blinks (Blink-No) and blinking time (Blin-No).
k-Dur) has a strong relationship with. That is, there is a tendency that drowsiness increases due to the monotonousness and familiarity of driving operations with the passage of time, and further fatigue, and as the drowsiness increases, the number of blinks decreases and the blinking time also increases. Furthermore, with the passage of driving time (increased drowsiness), Cz (α / β) and Pz (α / β) of the above-mentioned electroencephalogram also tend to increase, and HR on the contrary.
And Resp show a decreasing trend.

On the other hand, a plurality of drivers D (subject: Subj.1,
~ 12) in the driving simulation, driver D
Drowsiness expression, eg driver D evaluated by a third party
Drowsiness of the driver D, and the behavioral features (behavior) of the driver D that occur when the drowsiness is high are examined from the subjective evaluation MWS of the drowsiness of the driver D. Behavioral characteristics that result in significant changes. In other words, when drowsiness becomes stronger, the driver D commonly
Not only a remarkable change occurs in the blink, but the movement itself of the driver D tends to be slow. It was also found that the frequency of driving operations such as speed adjustment, mirror confirmation, blinker operation, etc. decreased with increasing drowsiness.

In particular, in the above-mentioned simulation experiment results, focusing on the behavioral features such as confirmation of the mirror and blinker operation at the time of changing lanes, as shown in FIG. It was found that there is a general tendency that the frequency of movement of the line of sight shown decreases and that the frequency of driving operations such as winker operation decreases.

Therefore, in the present invention, for the purpose of evaluating the arousal level (drowsiness fluctuation) in a non-contact manner with the driver D, the blinking of the driver and the horizontal movement of the eyeball indicating the line of sight movement are focused and predicted. We advanced the study to improve accuracy and reduce individual differences. Specifically, first, we focused on the long blink that increases with the decrease of the arousal level, and investigated the relationship between the occurrence rate of the long blink and the arousal level in a predetermined period. In particular, as the pre-processing, the blink evaluation time [Ts] for determining a long blink is set as the reference time [To], which is an average blink time specific to the driver D at the time of awakening, and this reference time [To]
The increase rate of blinking time against 5%, 10%, 15%, 2
Each blink evaluation time [Ts] when set to 0%
Under the following conditions, the occurrence ratios [Lrate] of long blinks of a plurality of drivers D in a predetermined time (for example, 5 minutes) are set to [Long5], [Lo
ng10], [Long15], and [Long20], respectively, and the relationship with the arousal level was investigated.

That is, the occurrence ratio of the long blink [Long5],
For [Long10], [Long15], and [Long20], the sample center time is, for example, a delay time given as a cycle of 60 seconds as one unit (Lag).
The relationship analysis with drowsiness (wakefulness) was performed while shifting the value by 1 unit, and the relationship coefficient R under the condition that the relationship coefficient was the largest was evaluated. As a result of this evaluation, the relation coefficient R in the occurrence ratio [Long20] becomes smaller than that in the above occurrence ratios [Long5], [Long10], [Long15], and the evaluation time [Ts] of long blinks becomes 20% or more. I found that it is better not to set it for a long time. In the analysis of variance for the Lag value, the occurrence rate of long blinks [Long5] as an index,
It was confirmed that there was no significant difference between [Long10] and [Long15]. However, each of these indicators is physiologically different data.

This evaluation is performed by multiple regression analysis using the sleepiness prediction model described above, and a prediction expression y = a + b.Lrate showing the relationship between the arousal level and the occurrence rate [Lrate (%)] of the long blinks. The sleepiness prediction value [y] is calculated based on AIC = 2n logeσ + 2p (where n is the number of data and p is the number of regression coefficients) According to the reference AIC, the prediction model in which the information amount reference AIC is the smallest is obtained.

In the analysis of variance, each index (probability of occurrence of long blinks) was calculated as an average for 300 seconds, and drowsiness expression was calculated as an average for 60 seconds. Therefore, this analysis of variance shows that the time characteristic of sleepiness prediction is, for example, 120 seconds to 180 seconds in the average of the indexes from 0 seconds to 300 seconds.
This means that the average sleepiness at the second time point is predicted, and the predicted delay is 120 seconds. Therefore, when the Lag value indicated by the cycle of 60 seconds is given by [-1], it is actually 2 when the index is obtained after 300 seconds.
You are predicting drowsiness at 40 seconds,
The predicted delay is shown to be 60 seconds. However, since the delay is less than half of the average sleepiness cycle of 150 seconds,
Practically, changes in sleepiness can be predicted 90 minutes ahead on average.

On the other hand, as a behavioral characteristic which is an index different from blinking, attention is paid to the information on the movement of the line of sight such as the gaze of the mirror, that is, the horizontal movement of the eyeball. In particular, the horizontal movement of the eyeball and the occurrence of the aforementioned long blinking We investigated the estimation of arousal level using the ratio and the ratio. Specifically, the rate of occurrence of long blinks [Lrate],
Especially [Long10] and standard deviation of horizontal movement of the eyeball [EM
[H-SD], two types of indices are used as predictive models of the relationship with arousal (sleepiness), and from the sample data from a plurality of drivers D obtained by simulation, a prediction formula showing the relationship is expressed as Y = A Approximation by a linear equation of + B * Lrate + C * EMH-SD was performed, and the average of the prediction coefficients A, B, C when the relationship was highest and the relationship coefficient R at that time were obtained. Then, in accordance with Akaike's information criterion AIC described above, the prediction model with the smallest information criterion AIC was evaluated. As a result, when the prediction coefficients A, B, C are given as A = 2.812, B = 0.032, C = -0.695, that is, the prediction formula is Y = 2.812 + 0.032. Lrate-0.695 ・
When given as EMH-SD, the occurrence rate of the long-term blink described above [Long
It was confirmed that a higher relationship with arousal was obtained than when only 10] was used.

[0039] Specifically, the occurrence ratio of long blinks [Long10]
In the prediction of alertness based on the prediction formula using
The average of the multiple relationship coefficients was [0.71]. On the other hand, as described above, the occurrence rate of long blinks [Long10]
In addition, when the arousal level is predicted under the prediction formula in which the standard deviation [EMH-SD] of the horizontal movement of the eyeball is added, the average of the multiple relation coefficients becomes [0.75], and the accuracy of the prediction model improves. I was able to confirm that. Also, the information amount standard AI of the above Akaike
It was confirmed that the value of C also improved from [-134.3] to [-138.7].

7 and 8 show the measured drowsiness values obtained from the drowsiness expressions of a plurality of drivers D (subjects: Subj.1 to 12) and the occurrence rate of the long blink [Lo].
sleepiness prediction value [y] calculated according to the prediction formula based on [ng10], and sleepiness calculated according to the prediction formula based on the above long blink occurrence ratio [Long10] and standard deviation of horizontal movement of the eyeball [EMH-SD] It is shown as a predicted value [Y]. 7 and 8, the characteristics indicated by thick lines indicate the sleepiness prediction values [y] and [Y], and the characteristics indicated by thin lines indicate actual changes in drowsiness.

As shown in FIG. 7 and FIG. 8, respectively, the sleepiness prediction value [y] obtained as described above,
[Y] is the drowsiness (awakening degree) in advance of the actual drowsiness.
Therefore, it can be said that the driver's D personality is absorbed almost well and the sleepiness is predicted. Further, as shown in FIG. 8, it is shown that the estimation accuracy can be improved by calculating the drowsiness prediction value according to the prediction model that also uses the standard deviation [EMH-SD] of the horizontal movement of the eyeball of the driver D. .

Therefore, the occurrence ratio of the long blink of the driver D [L
[rate], other behavioral characteristics, specifically, the standard deviation [EMH-SD] of horizontal movement of the eyeball is also used to predict a decrease in arousal level (sleepiness). It is possible to judge the decrease in the arousal level of the driver D with extremely high prediction accuracy and effectively absorbing various personalities.

Particularly, as described above, the occurrence rate of long blinks [L having different physiological and physical meanings]
rate] and the standard deviation [EMH-SD] of the horizontal movement of the eyeball showing the behavioral characteristics, and moreover, the sleepiness prediction value [Y] can be easily calculated with high accuracy according to the prediction formula shown by the linear expression. it can. Moreover, the processing load is light, and the system configuration can be simplified. Furthermore, since it suffices to perform the calculation in accordance with the prediction formula represented by the linear expression, the processing speed can be increased and the detection time delay can be reduced.

Furthermore, while improving the prediction accuracy while absorbing individual differences, the processing is simplified and the processing speed is increased,
Further, the system configuration can be simplified and the cost can be reduced. Therefore, there is a great practical advantage as a wakefulness estimation device mounted on each vehicle. The present invention is not limited to the above embodiment. For example, the increase rate [r%] with respect to the reference time [To] for determining the evaluation time [Ts] of a long blink may be set based on more simulation results and the like. That is, the blink evaluation time [Ts] may be set so that a higher relationship can be obtained. In addition to the horizontal movement of the eyeball, it is of course possible to construct a prediction model by introducing information indicating other behavioral characteristics such as the forward tilt and the backward tilt of the head of the driver D described above. It is also possible to activate the braking mechanism of the vehicle to decelerate it by using the estimation result of the decrease in the awakening degree, and to activate the automatic driving mode based on the recognition of the white line on the road and the control of the distance to the other vehicle. . In addition, the present invention can be variously modified and implemented without departing from the scope of the invention.

[0045]

As described above, according to the present invention, the blink of the driver is detected, the long blink of a predetermined time or more is detected based on the detected blink time, and the blink within the predetermined evaluation period is detected. Along with the long blink occurrence ratio calculation means for obtaining the occurrence ratio of long blinks from the total number of blinks and the number of long blinks, the behavior detection means for detecting the behavior indicating the behavioral characteristics of the driver is provided. The arousal level determination means evaluates the arousal level of the driver according to a prediction formula obtained by regression analysis using a drowsiness prediction model based on the occurrence rate of the long blink and the behavior information of the driver.

In particular, a long blink is detected from the blink time of the driver according to the blink reference time [To] peculiar to the driver,
The occurrence ratio [Lrate] of long blinks with respect to the total number of blinks within a predetermined time is obtained, and as the behavior information indicating the behavioral characteristics of the driver, for example, the standard deviation [EMH- SD] and, for example, a prediction formula Y = A + B [Lrate] + C [EMH-SD] obtained by regression analysis using a sleepiness prediction model.
The sleepiness prediction value [Y] is calculated based on (A, B, and C are prediction coefficients) to evaluate the arousal level.

Therefore, it is possible to simply and highly accurately detect the driver's arousal level and, consequently, a decrease in the arousal level, and also to effectively absorb individual differences and estimate the arousal level with high accuracy. it can. In particular, a drowsiness prediction value is calculated based on a prediction formula showing the relationship between the awakening degree obtained by regression analysis using a drowsiness prediction model and the occurrence rate of long blinks, and the drowsiness prediction value is discriminated by a predetermined threshold to determine the arousal level. Is evaluated, the system configuration can be simplified and the processing speed can be increased while the estimation accuracy is sufficiently improved.

[Brief description of drawings]

FIG. 1 is a diagram conceptually showing a configuration of a vehicle equipped with a wakefulness estimation device according to an embodiment of the present invention.

FIG. 2 is a diagram showing a functional block configuration of the arousal level estimation device shown in FIG.

FIG. 3 is a diagram showing a flow of awakening degree estimation processing in the awakening degree estimation device shown in FIG.

FIG. 4 is a diagram showing a simulation result of a relationship between a plurality of indicators of arousal levels having different physical meanings and sleepiness objectively evaluated.

FIG. 5 is an objective evaluation with a plurality of awakening index (behavioral characteristics) having different physical meanings based on simulation results of a plurality of drivers (subjects: Subj.1, to 12). The figure which shows the case where a strong relationship is recognized with drowsiness.

FIG. 6 is a diagram showing a change in behavioral characteristics such as lane change, confirmation of a mirror, and blinker operation, which occur with the passage of driving time, by simulation.

FIG. 7 is a diagram showing a comparison between a measured value of drowsiness of a plurality of drivers D (subjects: Subj. 1, to 12) and a drowsiness predicted value obtained from a long blink occurrence ratio.

FIG. 8 shows a comparison between a measured value of drowsiness of a plurality of drivers D (subjects: Subj. 1 to 12) and a drowsiness prediction value obtained from long blinks and horizontal movement of the eyeballs in comparison. Fig.

[Explanation of symbols]

1 vehicle 2 TV camera 3 display 4 speakers 10 Image processing section 11 Blink detector 12 Blink time calculator 14 Blinking reference time calculator 15 Long blink time setting section 16 Long blink detector 17 Long blink occurrence ratio calculator 21 Eye movement detector 22 Standard deviation calculator 23 Arousal level prediction calculator 24 Awakening degree determination unit

Continuation of the front page (56) Reference JP-A-6-333183 (JP, A) JP-A-7-156682 (JP, A) JP-A-6-219181 (JP, A) JP-A-6-270711 (JP , A) JP-A-63-258226 (JP, A) JP-A-61-175129 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) B60K 28/00-28/16 G08G 1/16 A61B 5/18 G08B 21/06 G08G 1/16

Claims (2)

(57) [Claims] 1. A blink detecting means for detecting a blink of a driver, a means for detecting a long blink of a predetermined time or more based on the detected blink time, a total number of blinks within a predetermined evaluation period, and a long blink. shows the occurrence ratio calculating means, the eye movement of the driver's number from the seek occurrence ratio of long blink
Calculate standard deviation of horizontal movement of eyeball [EMH - SD]
Behavior detection means, awakening degree [Y] and occurrence rate of long blinks
[L rate ] and standard deviation of horizontal movement of the eyeball [EM
[H - SD] regression analysis of sleepiness prediction model
Prediction formula Y = A + B [L rate ] + C [EMH - SD] (A ,
The drowsiness prediction value [Y] is calculated based on B and C, and the drowsiness prediction value is calculated.
And a wakefulness determining unit that evaluates the wakefulness of the driver by discriminating the wakeup level by a predetermined threshold value . 2. A means for detecting the long blink, the reference time inherent blink from blinking time of the driver to the driver at the time of awakening [the To] look, percentage of this reference time of predetermined [r%] The awakening degree estimation device according to claim 1, wherein a blink exceeding the time [Ts] increased by only is detected as a long blink.