JP3480483B2 - Arousal level estimation device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention pays attention to a driver's blink time to simply and surely detect a driver's arousal level and, by extension, a decrease in the arousal level, and gives a warning to arouse driving attention. The present invention relates to an awakening degree estimation device suitable for.

[0002]

[Related Background Art] Recently, various systems have been developed for estimating driver's arousal level based on various information, and issuing a warning when a decrease in the arousal level is detected to call attention to driving. Has been done. One of the methods to estimate this type of arousal
For example, a driver's blink is used as an index for evaluation. For example, Japanese Patent Laid-Open No. 61-175129 discloses a method of counting the number of blinks per unit time to determine a decrease in arousal level. There is. However, when the number of blinks per unit time is used as an index for arousal level evaluation, there is a problem in that an error due to individual differences in blinking is likely to occur and the estimation accuracy cannot be improved.

Therefore, the present applicant previously filed Japanese Patent Application No. 8-913.
No. 24, and a paper [52.
A method of estimating a decrease in arousal level by focusing on the blink time of the driver was proposed as a method of evaluating arousal level during car driving (9632415)]. The awakening level estimation method focusing on this blinking time is based on the frequency distribution of the blinking time, and is a threshold value (blinking time) for determining a long blink from the standard blinking distribution time width and the central time of the distribution width. ) Is set, the ratio between the total number of blinks within a predetermined period and the number of occurrences of long blinks exceeding the above threshold value is obtained, and this ratio is evaluated to determine a decrease in arousal level.

According to such a method, there is an advantage that the individual difference such as blinking time and blinking frequency can be absorbed and the arousal level can be evaluated accurately.

[0005]

However, when the frequency distribution of the blinking time is obtained and the judgment threshold for a long blink is set according to the frequency distribution to estimate the arousal level as described above, it is assumed that the same person is the same person. However, since the frequency of blinks changes depending on the physical condition and the surrounding environment, it is necessary to collect blink information over a certain long period in order to improve the estimation accuracy. Moreover, it cannot be denied that the processing load is large when the frequency distribution is obtained from the collected blink information and analyzed. For this reason, it is strongly desired to easily and highly accurately estimate the arousal level and to simplify the system (awakening level estimation device) mounted on the automobile.

The present invention has been made in consideration of such circumstances, and an object thereof is to easily and efficiently detect a decrease in arousal level with certainty and evokes driving attention. Moreover, it is another object of the present invention to provide a wakefulness estimation device capable of simplifying the system configuration.

[0007]

In order to achieve the above-mentioned object, a wakefulness estimating apparatus according to the present invention uses, for example, a driver's face image taken by a camera to determine the blinking time (blink duration) of the driver. And t), and a reference time calculation means for obtaining a blink reference time [To] unique to the driver based on the blink time [t] detected when the driver is awake, According to an evaluation time setting means for increasing the reference time [To] by a predetermined ratio [r%] and setting a blink evaluation time [Ts] for evaluating a long blink, and a set blink evaluation time [Ts]. A long blink occurrence ratio [L] based on a long blink detection means for detecting a long blink from the blink time [t], and the total number of blinks [Ntotal] and the number of long blinks [Nlong] within a predetermined time.
The present invention is characterized by comprising an occurrence ratio calculating means for obtaining the rate] and an occurrence ratio awakening degree judging means for evaluating the driver's awakening degree [Y1] based on the long blink occurrence rate [Lrate].

Particularly, in the occurrence ratio arousal level determination means,
Based on the following prediction formula Y1 = A1 + B1 · Lrate (A1 and B1 are prediction coefficients) obtained by regression analysis of a drowsiness prediction model showing the relationship between arousal level [Y1] and long blink occurrence rate [Lrate] It is characterized in that the sleepiness prediction value [Y1] is calculated in accordance with the calculated sleepiness prediction value, and the sleepiness prediction value is discriminated by a predetermined threshold value to evaluate the awakening degree.

In addition to the above-mentioned configuration, the arousal level estimation apparatus according to the present invention further comprises blink average time calculating means for obtaining the average [BLdur] of the blink times within a predetermined time, and the average time of this blink [ BLdur], the average time awakening degree determining means for evaluating the driver's awakening degree [Y2], and the determination results of the average time awakening degree determining means and the occurrence ratio awakening degree determining means are comprehensively determined to perform the driving. And a wakefulness decrease detecting means for detecting a decrease in wakefulness of the person.

Particularly, in the average time awakening degree judging means,
A drowsiness prediction value based on a prediction formula Y2 = A2 + B2.BLdur (A2 and B2 are prediction coefficients) showing the relationship between the wakefulness and the average blink time [BLdur] obtained by regression analysis using a sleepiness prediction model. Y2] is calculated, and the sleepiness prediction value is discriminated by a predetermined threshold value to evaluate the arousal level,
The drowsiness prediction value [Y
1] and [Y2] are comprehensively evaluated to accurately (reliably) estimate a decrease in driver's arousal level.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the arousal level estimation device according to the present invention will be described below with reference to the drawings. Figure 1
2 conceptually shows the configuration of the embodiment apparatus mounted on the vehicle 1. In the figure, reference numeral 2 denotes a TV camera for imaging the face of the driver D, especially the eye region. Further, in the figure, 3 is a display (multi-information display device) for displaying various information as an image and presenting it to the driver D, and 4 is a speaker for outputting a voice message, an alarm sound or the like. These TV camera 2, display 3, and speaker 4 are incorporated in, for example, an instrument panel in front of the driver's seat.

A wakefulness estimation apparatus according to this embodiment is a TV
The driver's blink is detected from the face image of the driver captured by the camera 2 to estimate the driver's arousal level, a message is displayed via the display 3 when the awakening level is lowered, and the speaker 4 is used. It plays a role of issuing an alarm to call attention to driving. This device is realized by, for example, an electronic control unit (ECU) mainly including a microprocessor, and is schematically configured as shown in FIG.

That is, as shown in the functional block configuration of FIG. 2, the apparatus of the embodiment recognizes the driver's face image imaged and input by the TV camera 2 by the image processing unit 10, and, for example, predetermined. The partial image of the eye region is extracted in the cycle. The blink detection unit 11 detects the blink by detecting the opening and closing of the eyelid from the change of the image with time. Each time the blink detection unit 11 detects a blink, the blink time calculation unit 12 measures the blink time (blink time) [t] indicated by the eye closing time from the start to the end of the eye blink ( Blink time detection means). Such blinking time detection processing is repeatedly executed over a predetermined period under the control of the timer 13.

The blink reference time calculation unit 14 is strongly conscious of "starting (starting) driving," for example, at the beginning of driving, and it is considered that the driver D is sufficiently awake. When possible, based on the blink time [t] obtained by the blink time detection means (blink time calculation unit 12) over a predetermined period, the blink reference time during awakening that is unique to the driver D [To] is calculated (reference time calculation means). The reference time [To] is calculated as the average value of the blink time [t] in the predetermined period, for example, 5 minutes at the start of operation.

The long-blink time setting unit 15 sets a blink evaluation time [Ts] for detecting a long blink associated with a decrease in arousal level from among the blinks of the driver D. By increasing [To] by a predetermined ratio [r (%)], the above evaluation time [Ts] is defined as Ts = To + To · r / 100 (evaluation time setting means). The above ratio is, for example, 5%, 10%, 15%, 2
It is given as 0%, and is actually selected and set to 10% or the like based on the results of simulation experiments described later. That is, in the long-blink time setting unit 15, in general, when the awakening level is lowered, a long-time blink is likely to occur. The blink evaluation time [Ts] for detecting the blink is set.

The long blink detection section 16 determines the blink evaluation time from the blink time [t] of the driver D sequentially detected by the blink time detection means in accordance with the blink evaluation time [Ts] set as described above. A blink exceeding [Ts] is detected as a long blink. In the long blink occurrence ratio calculation unit 17, for example, the total number of blinks [Ntotal] evaluated by the long blink detection unit 16 and the number of long blinks [Nlong] obtained as a detection result by the long blink detection unit 16 are calculated. And are respectively counted. Then, the occurrence ratio [Lrate (%)] of long blinks is calculated as Lrate = 100.Nlong / Ntotal from the total number [Ntotal] of the blinks in a predetermined period and the number of long blinks [Nlong] in the blinks. ing.

The number of long blinks [Nlong] and the standard number of other blinks [Nnormal] are respectively counted, and the occurrence ratio of long blinks [Lrate = 100.Nlong / (Nlong + Nnormal)] is calculated. It is needless to say that Lrate] may be obtained.

The occurrence ratio awakening degree determination unit 18 determines the awakening degree and the occurrence ratio of the long blink [Lrate (%)], which is obtained by regression analysis using a drowsiness prediction model as described later.
The sleepiness prediction value [Y1] is calculated based on the prediction formula Y1 = A1 + B1.Lrate (A1 and B1 are prediction coefficients).
The prediction coefficients A1 and B1 are defined as 5 levels of arousal level, for example, as follows, and an increase rate [r] for setting the above-mentioned blink evaluation time [Ts] based on the results of various simulation experiments. %] Is 10% as will be described later, for example, A1 = 1.238, B1 = 0.046. That is, the prediction formula is given as Y1 = [1.238] + [0.046] Lrate, and the occurrence ratio [Lrate (%)] of the long blink in the blink of the driver D is calculated as described above. The sleepiness prediction value [Y1] is obtained by calculating the prediction formula.

The five levels of the arousal level are, for example, level 1 ... totally sleepless (the line of sight moves quickly and frequently. The blink is stable and the movement is active.) Level 2 ... somewhat sleepy (line of sight Moves slowly. Lips open. Level 3 ... seems sleepy (blinks slowly and often. Mouth moves.) Level 4 seems quite sleepy (has conscious blinks, slow blinks and gaze movements) ..) Level 5 ... Set as very sleepy (closes eyelids. Head tilts back and forth). Therefore, when the sleepiness prediction value [Y1] exceeds, for example, [3], it is estimated (determined) that the sleepiness is low and the sleepiness is low as shown below.

However, the drowsiness prediction value [Y1] obtained by the occurrence ratio awakening degree judging unit 18 based on the long blink occurrence ratio [Lrate] is given to the awakening degree lowering judging unit 19. It is used for judgment of deterioration. When the awakening degree decrease determination unit 19 detects a decrease in the awakening degree, the warning using the display 3 and the speaker 4 described above is issued, and the driver D is alerted to the driving attention. become.

On the other hand, the blink average time calculation unit 21 averages the blink time [t] in a predetermined period, that is, the average, based on the blink time [t] of the driver D sequentially detected by the blink time detecting means. Seeking the blink time [BLdur]. Since the blink average time calculation unit 21 performs substantially the same calculation as the blink reference time calculation unit 14, it can be realized as a function of the blink reference time calculation unit 14.

Then, the average time awakening degree determination unit 22 to which the average blink time [BLdur] obtained by the blink average time calculation unit 21 is input, is obtained by regression analysis using a drowsiness prediction model as described later, The sleepiness prediction value [Y2] is calculated based on the prediction formula Y2 = A2 + B2.BLdur (A2 and B2 are prediction coefficients) showing the relationship between the arousal level and the average blink time [BLdur (sec / time)]. . The prediction coefficients A2 and B2 are, for example, 5 when the level of the arousal level is 5
When defined in stages, it is given as, for example, A2 = -1.378 , B2 = 0.029 based on the results of various simulation experiments.

In this way, the average time awakening degree determination unit 22
The sleepiness prediction value [Y2] obtained on the basis of the average blinking time [BLdur] is also provided to the awakening degree lowering determination unit 19 and used for the determination of the lowering of the awakening degree. When the awakening degree decrease determination unit 19 detects a decrease in the awakening degree, the warning using the display 3 and the speaker 4 described above is issued, and the driver D is alerted to the driving attention. become.

In particular, in the awakening degree lowering determination unit 19, the sleepiness prediction value [Y1] obtained on the basis of the occurrence ratio [Lrate] of the long blink and the average value [BLdur] of the blinking time.
The sleepiness prediction value [Y2] obtained based on the above is comprehensively determined as will be described later, so that a decrease in the awakening degree of the driver D is detected, and a warning is issued based on the detection result.

The wakefulness lowering determination process shown in the above-mentioned functional block is actually executed under the microprocessor according to the control routine shown in FIG. That is, at the beginning of operation, the blink time [t] is measured, for example, for 5 minutes (steps S1, S2, S3), and the average of the blink times is calculated as the reference time [To] (step S4). Then, one unit of measurement target period is set to 5 minutes, and the subsequent blink time [t] is measured (steps S5 and S6).

Thereafter, the reference time [To] is set to 1 in accordance with the blink time [t] detected for 5 minutes each as described above.
The occurrence ratio [Lrate] of the long blink [Long10] detected under the blink evaluation time [Ts] set to be 0% longer is calculated (step S8), and the average blink time [B] at that time is calculated.
Ldur] is calculated (step S9). Then, if the occurrence ratio [Lrate] and the average blinking time [BLdur] are obtained, then the sleepiness prediction value [Y1],
[Y2] are calculated respectively (steps S10, S1
1) For example, the sleepiness prediction value (awakening degree) obtained thereby is displayed on the display 3 (step S12).
This drowsiness prediction value (awakening degree) is displayed on the display, for example, by displaying the awakening degree in a bar graph or changing the display color of the information according to the levels set in five stages as described above.

Then, the sleepiness prediction values [Y1] and [Y2] obtained as described above are evaluated (step S13). For example, when the level (prediction value) exceeds [3],
An alarm is issued to awaken the driver D and urge driver's attention (step S14). When the level of the arousal level is equal to or lower than [3], the process from step S5 described above is repeatedly executed, and the awakening level estimation process based on the blink information in the next 5 minutes is executed again.

Now, the calculation of the sleepiness prediction values [Y1] and [Y2] based on the occurrence rate [Lrate] of the long blink [Long10] and the average blink time [BLdur] and the evaluation thereof will be described in a little more detail. As the evaluation index of the arousal level, for example, changes over time of physiological indices such as electroencephalogram, electrocardiogram, and respiration, and changes over time of performance indices such as steering operation characteristics indicated by the steering wheel angle are used. Therefore, when the relationship between each of these indexes and the drowsiness of the driver D objectively evaluated by a third party at that time was investigated, the simulation experiment results shown in FIG. 4 were obtained.

In FIG. 4, Cz (α / β) and Pz (α / β)
Is the ratio of the spectral power values of the α-wave and the β-wave of the electroencephalogram obtained at the driver's D parietal Cz region and Pz region. Blink-No is the blink rate in 5 seconds obtained from the eye movement of the driver D, for example, Blink-Dur is the average blink time, HR is the heart rate of the driver D per minute, and Resp is the respiratory rate. Is. Furthermore, Speed is the running speed of the vehicle, Steer is the average steering wheel angle, and Steer-SD is the deviation of the steering wheel angle. Sleepiness is a drowsiness of the driver D evaluated by a third party, and MWS is a subjective evaluation of drowsiness by the driver D himself.

As shown in the result of the simulation experiment illustrated in FIG. 4, the sleepiness of the driver D (Sleepine
ss) increases with the passage of driving time, and the number of blinks of drowsiness (Blink-No) and blink time (Blink-Dur)
Is shown to have a strong relationship with. That is, as the sleepiness increases with the passage of time, the number of blinks tends to decrease and the blinking time tends to increase. By the way, Cz (α / β), Pz (α / β) with the passage of operating time
Also shows an increasing tendency, while HR and Resp show a decreasing tendency. The increase in the drowsiness of the driver D with the passage of time is generally caused by the monotonousness and familiarity of the driving operation, and further fatigue.

Therefore, in the present invention, for the purpose of evaluating the degree of awakening (drowsiness) without contacting the driver D, attention is paid particularly to the blinking time, the prediction accuracy thereof is improved, and individual differences are reduced. We proceeded with the study. Specifically, we focused on the long blinks that increase with the decrease in arousal level, and investigated the relationship between the occurrence rate of long blinks and the arousal level during a predetermined period. In particular, as the pre-processing, the blink evaluation time [Ts] for determining a long blink is set as the reference time [To], which is the average blink time during awakening, and the increase ratio of the blink time to this reference time [To] is defined as The blink evaluation time [Ts] was set by setting 5%, 10%, 15%, and 20%, respectively.

Under each of these blink evaluation times [Ts], a plurality of drivers D at a predetermined time (for example, 5 minutes)
Occurrence rate of long blink of (subject: Subj.1, ~ 12) [Lrat
e] was obtained as [Long5], [Long10], [Long15], and [Long20], respectively. Further, the average blinking time [BLdur] of each driver D in the predetermined time was also obtained. And the occurrence ratio of long blink [Long5], [Long10], [Long1
5], [Long20] and the average blink time [BLdur], while shifting the sample center time according to the Lag value given as a cycle of 60 seconds, for example, sleepiness (awakening degree)
When the relational coefficient R was calculated and the relational coefficient R was obtained under the condition that the relational coefficient was the largest, the results shown in FIG. 5 were obtained.

According to the result of the analysis of variance regarding the relation coefficient R shown in FIG. 5, a plurality of five indexes consisting of the occurrence ratio of four types of long blinks and the average blinking time having different blink evaluation times are determined. A significant difference is recognized in the average value across the driver D (Subj). Occurrence rate of particularly long blinks [Long20]
The relation coefficient R in Table 1 was small, and the analysis of variance regarding the Lag value confirmed that there was no significant difference between the above indexes. However, each of these indicators is physiologically different data.

By the way, the analysis of variance results shown in FIG. 5 are obtained when each index (probability of occurrence of long blinks) is obtained as an average for 300 seconds, and a drowsiness expression is obtained as an average for 60 seconds. Therefore, this analysis of variance means that with respect to the time characteristics of sleepiness prediction, for example, the average of indices from 0 to 300 seconds predicts the average of sleepiness from 120 seconds to 180 seconds. Delay is 120
It indicates that it is seconds. Therefore, when the Lag value indicated by the cycle of 60 seconds is given by [-1], it is actually 300.
When the second index is obtained, it means that the sleepiness at the time of 240 seconds is predicted, and the prediction delay is 60 seconds. However, the average period of actual sleepiness is 15
Since the delay is less than half of 0 second, it is possible to practically predict the change in drowsiness in almost real time.

Now, based on the analysis results shown in FIG. 5 and the consideration results thereof, a long blink occurrence ratio (index) and awakening degree (sleepiness) are modeled, and a plurality of drivers D from the drivers D obtained by simulation are modeled. From the sample data, a prediction formula showing the relationship is approximated by a linear expression Y1 = A1 + B1.Lrate Y2 = A2 + B2.BLdur, and the prediction coefficients A1, B1, A2, B2 when the relationship becomes the highest And the average of the relation coefficient R at that time, and the average of AIC's information standard AIC indicated by AIC = 2n logeσ + 2p (where n is the number of data and p is the number of regression coefficients) For example, the results shown in FIG. 6 were obtained.

That is, the relationship between the arousal level and the occurrence rate [Lrate] of long blinks, which is obtained by regression analysis using a prediction model, is expressed by a prediction formula of Y1 = A1 + B1.Lrate (A1 and B1 are prediction coefficients). Moreover, the relationship between the arousal level and the blinking time is expressed by a prediction formula Y2 = A2 + B2 · BLdur (A2 and B2 are prediction coefficients), and the relationship coefficient is calculated based on a plurality of sample data obtained by simulation. R
The prediction coefficients A1, B1, A2, B2 at which the value of the value becomes the highest were obtained, and the relationship coefficient R and the information amount reference AIC at that time were obtained. As a result, in this example, the information amount standard AI
The prediction model with the smallest C is the blink evaluation time [T
It was confirmed that the occurrence ratio [Long10] of the long blink when [s] was set as 10% increase of the reference time [To]. It was also confirmed that the prediction model that gives the largest relationship coefficient R is when the blink time [Blink-Dur] is focused.

Therefore, in the present invention, based on the above-mentioned findings,
As described above, the occurrence ratio of the long blink of the driver D [Lrat
e] is calculated, and the sleepiness prediction value [Y1] corresponding to the occurrence ratio [Lrate] is calculated according to a prediction formula based on a prediction model showing the relationship between the arousal level and the occurrence ratio. Then, by evaluating the sleepiness prediction value [Y1], it is determined whether or not the awakening degree of the driver D has decreased. Further, the average blinking time [BLdur] of the driver D is obtained, and the sleepiness prediction value [Y2] corresponding to the above average blinking time [BLdur] is obtained according to the prediction formula based on the prediction model showing the relationship between the awakening degree and the average blinking time. ing. And by evaluating this sleepiness prediction value [Y1],
It is determined that the driver D's arousal level has decreased.

FIG. 7 shows a plurality of drivers D (subject: Subj.1,
~ 12) Actually measured drowsiness calculated from drowsiness expression,
As described above, the relationship with the sleepiness prediction value [Y1] obtained from the long-blinking occurrence ratio [Lrate] is shown in comparison. It should be noted that in FIG. 7, the characteristic indicated by the thick line indicates the sleepiness prediction value [Y1], and the characteristic indicated by the thin line indicates the actual fluctuation of the drowsiness. As shown in FIG. 7, it can be understood that the sleepiness prediction value [Y1] obtained as described above predicts the actual sleepiness and predicts the change in the sleepiness (awakening degree). Moreover, it can be said that the driver's D personality is absorbed almost satisfactorily and sleepiness is predicted.

Further, FIG. 8 shows the plurality of drivers D (examinee:
Subj. 1, to 12) shows the relationship between the actual measured value of sleepiness and the sleepiness predicted value [Y2] obtained from the average blink time [BLdur] as described above. In FIG. 8 as well, the characteristic indicated by the thick line indicates the sleepiness prediction value [Y2], and the characteristic indicated by the thin line indicates the actual fluctuation of the drowsiness. As shown in FIG. 7, the sleepiness prediction value [Y2] obtained based on the average blink time [BLdur] is slightly inferior in accuracy to the sleepiness prediction value [Y1] described above. It can be seen that, to some extent, the actual drowsiness is anticipated and the change in the drowsiness (awakening degree) is well predicted.

However, as shown in FIGS. 7 and 8, the sleepiness prediction values [Y1], [Y] obtained as described above are obtained.
[2] can predict the decrease in the arousal level as a whole, but the prediction accuracy may be insufficient for a specific driver D (individual). Specifically, in the comparative example shown in FIG.
In the driver D indicated by bj.1 and the driver D indicated by Subj.8, the sleepiness prediction value [Y1] obtained from the long blink occurrence ratio [Lrate] sufficiently predicts actual sleepiness. Absent. In the comparative example shown in FIG. 8, Subj.5, Subj.
For driver D indicated by 7, Subj.10, Subj.11, sleepiness prediction value [Y obtained from average blink time [BLdur]
2] does not fully predict actual sleepiness.

Such inadequate prediction for a specific driver D is due to the peculiarity of the blinking of the driver D, and this peculiarity is the sleepiness prediction value obtained from the occurrence rate [Lrate] of long blinks. [Y1] and average blink time [BLdu
r] shows a reverse tendency to the sleepiness prediction value [Y2]. In other words, the peculiarity of blinking has an adverse effect on the drowsiness prediction accuracy based on the long blink occurrence ratio [Lrate] and the drowsiness prediction accuracy based on the average blinking time [BLdur], and rather on the other hand. When the prediction accuracy is poor, the other prediction accuracy tends to increase.

Therefore, as described above, the sleepiness prediction values [Y1] and [Y2] obtained respectively from the occurrence rate [Lrate] of the long blink of the driver D and the average blink time [BLdur] are comprehensively determined, and According to the present device for predicting a decrease in arousal level (drowsiness), it is possible to determine a decrease in awakening level of the driver D with very high prediction accuracy and effectively absorbing various personalities. . In particular, as described above, the occurrence rate of long blinks [L
rate] and average blink time [BLdur], and 1
A sleepiness prediction value [Y
Since [1] and [Y2] can be calculated, the processing load is light and the system configuration can be simplified. Moreover, it becomes possible to increase the processing speed,
The detection time delay can be reduced.

In other words, the prediction accuracy is increased while absorbing the individuality, and then the processing is simplified and the processing speed is increased,
Further, the system configuration can be simplified and the cost can be reduced. Therefore, there is a great practical advantage as a wakefulness estimation device mounted on each vehicle. The present invention is not limited to the above embodiment. For example, the reference time [To] of the blink evaluation time [Ts] for detecting a long blink.
The increase rate [r%] with respect to can be set based on more simulation experiment results,
It is not limited to the example described above. That is, the blink evaluation time [Ts] may be set so that a higher relationship can be obtained. In addition, when realizing a simple type of arousal level estimation device,
It is also possible to omit the above-described estimation of the awakening degree based on the average blink time [BLdur].

Further, instead of the average blinking time [BLdur], other behavioral characteristics of the driver, such as horizontal movement of the eyeball indicating the movement of the line of sight, backward and forward leaning of the head, and further yawning, sighing, and lips It is also possible to detect behaviors such as relaxation of the human body, estimate the awakening degree from a prediction model based on these information, and make a comprehensive determination with the evaluation result of the awakening degree based on the occurrence ratio of the long blink described above. Furthermore, by using the above-mentioned estimation result of the decrease in arousal level, the brake mechanism of the vehicle is operated to decelerate,
It is also possible to activate the automatic driving mode based on the recognition of the white line on the road, the control of the distance to another vehicle, and the like. In addition, the present invention can be variously modified and implemented without departing from the scope of the invention.

[0045]

As described above, according to the present invention, the blinking time of the driver is detected, the reference time of blinking unique to the driver is obtained based on the blinking time at the time of awakening, and this reference time is set. Detecting a long blink of the driver according to the blink evaluation time set by increasing by a predetermined ratio, the driving based on the occurrence ratio of long blinks obtained from the total number of blinks and the number of long blinks within a predetermined time Since the arousal level of the person is evaluated, the personality can be absorbed easily and effectively,
The arousal level can be estimated with high accuracy.

In particular, a drowsiness prediction value is calculated based on regression analysis of a prediction formula by a drowsiness prediction model showing a relationship between the awakening degree and the occurrence rate of long blinks, and the drowsiness prediction value is discriminated by a predetermined threshold to determine the awakening degree. Since the evaluation is performed, the system configuration can be simplified and the processing speed can be increased while sufficiently increasing the estimation accuracy. Further, as described in claim 3, a means is further provided for calculating an average of blinking times within a predetermined time, and the driver's arousal level is evaluated based on the average blinking time. The average blinking time is obtained. Since the awakening degree of the driver is detected by comprehensively determining the awakening degree and the awakening degree obtained from the occurrence ratio of the long blink, a high-precision awakening using two types of indicators having different physical meanings. Degree estimation,
It has a great effect in practical use such as being easily performed.

[Brief description of drawings]

FIG. 1 is a diagram conceptually showing a configuration of a vehicle equipped with a wakefulness estimation device according to an embodiment of the present invention.

FIG. 2 is a diagram showing a functional block configuration of the arousal level estimation device shown in FIG.

FIG. 3 is a diagram showing a flow of awakening degree estimation processing in the awakening degree estimation device shown in FIG.

FIG. 4 is a diagram showing a simulation experiment result of a relationship between a plurality of indicators of arousal levels having different physical meanings and objectively evaluated sleepiness.

FIG. 5 shows a relation coefficient R when a plurality of drivers (subjects: Subj.1 to 12) have long blink occurrence rates and a relation coefficient between average blink time and drowsiness (wakefulness) is the largest. FIG.

[Fig. 6] Occurrence ratio of long blinks (index) and arousal level (sleepiness)
The figure which shows the relationship between the prediction formula at the time of making into a prediction model, the average of the relation coefficient R, and information amount standard AIC.

FIG. 7 is a diagram showing a comparison between a measured value of drowsiness of a plurality of drivers D (subjects: Subj. 1 to 12) and a predicted drowsiness value obtained from the occurrence ratio of long blinks.

FIG. 8 is a diagram showing the relationship between the measured drowsiness values of a plurality of drivers D (subjects: Subj.1, to 12) and the drowsiness prediction value obtained from the average blink time, for comparison.

[Explanation of symbols]

1 vehicle 2 TV camera 3 display 4 speakers 10 Image processing section 11 Blink detector 12 Blink time calculator 13 timer 14 Blinking reference time calculator 15 Long blink time setting section 16 Long blink detector 17 Long blink occurrence ratio calculator 18 Occurrence ratio Arousal level judgment unit 19 Decreased awakening level 21 Blink Average Time Calculation Unit 22 Average time awakening degree determination unit

Continuation of the front page (56) Reference JP-A-6-333183 (JP, A) JP-A-7-156682 (JP, A) JP-A-6-219181 (JP, A) JP-A-6-270711 (JP , A) JP 63-258226 (JP, A) JP 61-175129 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) B60K 28/00-28/16 A61B 5/18 G08B 21/06 G08G 1/16

Claims (3)

(57) [Claims] 1. A blink time detecting means for detecting a blink time of a driver, a reference time calculating means for obtaining a blink reference time unique to the driver based on the blink time when the driver is awake. An evaluation time setting means for setting a blink evaluation time for evaluating a long blink by increasing the reference time by a predetermined ratio, and a long blink detection means for detecting a long blink from the blink time according to the set blink evaluation time. ,
Occurrence ratio calculating means for obtaining the occurrence ratio of long blinks from the total number of blinks and the number of long blinks within a predetermined time, and the occurrence ratio awakening degree for evaluating the awakening degree of the driver based on the occurrence ratio of the long blinks. Judgment means and within a predetermined time
Blink average time calculating means for obtaining the average of the blink time
And evaluate the driver's arousal level based on the average time of this blink.
And an average time awakening degree determining means for evaluating the occurrence rate.
Using the awakening degree determining means and the average time awakening degree determining means
Awakening level decrease detecting means for detecting a decrease in driver's awakening level
Awakening level estimation apparatus comprising the and. 2. A drowsiness prediction value obtained based on a regression analysis of a prediction formula by a drowsiness prediction model showing a relationship between the awakening level and the occurrence ratio of a long blink is valved by a predetermined threshold value. The arousal level estimation device according to claim 1, wherein the arousal level is evaluated separately. 3. The average time awakening degree determining means discriminates a sleepiness prediction value obtained based on a regression analysis of a prediction formula by a sleepiness prediction model showing a relationship between the awakening degree and the average time of blinking, with a predetermined threshold value. The arousal level estimation device according to claim 1 , wherein the arousal level is evaluated.