JPH08225028A - Awakening condition detecting device for driver - Google Patents

Abstract

PURPOSE: To detect a driving asleep and the like adequately, by detecting the lateral displacement amount of a vehicle from an object running locus, as well as extracting a winking parameter from the movement of the eyelid of a driver, and detecting the awakening condition of the driver, depending on the mutual relation of the winking parameter and the lateral displacement amount of the vehicle. CONSTITUTION: A winking detector 10 detects the eye closing condition of the eyelid of a driver by an image processing, and the eyelid opening and closing is decided through a TV camera to photograph the face image of the driver 11; an A/D converter 13; a frame memory 14; and the operation process of an image data; and it is composed of the above members and a CPU 15 to output an eye closing data. And in an awakening condition detector 40, the awakening condition of the driver is detected from the eye closing time extracted from the eye closing data output from the winking detector 10 in a winking parameter extractor 20; and the lateral displacement amount to show how much amount the present position of the car is slipped from the object running locus, detected by a vehicle condition detector 30; and it is to be reported.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a driver's arousal level, and more particularly, to a driver's arousal level capable of detecting a dozing driving state from a blink parameter and a vehicle lateral displacement amount. The present invention relates to a driver awakening level detection device.

[0002]

2. Description of the Related Art Conventionally, there has been a technique for estimating the arousal level by detecting how much blink parameters (eye closing time, eye closing frequency, eye closing interval, etc.) have changed from their normal values, and detecting a dozing state. Proposed. However, there is a problem in that it is not possible to accurately detect the dozing state because the difference in blink parameter with respect to the normal value varies greatly among subjects. As a method of correcting this individual difference, a method of varying the threshold value for determining the magnitude of the change in the blink parameter has been proposed, but there is no standard for how much the initial value should be set, It is insufficient as a method for correcting individual differences.

Further, based on statistical assumptions from the average value and the variance value of blink parameters in normal times, (average value)
A method of determining a value of + (dispersion value) to (average value) + 2 × (dispersion value) as a threshold value (Japanese Patent Laid-Open No. 60-592).
However, there is no reason to regard the threshold value determined based on statistical assumptions as a decrease in the alertness of an individual.

Further, there has been proposed a method (Japanese Patent Laid-Open No. 5-92039) in which the characteristics of an individual are estimated based on the work performance and the awakening level corresponding to the individual difference is calculated. In this method, a task such as pressing a button is performed for a stimulus such as video, sound, or vibration, and the correlation (regression line) between the reaction time for this task and the physiological amount (electroencephalogram, heartbeat, blink frequency) Ask in advance. Then, based on the correlation between the physiological amount and the reaction time, the reaction time for the detected physiological amount is obtained, the arousal level is estimated from the magnitude of the obtained reaction time, and a decrease in the arousal level is determined from the estimated arousal level. There is. Therefore, it is possible to estimate the arousal level according to the individual characteristics, but it is necessary to learn in advance the individual characteristics such as the correlation between the reaction time to the stimulus and the physiological amount, and the actual reaction time to the stimulus. Since the characteristics of an individual obtained by a work different from the driving state are used, there is a problem that it does not follow the actual driving state, and it is not suitable for detecting the drowsiness of a car driver.

[0005]

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and does not require learning the characteristics of an individual in advance and that he / she can doze while driving while driving. It is an object of the present invention to provide a driver's arousal level detection device capable of appropriately detecting the vehicle.

[0006]

In order to achieve the above object, the invention of claim 1 is to extract the blinking parameter from the movement of the eyelid of the driver, and the lateral deviation amount of the vehicle from the target travel locus. Lateral displacement amount detecting means for detecting, a calculation means for calculating the correlation between the blink parameter and the lateral displacement amount of the vehicle based on the blink parameter and the lateral displacement amount, and based on the calculated correlation And a wakefulness detecting means for detecting the wakefulness of the driver.

The invention according to claim 2 further comprises extraction means for extracting blink parameters from the movement of the eyelids of the driver.
Lateral deviation amount detecting means for detecting the lateral deviation amount of the vehicle from the target travel locus, and regression representing the relationship between the blink parameter and the lateral deviation amount of the vehicle based on the blink parameter and the lateral deviation amount of the vehicle. Regression formula calculation means for calculating a formula, significance test means for testing whether the regression formula is significant, and awakening degree detection means for detecting the awakening degree of the driver based on the regression formula tested as significant It is configured to include ,.

[0008]

According to the experiments conducted by the present inventors, it has been confirmed that there is a high correlation between the blink parameter extracted from the movement of the eyelid of the driver and the lateral deviation amount of the vehicle from the target traveling locus. Has been done. As blink parameters,
The eye closing time, the eyelid opening degree, the eyelid movement amplitude, the blinking frequency, the blinking interval time, or an average value thereof can be used.

Therefore, in the present invention, the correlation between the blink parameter and the lateral displacement amount of the vehicle is calculated based on the blink parameter and the lateral displacement amount, and the driver's awakening degree is calculated based on the calculated correlation. It is detecting. This awakening degree is obtained from the calculated correlation and can be detected by comparing the value of the blink parameter with respect to the predetermined reference value of the lateral displacement amount and the current value of the blink parameter. .

Further, a regression equation representing the relationship between the blink parameter and the lateral displacement amount of the vehicle is calculated based on the blink parameter and the lateral displacement amount of the vehicle, and whether the calculated regression formula is significant or not is tested. It is also possible to detect the awakening degree of the driver based on the regression equation that is tested as significant. Even in this case, it is possible to detect the arousal level by comparing the value of the blink parameter with respect to the predetermined reference value of the lateral displacement amount obtained from the regression equation and the current value of the blink parameter. .

When a regression line (first-order regression equation) is obtained as the correlation, the value of the blink parameter with respect to the predetermined reference value of the lateral deviation obtained from the regression line and the current blinking. The arousal level can be detected by comparing with the value of the parameter. Further, the arousal level can be detected from the magnitude of the slope of the regression line.

Further, the blink parameter detecting means inputs a grayscale image of a region including eyes, extracts a one-dimensional image showing a grayscale change of pixels on a reference line in the blinking direction in the region, and at the same time, the one-dimensional image. Edge image calculation means for calculating a one-dimensional edge image representing the magnitude of the grayscale change for each pixel on the reference line, and an upper eyelid and an eyeball based on the extreme value of the grayscale change in the one-dimensional edge image. Eyelid detection means for detecting a first boundary point indicating a boundary and a second boundary point indicating a boundary between the lower eyelid and the eyeball, and a blink parameter based on the detected first boundary point and second boundary point. Parameter calculation means for calculating
Can be included.

In the above arrangement, the edge image calculation means inputs the grayscale image of the area including the eyes and extracts the one-dimensional image representing the grayscale change of the pixels on the reference line in the blinking direction in this area. Since a grayscale image is used instead of a color image, it can be used in a wide environment without requiring a white light source with a sufficient amount of light.

Further, the edge image calculating means calculates the magnitude of the grayscale change on the reference line for each pixel based on the one-dimensional image to calculate a one-dimensional edge image representing the magnitude of the grayscale change for each pixel, The eyelid detection means is a first boundary point indicating the boundary between the upper eyelid and the eyeball and a second boundary indicating the boundary between the lower eyelid and the eyeball based on the extreme value of the grayscale change in the one-dimensional edge image.
Detect the boundary points of. Since the one-dimensional edge image is processed using the extreme value of the grayscale change without being binarized, it is less likely to be affected by individual differences such as lighting conditions and the facial structure of the target person.

The parameter calculation means calculates the blink parameter based on the extracted first boundary point and second extracted boundary point. At this time, the distance between the extracted first boundary point and the second boundary point is detected as the eyelid opening, or the time when the eyelid opening is minimized is detected as the eye-closing time, The interval during which the eyelid opening is minimized can be detected as the blink occurrence interval.

[0016]

Embodiments of the present invention will be described below in detail with reference to the drawings. In the present embodiment, the arousal level is determined using the correlation between the average eye-closing time as a blink parameter and the average value of the lateral displacement amounts of the vehicle (average lateral displacement amount).

FIG. 7 is an example of a graph showing the time transition of the average eye-closing time cn of blinking and the average lateral displacement amount dn of the vehicle for each unit measurement time T of the subject obtained in the experiment. Average eye closure time cn indicating the time the eyes are closed
Means that the awakening degree is lowered, and that an increase in the average lateral displacement amount dn indicating a meandering traveling state of the vehicle means that the vehicle cannot travel safely.

FIG. 8 shows a correlation diagram between the average eye closing time cn and the average lateral deviation amount dn of this experimental result. The straight line in FIG. 8 shows the average eye closing time cn and the average lateral deviation amount dn. A bivariate linear regression line is shown. As can be seen from FIG. 8, there is a high correlation between these two variables. According to an experiment by the present inventors, the bivariate correlation coefficient among the five subjects was 0.72 to 0.95. From this, it can be seen that one value can predict the other value. That is, it is possible to evaluate the awakening degree indicated by blinking driver's physiological amount, which is an average lateral deviation amount dn, which is a vehicle state amount indicating whether or not the vehicle can travel safely.

FIG. 1 shows the basic configuration of this embodiment. In this embodiment, a blink detection unit 10 that detects a closed state of the eyelids and outputs closed eye information, and a blink parameter extraction unit 20 that extracts a closed-eye time that is a blink parameter from the closed eye information of the eyelids output from the blink detection unit 10. , A vehicle state detection unit 3 that detects a lateral displacement amount Y indicating how much the current position of the automobile deviates from the target travel locus.
0. The awakening degree detection unit 40 is composed of a CPU or the like that keeps track of individual characteristics from the data obtained during the driving work of the automobile.

Next, the configuration of each part will be described in detail. The blink detection unit 10 detects the eyelid closed state of the driver by image processing, and as shown in FIG.
Image data converted by the A / D converter 13 and the A / D converter 13 for converting the video signal from the TV camera 12 installed in the vehicle interior to capture the face image of the And a CPU 15 that outputs eye-closure information indicating that the eyelids are closed by determining whether the eyelids are open or closed by calculating the image data.

The blink parameter extraction unit 20 calculates the average eye closing time cn, which is the average value of the eye closing time of the blinks, for each unit measurement time T from the eye closing information of the eyelids output from the blink detecting unit 10. As shown in, the eyelid closing time calculation means for calculating the time (eyelid closing time) from the time when the eyelids start to close until the eyelids close and reopen from the eyelid closing information obtained by the blink detection unit 10. 21, time measuring means 2 for measuring the unit measurement time T
2. It is composed of an average eye-closure time calculation means 23 composed of a CPU or the like for calculating an average value of eye-eye closure time (average eye-closure time) for each unit measurement time T.

By the way, generally, at the start of driving (start of data acquisition), the driver's awakening degree is high, and the driver does not start the driving in a low awakening degree. Then, as the driving is continued, the state (a) having a high arousal level shifts to the state (c) in which the awakening level is lowered and the driving level is dangerous, through the state (b) in which the arousal level tends to decrease. This (a)-(b)-
The state of state transition (c) (elapsed time, degree of decrease in arousal level, etc.) differs for each driver, or even for the same driver, depending on the physical condition of the driver and other conditions.

Further, according to the experiments by the present inventors, the transition from the high awakening state to the inoperable state is
For example, it occurs within a few minutes to 10 minutes or more, and does not suddenly fall into the inoperable state from the high awakening state.
Therefore, in the present embodiment, as described above, the time shorter than the predetermined time is measured as the unit measurement time T (for example, 1 minute).

The vehicle state detecting section 30 detects a lateral deviation amount Y which is a difference between a target traveling locus which is a vehicle position to travel and an actual present position based on the steering wheel steering information of the driver and the vehicle speed information. Further, the average lateral displacement amount dn, which is the average value of the lateral displacement amounts Y for each unit measurement time T, is detected. The lateral displacement amount Y is, for example, disclosed in Japanese Patent Laid-Open No. 5-85221, in the steering amount of the driver and the vehicle speed information (vehicle speed, vehicle acceleration, etc.) which is a physical quantity related to the vehicle speed. It can be detected based on. As shown in FIG. 4, the vehicle state detecting unit 30 includes a steering amount detecting means 31 for detecting the steering amount of a steering wheel, a vehicle speed (vehicle speed).
Based on the steering amount detected by the vehicle speed detecting unit 32 and the steering amount detecting unit 31 and the vehicle speed detected by the vehicle speed detecting unit 32, the vehicle current from a target travel locus on which the vehicle should travel is described later. Lateral deviation amount calculation means 33 for calculating the lateral deviation of the position, average lateral deviation amount calculation means 34 for calculating the average value of the lateral deviation amount Y for each unit measurement time T, and timing for measuring the unit measurement time T. It comprises means 35.

The unit measurement time T measured by the time measurement means 22 in the blink parameter extraction section 20 and the unit measurement time T measured by the time measurement means 35 in the vehicle state detection section 30 are the same time intervals (for example, 1 Minutes)
In addition, it is desirable to perform the statistical processing to be described later by setting the start and end at the same time. Therefore, the timekeeping means 22 and the timekeeping means 35 can be combined into one.

The operation of this embodiment will be described below. In the blink detection unit 10, first, the face image of the driver 11 is displayed as T
Take a picture with the V camera 12. At this time, the image may be taken only by the TV camera 12, but in order to reduce the influence of ambient light in the surroundings, light other than visible light is emitted so as not to affect the driver, for example, infrared light (not shown). It is effective to illuminate the driver's face by causing the illumination unit including a strobe to emit light in synchronization with the shooting by the TV camera 12.
As the illumination unit, an infrared LED that emits infrared light may be used instead of the infrared strobe. Moreover, if continuous light is emitted from the illumination unit, synchronization with the TV camera 12 is not necessary,
Simpler configuration.

Further, the TV camera 12 is, for example, 1/1
If the infrared LED is pulsed in synchronism with the high-speed shutter operation in about 000 seconds, a stable image can be obtained even in an environment with a lot of ambient light such as outdoors.

The face image thus photographed is taken into the A / D converter 13 as a video signal, A / D converted, and the two-dimensional digital image is stored in the frame memory 14. FIG. 12A shows a two-dimensional digital image of the face stored in the frame memory 14.

Next, image processing by the CPU 15 will be described with reference to FIG. In step 102, the digital image stored in the frame memory 14 is fetched, and in step 104, for example, a small area in which eyes are present is searched from the face image by image processing by the template matching method, and in step 106, the eyes are included. A small area is extracted as an eye image. FIG. 6A shows an example of the eye image.

When the eye image as the original image as shown in FIG. 6A is obtained, the luminance change of the original image on the center line L becomes as shown in FIG. 6B. In step 108, this FIG.
A one-dimensional original image as shown in (b) is extracted and input, edge processing is performed, and in step 110, as shown in FIG. 6 (c), a one-dimensional representing the magnitude of the grayscale change for each pixel on the reference line. Calculate the edge image.

An operator for performing edge processing may be, for example, an operator for obtaining a simple difference as shown in (1) of the following table. In addition, an operator having a smoothing function as shown in (2) can reduce the influence of noise.

[0032] A Sobel operator, a Pelewitt operator, or a Roberts operator may be used as an operator that performs edge processing.

Here, the opening of the eyelid is defined as the distance h between the boundary point A between the upper eyelid and the eyeball and the boundary point B between the lower eyelid and the eyeball on the center line L of FIG. 6A. For example, in order to measure the opening degree, it is sufficient to detect the boundary points A and B, respectively. In general, the eyeball portion is photographed with a smaller reflectance and darker than the eyelid that is the skin portion. Therefore, at the preceding boundary point A, from the top to the bottom, light to dark, and at the subsequent boundary point B, The brightness changes suddenly from dark to bright from top to bottom. At such a sudden brightness change point, the value of the edge image has an extreme value point. Therefore, if a point having an extreme value is detected on the one-dimensional edge image as shown in FIG. It is possible to measure the opening.

However, such a sudden brightness change point occurs not only on the boundary between the eyelid and the eyeball, but also on the boundary between the pupil and the iris, and the wrinkles and shadows around the eyes. Therefore,
In the present embodiment, the opening of the eyelid is selected by selecting the point having the extreme value on the edge image by the method described below so as to prevent erroneous detection due to wrinkles and shadows around the eyes.

First, at step 112, on the one-dimensional edge image of FIG. 6C, a point Pi (i = 0,1) having an edge value having a positive maximum value and a minimum edge value having a negative value. A point Mi (i = 0,1) having a value (absolute value is large) is calculated. Hereinafter, both the maximum value and the minimum value are collectively referred to as an edge extreme point.

Further, in step 114, as the initial position of the search, one of these edge extreme points Pi and Mi shown in FIG.
(B) The darkest point C in the one-dimensional image on the middle dotted line L
, And the innermost edge extreme point combination is selected one by one from the point Pi and the point Mi. In the case of FIG. 5, point P
The combination of 0 and the point M0 corresponds to the edge extreme point which is the initial position of the search.

Next, in step 116, step 11
The first search section is determined based on the edge extreme value point searched in 2 and the edge extreme value point that is the initial position of the search. That is, the search section is determined such that the positive extreme point is searched upward and the negative extreme point is searched downward so as to search the outer edge extreme point from the initial position of each search. In FIG. 6, since the points P1 and M1 are the next edge extreme points,
The area between points P0 and P1 is the next upward search section,
The area between points M0 and M1 is the next downward search section.
In the search section thus determined, it is determined in step 118 whether the sign of the edge value is inverted, and if it is not inverted, it is determined in step 120 whether a new edge extreme point exists. That is, it is determined whether or not a new search section is found. Then, the search is repeated upward and downward, respectively, until no new search section is found or the sign of the edge value is reversed in the search section (steps 116, 118, 120). In FIG. 6, the point P
Since the edge value is always positive between 0 and the point P1 and the edge value is always negative between the points M0 and M1, the points P1 and M1 are set as new starting points, and the search is performed. Will be repeated. Similarly, based on the above search results, the search sections are determined to be upward and downward, respectively, as shown in FIG.
Then, there is no new edge extreme point above P1 and below M1. Therefore, it is determined that the search has ended, and in step 122, the points P1 and M1 are defined as the boundary points A and B between the eyelids and the eyeballs.

Then, in step 124, the boundary point A,
It is determined whether the eyelids are closed or open by counting the distance between B to calculate the eyelid opening and determining whether the eyelid opening is minimized. And
Outputs eye-closure information indicating that the eyelids are closed.

With this processing, in this embodiment, it is possible to extract the boundary point between the eyelid and the eyeball regardless of individual differences such as the facial structure of the individual, and to accurately determine the eyelid closed state. It will be possible.

The following method may be used to determine whether the eyelids are closed.

First, the face image stored in the frame memory is fetched, and the face image is divided into two as shown in FIG.
The binarization is performed, and the region where the eyelids are present is extracted from the binary image of the face based on the left-right symmetry of the face, the positional relationship between the hair, the eyebrows, and the eyelids. Next, a grayscale image (FIG. 12C) showing the edge (outline) of the eyelid image is obtained from the area corresponding to this eyelid area from the face image. Next, a threshold value for the density at which the contour of the eyelid can be detected optimally is determined, and this threshold value is compared with the density of the grayscale image to obtain a binary image of the contour of the eyelid (FIG. 12 (4)). To get Then, the distance between the upper and lower eyelids is measured from the obtained eyelid contour binary image,
It is determined whether or not this distance is minimized to determine whether the eyelids are open or closed.

In the blinking parameter extraction section 20, the eyelid closing time calculating means 21 calculates the time during which the eyelids are continuously closed, based on the eyelid opening / closing information, and uses this as the eyelid closing time. Further, the average value of the eye-closing time of each blink that occurs at each unit measurement time T from the measurement start time measured by the time-measuring means 22 is calculated by the average eye-closing time calculating means 23, and this is set as the average eye-closing time cn. Here, n is a subscript of a variable and represents a serial number for each unit measurement time T from the start of measurement. That is, when the elapsed time from the measurement start time is t, n = t / T (1).

Steering amount detecting means 31 of the vehicle state detecting section 30.
In the above, the steering amount of the driver's steering wheel is detected, and the corrected steering amount δc is extracted from the detected steering amount by subtracting the steering amount necessary for maintaining traveling in a lane on a curve or the like,
The corrected steering amount δc information is output.

As shown in FIG. 13, the extraction of the corrected steering amount is performed for a short time (for example, 10 seconds) regardless of the road shape.
It is assumed that the road section traveling between is a straight road, and this straight road is approximated by a straight line connecting the start point and the end point of the road section, and the steering amount on this straight line is the reference (0 level, shown by the dotted line in FIG. 13). And The difference between the reference of the steering amount determined by this and the detected steering amount is defined as the corrected steering amount δc.

The vehicle speed detecting means 32 of the vehicle state detecting section 30 detects the vehicle speed U. The lateral displacement amount calculating means 33 calculates the second-order integral of the product of the corrected steering amount δc and the lateral acceleration gain Gy represented by the function of the vehicle speed U, as shown in the following equation (2). The deviation amount Y is calculated.

Y = (δc · Gy) / S ² (2) where S is a Laplace operator.

The calculated lateral displacement amount Y is input to the average lateral displacement amount calculating means 34, and the blinking parameter extracting section 20.
The average lateral displacement amount dn, which is the average value of the lateral displacement amounts Y, is calculated for each unit measurement time T in synchronization with the unit measurement time T in FIG.

As described above, the average eye-closure time cn of the driver's blink and the average lateral deviation amount dn from the target running trajectory of the vehicle are measured for each unit measurement time T.

Next, the detection processing routine by the CPU of the awakening degree detection unit 40 will be described with reference to FIG. First, in step S401, the elapsed time t from the start of measurement is set to 0,
The counter n is set to 1, and the variable sum used for obtaining the average value of the average eye closing time cn is set to 0, that is, various variables are set to initial values.

In step S402, the average eye-closure time cn, which is the output of the blinking parameter extraction unit 20, and the average lateral deviation amount dn, which is the output of the vehicle state detection unit 30, are read.

At the start of driving, the driver's awakening degree is high because the operation involves boarding, starting the engine, starting the vehicle, and the like. Therefore, in step S403,
It is determined whether the elapsed time t from the measurement start time is within a predetermined time td (for example, 3 minutes) in which it is determined that the awakening degree of the driver is high.

If the elapsed time t is within the predetermined time td in step S403, the average eye closing time cn is added to the variable sum in step S404.

In the next step S405, it is determined whether the elapsed time t is equal to the predetermined time td. If t = td,
That is, when the predetermined time td at which the driver's arousal level is determined to be high has elapsed from the measurement start time, step S40
In 6, the average value of the average eye-closing time cn within the predetermined time td is obtained, and this average value is set as the normal value co of the average eye-closing time during awakening of the driver currently measured.

In step S403, if the elapsed time t exceeds the predetermined time td, the arousal level may decrease. Therefore, in step S407, the unit measurement time from the measurement start time until the time t has elapsed. The average eye closing time cn for each T is divided by the normal value co, and this ratio cn / co is set as the average eye closing time ratio crn. This makes it possible to evaluate the magnitude of change in the average eye closing time cn with respect to the normal value co.

In step S408, a regression line is calculated using the average eye-closing time ratio crn and the average lateral displacement amount dn for each unit measurement time T obtained by the elapsed time t. Hereinafter, with reference to FIGS. 10A to 10C, description will be given based on the relationship between the continuation of driving and the awakening degree described above. The states (a) to (c) described above correspond to FIGS. 10 (a) to 10 (c), respectively.

As shown in FIG. 10 (a), in the state in which the awakening level does not change at this point, and the average eye closure time ratio crn and the average lateral deviation amount dn do not change with respect to their normal values, Data will be concentrated in the lower left of the scatter plot. Therefore, the average eye closing time ratio crn and the average lateral deviation amount d
No clear relationship is obtained with n. Also, the statistical test result of the regression line described later does not become significant (it becomes statistically meaningless).

On the other hand, when the arousal level tends to decrease, the average eye closing time ratio crn and the average lateral deviation amount dn change so as to increase, and as shown in FIG.
The rn and the average lateral displacement amount dn show a linear relationship rising to the right on the scatter diagram. That is, from the state of FIG.
In the process of shifting to the state of (b), the slope of the regression line increases with the passage of time, but the slope becomes almost constant as the data increases in the state where the arousal level tends to decrease. Then, as the driving is further continued and the arousal level decreases and the driving becomes dangerous, as shown in FIG. 10C, the point on the scatter diagram further shifts from the lower left to the upper right. At this time, the slope of the regression line hardly changes from the state of FIG. That is, in the state where the arousal level shown in FIG. 10 (b) tends to decrease, a significant regression line obtained by using the data from the start of driving to the time point is that the awakening level decreases as the driving continues. , Which is almost the same as the (significant) regression line of FIG. 10 (c) obtained by using the data until the driving becomes dangerous. Therefore, without using data until falling into such a dangerous state, that is, before reaching a dangerous state for driving, using only data from a state of high arousal state to a state of decreasing It is possible to obtain a regression line that correctly reflects each feature. In addition, the regression line is correctly obtained even when the above-mentioned states (a) to (b) and (c) change for each driver or even if the same driver has different physical conditions and other conditions. be able to.

In step S409, as will be described below, a test is made as to whether or not the obtained regression line is statistically significant. If the regression line is significant, it is possible to estimate the arousal level, assuming that the state (b) is in a tendency that the arousal level tends to decrease.

That is, in the regression line represented by the following equation (3) which can be represented by a set of bivariates (xk, yk) (where k = 1, 2, ... N), the following zero hypothesis Perform a test.

Null hypothesis H0: β1 = 0 Alternative hypothesis H1: β1 ≠ 0 y = α1 + β1x (3) Here, when the slope β1 of the straight line is 0, y = α1 for all x. , Y and x have no linear relationship.

Under the null hypothesis H0, β1 = 0. Therefore, when the predicted value of β1 is represented by βh1, the to expressed by the following equation (4) follows the t distribution with n-2 degrees of freedom. become.

To = βh1 / (√ (Ve / Sxx)) (4) However, Ve, Sxx, etc. are expressed as follows.

Ve = Se / (n−2) (5) Se = Syy−Sxy ² / Sxx (6)

[0064]

[Equation 1]

Therefore, when the condition of the following expression (12) is satisfied, the hypothesis H0 is rejected at the significance level α.

| To | ≧ t (n−2, α) (12) That is, when the significance level α is, for example, 5%, the value of the t distribution table is t (n−2,0.05). ), It holds that to ≧ (n−2,0.05) (13) does not hold the hypothesis that the slope β1 of the regression line is 0. Therefore, it can be confirmed that the regression line is significant, that is, statistically meaningful.

The statistical test may be performed by the following method which is mathematically equivalent to the method described above.

Under the hypothesis H0, the dispersion ratio Fo expressed by the following equation follows the F distribution with the degree of freedom (1, n-2).

Fo = VR / Ve (14) However, VR and Ve are represented by the following equations.

VR = SR (15) Ve = Se / (n−2) (16) SR = Sxy ² / Sxx (17) Se = Syy-SR (18) Note that Sxx, Syy, and Sxy are expressed by the above equations (7) to
It is similar to (9).

Therefore, when the following expression (19) is satisfied, the hypothesis H0 is rejected at the significance level α. Fo ≧ F (1, n−2, α) (19) If the calculated regression line is significant as a result of the test, the process proceeds to step S410, and if not significant, the process proceeds to step S411. In step S411, the obtained regression line is not used, but the regression line having the slope and intercept of the average value obtained in advance is used.

In step S410, as shown in FIG. 11, a reference value D (for example, 1) of a predetermined average lateral deviation amount, which means a dangerous traveling state, is obtained from the obtained regression line.
m), the average eye-closing time ratio R is calculated, and the average eye-closing time ratio R is used as an average eye-closing time ratio dangerous for driving. Ratio crn to ratio R
/ R is obtained, and this ratio crn / R is set as the awakening level.
When this ratio crn / R is 1 or more, the awakening level is a dangerous level.

In the above description, the reference value D of the average lateral deviation amount, which means a dangerous traveling state, is set to 1 m, which is an amount corresponding to the protrusion to the adjacent lane, but this value is set according to the traveling environment. It is also possible to change.

In step S411, a regression line having an average intercept obtained from the characteristics of a plurality of drivers stored in advance and an average intercept is used, and step S4 is used.
The same process as 10 is performed.

In step S412, the awakening level is output. However, when t ≦ td, since the normal value is being measured, the awakening level is not output. When t> td, the value of the awakening level estimated in step S410 or step S411 is set as the awakening state of the driver, and the driver is notified by a numerical display, a blinking speed of a lamp, or the like. It is also possible to use the output awakening level as an input of the driver's awakening maintenance device that stimulates the driver with sound, vibration, scent, or the like.

In step S413, the counter n is incremented by 1, and the process proceeds to the awakening level estimation process in the next unit measurement time.

In step S408, when the regression line is obtained, the measured average eye-closure time cn does not change from the normal value co even after a lapse of a certain time or more from the measurement time, and the awakening degree is decreased. If not, it is possible to thin out the average eye closing time cn and the average lateral deviation amount dn obtained for each unit measurement time T so that they are not used in the calculation for obtaining the regression line. By this method, the regression line can be accurately obtained when a change in the arousal level appears over time.

Since blink detection by the image processing described in the above embodiment does not have to be worn by the driver, it is desirable in that it does not burden the driver. However, the electrodes are attached to the eyelids of the driver. A method of measuring the electro-oculogram and detecting the opening and closing of the eyelids from the electro-oculogram waveform can also be used.

In the above-described embodiment, an example in which the eye-closing time is used as the blink parameter in the blink parameter extraction unit has been described, but the present invention is not limited to this, and the amplitude when the eyelid moves and the blinking time. It is also possible to estimate the arousal level using the occurrence interval time and the like.

Further, in the vehicle state detecting section of the above-mentioned embodiment, the lateral displacement amount Y which is the state amount of the vehicle is obtained on the basis of the vehicle speed and the steering amount of the steering wheel. The output value may be obtained by second-order integration.

[0081]

As described above, according to the inventions of claims 1 and 2, since the characteristics of the individual are grasped from the start of the measurement, without learning the characteristics of the individual in advance,
Since the state corresponding to the individual difference in that situation is obtained based on the data obtained as it is from the driving work of the automobile, it is possible to obtain an effect that an appropriate arousal level can be detected.

[Brief description of drawings]

FIG. 1 is a block diagram showing a basic configuration of this embodiment.

FIG. 2 is a block diagram showing details of a blink detection section in FIG.

FIG. 3 is a block diagram showing details of a blink parameter extraction unit in FIG.

FIG. 4 is a block diagram showing details of a vehicle state extraction unit in FIG.

FIG. 5 is a flowchart showing a processing routine of a blink detection section.

6A is a diagram showing an original image of an eye image, FIG. 6B is a diagram showing a one-dimensional original image obtained from the original image, and FIG. 6C is a diagram showing a one-dimensional edge image obtained from the one-dimensional original image. .

FIG. 7 is a diagram showing a time transition of an average eye closing time and an average lateral displacement amount.

FIG. 8 is a diagram showing a correlation between an average eye closing time and an average lateral displacement amount and a linear regression line.

FIG. 9 is a flowchart showing a processing routine of an awakening degree detection unit.

FIG. 10 (a) is a diagram showing the correlation between the ratio of the average eye closing time to the normal value and the average lateral deviation amount after a few minutes have passed since the start of driving (state of high arousal), and FIG. 10 (b) is awakening. Diagram showing the correlation between the ratio of the average eye closure time to the normal value and the average lateral deviation in a state where the degree of declining tends to decrease, and (c) is the ratio of the average eye closure time to the normal value when the arousal level is decreased. It is a diagram showing a correlation with the average lateral displacement amount.

FIG. 11 is a diagram showing a linear regression line obtained from the correlation between the average eye closure time and the average lateral deviation.

12A is a face image stored in a frame memory; FIG. 12B is a binary image of a face; FIG. 12C is a gray image of an eye contour; It is a figure which each shows.

FIG. 13 is a diagram for explaining a method of extracting a corrected steering amount.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Toshihito Ishihara Aichi-gun Aichi-gun Nagakute-machi large character Nagatogi 1 41 of Yokomichi Toyota Central Research Institute Co., Ltd. 1 at 41 Central Road, Toyota Central Research Institute (72) Inventor Shin Yamamoto, Nagakute Town, Aichi District, Aichi Prefecture 1 at 41 Central Road, Toyota Central Research Center

Claims (2)

[Claims] 1. Extraction means for extracting a blinking parameter from the movement of a driver's eyelid, lateral deviation amount detecting means for detecting a lateral deviation amount of a vehicle from a target travel locus, and blink parameter and lateral deviation amount. A driver that calculates the correlation between the blinking parameter and the lateral deviation amount of the vehicle based on the following, and an awakening degree detection unit that detects the driver's awakening degree based on the calculated correlation. Wakefulness detection device. 2. An extracting means for extracting a blinking parameter from a driver's eyelid movement, a lateral deviation amount detecting means for detecting a lateral deviation amount of the vehicle from a target traveling locus, and a blinking parameter and a lateral deviation of the vehicle. Regression formula computing means for computing a regression formula representing the relationship between the blinking parameter and the lateral deviation amount of the vehicle based on the unit quantity, and a significance test means for testing whether or not the regression formula is significant, and A wakefulness detection device for the driver, comprising: a wakefulness detection means for detecting the wakefulness of the driver based on the tested regression equation.