JP4193337B2 - Arousal level drop determination device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a wakefulness decrease determination device, and more particularly to a wakefulness decrease determination device that detects a driver's blink by processing an image of a driver's face and determines the driver's wakefulness by the length of the blink time. is there.
[0002]
[Prior art]
Conventionally, various awakening level fall determination apparatuses have been proposed.
For example, recently, a system that detects blinks and pupil diameter and estimates and alerts the driver's wakefulness and fatigue is proposed in Japanese Patent Application Laid-Open No. 11-316884, and pupil reflection is used to detect blinks. Japanese Patent Publication No. 11-105578 proposes a blink detection system that estimates the driver's arousal level.
[0003]
Furthermore, Japanese Patent Application Laid-Open No. 11-339200 proposes a system for determining the arousal level based on the blinking state and further determining the final position based on the vehicle position and driving operation with respect to the lane in order to further improve the accuracy.
[0004]
[Problems to be solved by the invention]
In general, passenger cars and commercial vehicles have considerably different interior layouts as shown in Fig. 10 (1) and (2), such as the left and right mirror angles, the meter look-down angle, and the driver's viewpoint from the road surface. .
[0005]
In other words, in the case of the passenger car shown in Fig. 1 (1), the look-down angle (the depression angle) to the meters 30 is about -25 °, but in the case of the commercial vehicle shown in Fig. 2 (2), the meters The look-down angle with respect to 30 expands to about -35 °.
Therefore, the amount of movement of the driver's line of sight is larger than that of a passenger car in a commercial vehicle, and the frequency of distributing the line of sight is different for professional drivers.
[0006]
In such a commercial vehicle, in order to perform blink detection by conventional face image processing, a CCD camera is installed so as to capture the upper body of the driver as shown in FIG. Even if the line-of-sight movement amount is large to some extent, it is necessary to install the driver so that the line of sight of the driver is captured in the image.
[0007]
However, when performing the image processing shown in the figure, the upper body of the driver is copied, so the resolution around the eyes of the driver, which is a part of it, is relatively low, as shown in FIG. Whether the eye is in a closed eye state as shown in (2) or looking down on the meters as shown in (3) in the forward gaze state Becomes difficult to distinguish.
[0008]
Furthermore, in the case of commercial vehicles, the gaze angle in the forward gaze state shown in (1) and the gaze angle in the meter gaze state shown in (3) are greatly different from each other. It has been confirmed that the latter state of looking down on the meters is often mistaken as the blinking state of (2) in the figure.
[0009]
Therefore, the present invention detects a blink of a driver and determines a decrease in arousal level by detecting a blink with high accuracy without being affected by the visual recognition of a meter, etc. The purpose is to judge.
[0010]
The awareness decrease determination device according to the present invention for achieving the purpose of, Rutotomoni determined frequency distribution data of the blink time data at every predetermined time, the frequency distribution data in said predetermined time previously determined, from the predetermined time The frequency distribution data at the latest unit time or data output interval obtained every short unit time or data output interval is added, and the frequency distribution data at the oldest unit time or data output interval in the previous frequency distribution data is removed. Thus, the frequency distribution data for this time is obtained, and when a peak occurs within a certain blinking time in the obtained frequency distribution data , the data is excluded from the determination and based on the frequency distribution data. It is characterized by determining a decrease in arousal level .
[0011]
Now, if the frequency distribution of the blinking time per fixed time (ΔAn: for example, 1 minute) is taken, it is as shown in FIG. That is, the frequency of short blinking times is the highest, and the frequency gradually decreases gradually as the blinking time increases.
In this case, the characteristic (1) indicated by the solid line indicates the frequency distribution at the normal time (when awakening), and the characteristic (2) indicated by the broken line indicates the frequency distribution when the arousal level is reduced. The characteristic {circle around (2)} when the degree of arousal is lowered exhibits a characteristic curve almost the same as the characteristic {circle around (1)} at normal times, but it can be seen that the overall blinking time becomes longer and the average value becomes longer.
[0012]
Therefore, for example, if the difference between the two characteristics 1) and 2) is obtained by a conventionally known calculation method, it can be determined whether it is normal or when the arousal level is lowered.
On the other hand, at the time of visual recognition of the meter as shown in FIG. 12 (3), a peak P1 appears at a specific blink time as shown by the characteristic C in FIG. 1 (2). The blinking time when the meter is visually recognized is about 0.5 to 0.7 seconds, which is longer than the normal blinking time.
[0013]
On the other hand, when the meter is visible even when the arousal level is reduced, peak P2 occurs as shown in (1) of the figure. Therefore, it is necessary to distinguish both peaks P1 and P2 with a certain blink time Tτ as a boundary. There is.
Therefore, if peak P1 is detected within a certain blink time Tτ as shown in Fig. 2 (2), if the blink data at a certain time ΔAn is not used for the determination for arousal level reduction, It will not be erroneously detected as blinking when the user visually recognizes
In this case, the latest one of the blink time data in the unit time or data output interval shorter than the certain time is added to the frequency distribution data of the blink time data in the fixed time obtained last time and the oldest unit time or data output interval By obtaining the frequency distribution for this time by removing the frequency distribution, it is possible to sufficiently determine the tendency to decrease the arousal level in a short time.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 2 shows a configuration example of a wakefulness reduction determination apparatus according to the present invention, in which a camera 1 such as a CCD is connected in cascade to an image processing unit 2, a blinking time detection unit 3, and a false detection determination unit 4. Yes.
[0015]
FIG. 3 is a flowchart showing an operation example of each part of the wakefulness reduction determination apparatus according to the present invention shown in FIG.
The operation of the wakefulness reduction determination apparatus according to the present invention will be described below along the flowchart shown in FIG.
[0016]
Operation of image processor 2 :
First, a face image as shown in FIG. 4 (1) is input from the camera 1 (step S1 in FIG. 3), and preprocessing such as filtering is performed on the entire screen (S2). Next, a binarization process is performed on the entire image (S3), and preparations are made to search for a driver's nostril.
[0017]
Then, using a method such as pattern matching, the position of the nostril of the driver is specified as shown in FIG. 4 (2) (S4). Next, as shown in FIGS. 3 (3) and 5 (1), the position of the eye is further specified from the position of the nostril, and the peripheral areas A and B of the eye are set (S5).
[0018]
Then, it is determined whether or not it can be estimated as a peripheral region of the eye by using a method such as pattern matching (S6), and if it cannot be estimated, the process returns to step S1, but when the peripheral region of the eye can be estimated Then, the face image is input again (S7), the eye peripheral area as shown in FIG. 2B is tracked (S8), and it is further determined whether or not tracking is possible (S9).
[0019]
As a result, when tracking is not possible, the process returns to step S1, but when tracking is possible, preprocessing is performed on the peripheral area of the eye (S10), and edge extraction processing is performed on the peripheral area of the eye (same as above). S11).
In this way, by performing edge extraction processing on the peripheral region of the eye position estimated from the position of the nostril, the boundary between the upper eyelid and the lower eyelid is clarified and the boundary is extracted. Based on the positional relationship between the upper eyelid boundary and the lower eyelid boundary, it is determined whether the eye is open or closed as shown in FIG.
[0020]
Operation of blink time detector 3 :
After performing the edge extraction process as described above to detect blinking, the blinking time is measured for a certain time (ΔAn) (S12). Then, the blink time data is accumulated (S13), and the frequency distribution of the blink time per fixed time (ΔAn) is obtained as shown in FIG. 1 (S14).
[0021]
Operation of false detection determination unit 4 :
First, peak detection is performed by a conventionally known method (S15), and it is determined whether or not peaks P1 and P2 as shown in FIGS. 1 (1) and (2) exist (S16). . When no peak is detected, the data can be used for arousal level determination as shown in FIG. 1A. In this case, either the characteristic (1) or (2) is output and output in step S7. Return to (S17).
[0022]
On the other hand, since it is necessary to distinguish whether the peak is in the false detection range, that is, whether it is peak P1 or P2 (S18), if the peak position is below a certain value Tτ, Assuming that the peak P1 shown is detected, the blink data is invalidated (S19), and the process returns to Step S7.If the peak is equal to or greater than Tτ, the peak P2 shown in FIG. The blink data is used to determine the arousal level (S20).
[0023]
In the above embodiment, as shown in FIG. 6, the frequency distribution is obtained by taking the blink data of the past fixed time ΔAn, but if this fixed time ΔAn is short, an accurate frequency distribution cannot be obtained. On the contrary, if it is long, the determination time of the arousal level becomes long, and it is possible that the generation of an alarm accompanying the determination result is delayed.
[0024]
Therefore, as shown in FIG. 6, the frequency distribution of the blinking time is added to the frequency distribution in the above ΔAn at a unit time shorter than the certain time ΔAn or the data output interval ΔT, and the old unit time in the above certain time ΔAn or By excluding the frequency distribution data of the data output interval ΔT, the presence of the peak may be determined by obtaining the past frequency distribution of ΔAn added with the new unit time or the data output interval ΔT.
[0025]
Since the peak in this case gradually increases or decreases, it is only necessary to detect the occurrence of the peak by setting an empirical threshold.
In this way, it is possible to obtain blink data for each data output interval ΔT shorter than the predetermined time ΔAn for determining the arousal level.
[0026]
In the above embodiment, the detection of the blinking time during daytime driving and the frequency distribution thereof are obtained. In the case of nighttime, for example, as shown in FIG. 7, the infrared camera 1 and the infrared for illumination are used. The LED 10 may be used to take an image in which the pupil of the driver's eyeball 20 is illuminated.
[0027]
In other words, this is a method that uses the same principle as when the subject's eye may glow red when taking a picture using a flash, and the illumination is as close as possible to the optical axis of the camera 1, or If the driver's face (see Fig. 9 (1)) is photographed using a half mirror, the images shown in Fig. 8 (1) and Fig. 9 (2) can be obtained from the eye image shown in Fig. It is possible to detect blinks using a face image in which the pupil is shining, called pupil reflection.
[0028]
【The invention's effect】
As described above, according to the arousal level decrease determination device according to the present invention, the frequency distribution of blink time data per fixed time is obtained, and a peak occurs within the fixed blink time in the frequency distribution. Since the blink time data is excluded from the determination of arousal level reduction, especially when viewing meters like a commercial vehicle, misjudgment associated with blink time detection when presenting a state similar to the blink state Can be eliminated.
[Brief description of the drawings]
FIG. 1 is a characteristic graph showing a frequency distribution of blinking time in order to explain the principle of a wakefulness reduction determination apparatus according to the present invention.
FIG. 2 is a block diagram showing a configuration example of a wakefulness reduction determination apparatus according to the present invention.
FIG. 3 is a flowchart for explaining the operation of the wakefulness reduction determination apparatus according to the present invention.
FIG. 4 is a diagram illustrating a process of estimating an eye position from a daytime image.
FIG. 5 is a diagram showing a detailed processing method in an eye image processing region.
FIG. 6 is a diagram for explaining a relationship between a data processing interval and a data processing interval.
FIG. 7 is a diagram illustrating an illumination state of an eyeball by an infrared LED during night driving.
FIG. 8 is a diagram showing bright pupil reflection.
FIG. 9 is a diagram for explaining a driver's pupil position accompanying bright pupil reflection;
FIG. 10 is a schematic side view showing a state in which the look-down angle of the meters is different depending on the type of vehicle.
FIG. 11 is a diagram showing an example of a driver's face image by a CCD camera.
FIG. 12 is a diagram for explaining line-of-sight angles at the time of forward gaze and meter viewing;
[Explanation of symbols]
1 Camera
2 Image processing section
3 Blink time detector
4 False detection judgment part
10 Infrared LED
20 Eyeball
In the meter diagrams, the same symbols indicate the same or corresponding parts.

Claims (1)

In a wakefulness decrease determination device that detects blinking of a driver and determines a decrease in wakefulness of the driver,
Rutotomoni determined frequency distribution data of the blink time data at every predetermined time, the frequency distribution data in said predetermined time previously determined, the latest calculated a predetermined time shorter than the unit time or per data output interval of the unit time or data output The frequency distribution data at the interval is added, and the frequency distribution data for the current time is obtained by removing the frequency distribution data at the oldest unit time or data output interval from the previous frequency distribution data . When a peak occurs within a certain blinking time, a wakefulness level reduction determination apparatus characterized by determining a reduction in wakefulness level based on the frequency distribution data after excluding the data from the determination.

Images