JPH0796768A - Forward watching degree detector - Google Patents

Abstract

PURPOSE:To detect a drop in a forward watching degree stably with certainty by measuring a driver's eying line at all times, judging whether the forward watching degree of a driver is lowered or not on the basis of this measuring result, and when this watching degree is lowered, emitting an alarm to the driver at once. CONSTITUTION:A car driver 5 usually looks at a watching area 30 of a windshield 1. In this case, an eying line L of the driver 5 is always measured by an eying line measuring instrument 2. In addition, on the basis of a direction of the eying line L measured, whether a watching degree of the driver 5 to the front is lowered or not is judged by a watching degree judging device 3. When this watching degree of the driver 5 to the front is lowered, an alarm consisting of a voice and vibration or the like for urging a warning is emitted to the driver 5 with an alarm generator 4. With this, there is no effect of incidence or the like of the direct sunlight from the horizontal direction as comapred with such being subjected to the measurement of the whole face of the driver 5, whereby a direction of the eying line L is easily measured and also any drop in the watching degree is stably judged with certainty.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicular forward gaze degree detecting device for detecting a degree of gaze forward of a driver of a vehicle, such as a decrease in awakening degree or a look aside.

[0002]

2. Description of the Related Art When the degree of gazing ahead decreases or the degree of awakening decreases during driving, the inter-vehicle distance from the preceding vehicle becomes short and the vehicle may approach the preceding vehicle abnormally. As a conventional device of this kind, for example, as disclosed in Japanese Patent Laid-Open No. 60-178596, the driver's arousal level is lowered by detecting the periodical fluctuation of the body position from the front of the driver. Is determined.

[0003]

However, in such a conventional device, two measuring cameras are used to detect the periodical fluctuation of the upper body position of the driver, and therefore the device is used. There is a problem that the configuration is complicated and it takes time to process. In addition, since the features to be extracted are facial regions, they are exposed to unstable conditions, such as the incidence of direct light from the lateral direction and other cases where the light conditions are poor, and as a result, there is an error in feature extraction. There was a problem of high possibility. Therefore, it is difficult for the conventional device to perform quick and stable detection. The present invention has been made in view of such conventional problems, and an object of the present invention is to provide a front carelessness detection device for a vehicle capable of stably and reliably detecting a decrease in front gaze degree. And

[0004]

Therefore, according to the present invention as set forth in claim 1, the image input means provided toward the eyeball region of the driver and the image data of the eyeball portion obtained by the image input means are provided. Based on the line-of-sight detection means for detecting the direction of the driver's line-of-sight, based on the area determination means for determining whether or not the line-of-sight direction is within a predetermined gaze area, and measuring the time when the line-of-sight direction is outside the gaze area. , And time measuring means for comparing with a predetermined reference time.

According to a third aspect of the present invention, instead of the area discriminating means and the time measuring means, pattern detecting means for detecting a movement pattern in the line-of-sight direction and pattern comparison means for comparing the movement pattern with a predetermined reference pattern are provided. To have. Further, the inventions according to claim 2 and claim 4, respectively, further acquire the traveling speed, the elapsed time from the start of traveling, the steering angle of the steering wheel, the obstacle information, and the environmental information including at least one of the road shapes. An environment detection unit is provided, and the predetermined gaze area, reference time, or reference pattern is set based on the environment information.

[0006]

According to the present invention, the line-of-sight direction detecting means processes the image data of the eyeball portion to obtain the line-of-sight direction of the driver, and the line-of-sight direction is set in advance during normal driving. The area determination means determines whether or not the target area is actually facing. When the non-looking time continues for a predetermined reference time or longer, it is determined that the forward gaze degree is low.

According to the third aspect of the invention, when the movement pattern in the line-of-sight direction is in an abnormal state different from the reference pattern during normal driving, for example, after a temporary stop in the lower visual field, a jumping direction change occurs, It is determined that the arousal level has decreased. As a result, it is possible to stably and reliably detect the decrease in the degree of forward gaze of the driver.

When the gaze area, the reference time, or the reference pattern is set on the basis of the environmental information by providing the environment detecting means as in the second and fourth aspects, it is detected by the traveling environment. Since the level is controlled, it is possible to give a quicker warning or the like according to the urgency of the environment.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing the overall configuration of a forward gaze degree warning device as a first embodiment of the present invention. This device includes a gaze measuring device 2, a gaze degree determining device 3, and an alarm generating device 4. The line-of-sight measurement device 2 constantly measures the line-of-sight direction L of the driver 5. The gaze degree determination device 3 determines whether or not the front gaze degree is lowered based on the gaze direction obtained by the gaze measurement device 2, and sends a command signal to the alarm generation device 4. The alarm generation device 4 issues an alarm to the driver 5 with a voice, a vibration, or the like for attention.

FIG. 2 shows the configuration of the visual axis measuring device 2. A first light source 22 such as an LED that emits invisible near-infrared light is installed on the front surface of the camera 21, which is an imaging unit, so that the optical axis of the camera 21 and the irradiation direction coincide with each other. The second light source 23 such as an LED that emits
Is installed at a predetermined position where the relative relationship with the first light source 22 outside the optical axis is known.

Further, an A / D converter 24 for converting the image data of the eyeball 31 captured by the camera 21 into digital data, and a corneal reflection image and a retinal reflection which are features showing the line-of-sight direction from the input image data. A feature extraction unit 25 that extracts an image, a line-of-sight direction calculation unit 26 that calculates the line-of-sight direction from the existing position of each extracted feature, an illumination light emission control unit 27 that controls the light emission of the first light source 22 and the second light source 23, and a device. An overall control unit 28 for controlling the overall operation and a memory 32 for holding A / D converted image data are provided. In addition, 30 shows the gaze area mentioned later.

The principle of the line-of-sight measuring device 2 will be described with reference to FIG. FIG. 3 shows how the eyeball 31 is illuminated from the outside. When the eyeball 31 is illuminated from the outside, a specular reflection image at the specular reflection position P of the corneal sphere surface 51 of the cornea sphere 50 whose center is O is captured by the camera 21 and observed at the point R as a bright spot. This is called a corneal reflection image. In particular, when the camera 21 and the first light source 22 are coaxially arranged, the three points of the corneal sphere center O, the specular reflection position P, and the position R of the corneal reflection image are on the same straight line.

That is, the camera 2 is attached to the front of the camera 21.
When the eyeball 31 is observed by the coaxial sensor in which the first light source 22 is installed so that the irradiation direction coincides with the optical axis of No. 1, the corneal reflection image position R on the camera 21 and the position of the first light source 22 (camera). The corneal center O exists on the line connecting the focal points).

When the eyeball 31 is observed by the coaxial sensor, the light flux passing through the pupil is reflected on the retina and returns in the incident direction, exits from the pupil, reaches the camera 21, and the pupil area is observed brightly. . This is known as a red-eye phenomenon in which the pupil glows red when a human face is stroboscopically photographed by a camera, and the image thus generated is called a retina reflection image. The retina reflection image is generally observed in an elliptical shape, and the pupil center Q exists on the straight line connecting the center of gravity of the retina reflection image on the camera 21 and the camera focus.

The line-of-sight measuring device 2 extracts the three-dimensional position of the center O of the cornea and the center Q of the pupil on the basis of the principle described above, and obtains the direction L of the line of sight passing through O and Q. Next, how to obtain the center O of the cornea and the center Q of the pupil will be described. First, referring to the corneal reflection image position extraction processing diagram of FIG. 6 and the retinal reflection image position extraction processing diagram of FIG. 7, measurement of the barycentric position of the extracted corneal reflection image region and the extracted barycentric position of the retinal reflection image region is performed. 4 will be described with reference to the flowcharts of FIGS.

Measurement is started by a measurement start signal from the overall control unit 28. In step 101, the illumination light emission control unit 27 sends a trigger signal to the first light source 22, lights the first light source 22, and proceeds to step 102. The second light source 23 is off. In step 102, the camera 2
Capture the image from 1. The image is A / D converted by the A / D converter 24 in step 103, and the generated digital image data I1 (x, y) is converted into step 104.
Is stored and held in the memory 32.

In step 105, the illumination light emission control unit 27
Sends a trigger signal to the second light source 23 to turn on the second light source 23 and turn off the first light source 22. Step 106
Then, the image is captured from the camera 21, and step 107
Then, the image is A / D converted by the A / D converter 24 to generate digital image data I2 (x, y), and the image data is stored in the memory 32 in step 108. When the image input is completed, in step 109, the second light source 23
Turn off. The image input of two frames by the above different illuminations is executed in an extremely short time, and becomes a pair of images input at substantially the same time.

Next, in steps 110 to 111, the corneal reflection image position is processed based on the image data I1 (x, y) and I2 (x, y) stored in the memory 32 as shown in FIG. Go and ask.

That is, in step 110, the feature extraction unit 25 extracts the corneal reflection image area 62 necessary for the line-of-sight measurement from the image data I1 (x, y). Since the corneal reflection image is observed as a very bright bright spot, by binarizing the image data I1 (x, y) shown in (a) of FIG. 6 with a certain threshold Th1, (b) Can be easily extracted. Image I2 (x,
The same applies to y). That is, the regions of I1 (x, y) ≧ Th1 I2 (x, y) ≧ Th1 are respectively determined as the corneal reflection image regions. In step 111, the center of gravity position of the extracted corneal reflection image region 62 is calculated, and the corneal reflection image positions B (x1, y1), C
(X2, y2) is calculated.

Then, in steps 112 to 118, the retinal reflection image position is obtained by performing the processing shown in FIG. 7 based on the image data I1 (x, y). First, at step 112, the retinal reflection image candidate region is extracted by binarization. As described above, the bright retinal reflection image is observed by using the coaxial first light source 22 and the camera 21, but the brightness of this image is darker than that of the corneal reflection image. Therefore, the image data I1 shown in FIG.
(X, y) is set to a certain threshold Th2 (where Th2 <T
By binarizing in h1), the retina reflection image candidate region can be extracted. That is, the region of I1 (x, y) ≧ Th2 is set as the retinal reflection image candidate.

Since noises other than the pupil are generally mixed in the candidate region thus extracted, a process of further extracting only the pupil is performed in the next step. That is,
In step 113, the retinal reflection image candidate area obtained in step 112 is subjected to labeling processing by setting a threshold value for the density, and the processed image of FIG. 7B is obtained. Find the area of. In step 115, as shown in FIG. 7C, a region having a predetermined area (a−threshold, a> 0) from (S−a) to (S + a) with respect to the expected pupil area S. 63A, 63B
Only those are selected and the process proceeds to step 116.

In step 116, it is checked whether or not the extracted predetermined area regions are uniquely determined. If yes, the process proceeds to step 118. When a plurality of areas are extracted as shown in FIG. 7C, the circularity of each area is obtained in step 117, and the area 63A (= 63) closest to a circle is extracted as shown in FIG. 7D. Then, the process proceeds to step 118. In step 118, the center of gravity of the extracted region 63 of the retina reflection image is calculated, and the retina reflection image position A (xp, y
p) is calculated. The pupil center is located on this retina reflection image position A. The result of extraction as described above is shown in FIG.
FIG. 8A shows a corneal reflection image position B (x1, y1) and a retina reflection image position A extracted from the image data I1 (x, y).
(Xp, yp) is shown, and (b) is image data I2 (x,
The corneal reflection image position C (x2, y2) extracted from y) is shown.

Next, FIG. 9 shows how the line-of-sight direction calculation unit 26 calculates the line-of-sight direction. Here, the focal length of the camera 21 is f, and the focal position is the origin F1 (0, 0, 0), and the x-axis (horizontal axis) direction of the CCD surface of the camera 21 is X.
Consider the world coordinates F1 -XYZ with the Y axis as the y-axis (vertical axis) direction and the Z axis as the optical axis direction. As described above, if the illumination light source and the optical axis of the camera are aligned, the light source position, the corneal reflection image, and the center of the corneal sphere are on a straight line. Therefore, the focus F1 (0,0,0) of the camera 21 and the camera 21
It is considered that the straight line connecting the corneal reflection image position B on the CCD surface passes through the corneal sphere center O. That is, the center O of the corneal sphere is on the straight line F1 -B. Similarly, the pupil center Q is on a straight line F1 -A connecting the focal point F1 (0,0,0) of the camera 21 and the retina reflection image position A.

The second light source 23 is assumed to be at a predetermined position F2 (a, b, c) outside the optical axis of the camera 21. The illumination light of the second light source 23 is specularly reflected on the surface of the corneal sphere, and the camera 21
Observed. Therefore, the specular reflection point P is
Is on a straight line F1-C connecting the focal point F1 (0,0,0) of the eye and the corneal reflection image position C. Here, the radius of the human corneal sphere 50 is substantially constant regardless of the human being, and is about 7.8 mm. Therefore, the regular reflection point P is temporarily placed on the straight line F1 -C, and the radius 7.
If a spherical surface of 8 mm is drawn, a candidate point O at the center of the corneal sphere is uniquely determined as the intersection of the spherical surface and the straight line F1-B. Here, the specular reflection condition at the point P

[Equation 1] If it is confirmed that (where T is a point on the extension of the half line OP) is satisfied, the corneal sphere center O is determined.

Further, since the distance between the center O of the cornea and the center Q of the pupil (front surface of the lens) is 4.2 mm, the intersection point between the spherical surface centering on the determined point O and having a radius of 4.2 mm and the straight line F1 -A can be obtained. For example, the pupil center Q is determined. In this way, since the corneal sphere center O and the pupil center Q are determined, the line-of-sight vector (line-of-sight direction) OQ is determined. The optical constants of the eyeballs used here are based on Gullstrand's model eyes.

The gaze degree determining device 3 determines whether or not the line-of-sight direction L is within a predetermined gaze area of the windshield 1 viewed by the driver 5 during normal driving. This determination is made by registering the three-dimensional coordinate data of the windshield 1 in advance and checking whether the extension line in the line-of-sight direction L intersects the gaze area 30. And driver 5
When the time period in which the line-of-sight direction L does not intersect with the predetermined gazing area of the windshield 1 continues for a predetermined time, for example, 1 second or more, it is determined that the forward gaze degree is lowered,
A command signal is sent to the alarm generator 4.

This embodiment is configured as described above, illuminates the driver's eyeball with the first and second light sources that emit invisible light, and uses the image data captured by a camera coaxial with the first light source to form the cornea. The reflection image and the retina reflection image were obtained, and the driver's line-of-sight direction was obtained based on these. Then, it is determined whether or not the line-of-sight direction is actually set to the gaze area to which the driver's line-of-sight direction is set during preset normal driving, and when the time when the line of sight is not facing continues for a predetermined reference time or longer, the forward gaze degree is determined. Since an alarm is issued as if the target area has decreased, compared to the case where the entire face area is subject to feature extraction, it is not affected by the incidence of direct light from the lateral direction or other light ray conditions, and it is stable and reliably forward. It is possible to detect a state of diminished gaze. Further, there is an effect that it is not necessary to use two cameras as in the conventional case, the device configuration is simple, the detection can be performed quickly, and the cost is low.

In addition to the above, as the determination mode in the gaze degree determination device 3, for example, the angle when the end of the windshield 1 is viewed from the center of the driver's eye range is recorded in advance, and this is also the direction of the line of sight. It is also possible to compare it with the angle of. Furthermore, the driver 5 may be allowed to adjust the reference time for determining that the degree of forward gaze is reduced.

10 to 12 show a second embodiment of the present invention. In this embodiment, the criterion for lowering the degree of forward gaze is controlled according to the traveling environment of the vehicle. First, the configuration will be described with reference to FIG. A line-of-sight measurement unit 201 that measures the line-of-sight direction of the driver is provided by the same configuration as the line-of-sight measurement device of the first embodiment, and an environment measurement unit 203 that measures various parameters as environment information of vehicle traveling and time Time measuring unit 20 for measuring
4 is installed. And these measuring units 201 and 20
An information processing unit 202 that processes signals from Nos. 3 and 204 and detects a driver's forward gaze reduction or awakening reduction, and an alarm output unit 205 that outputs an alarm based on the signal from this information processing unit 202. ing.

The environment measuring unit 203 displays the traveling speed v of the vehicle.
Vehicle speed sensor 206 for measuring the steering angle θ of the steering wheel
Steering angle sensor 207, obstacle sensor 208 using radar or camera, vehicle position detection device 20
9, a vehicle-mounted database 210, and a vehicle-to-vehicle communication device 211 for exchanging information with the vehicle exterior database 212 are provided. Further, the warning output unit 205 includes a first warning output unit 205a that outputs a front gaze reduction warning and a second warning output unit 205b that outputs a wakefulness reduction warning.

The control operation of this embodiment will be described with reference to the flowchart of FIG. First, in step 301,
The line-of-sight measurement unit 201 measures the line-of-sight direction of the driver in the same flow as in the first embodiment described above. At this time, the measured line-of-sight direction is represented by the intersection (Xi, Yi) of the virtual plane 352 and the line-of-sight 351 as shown in FIG. Further, the variances SX, SY of Xi, Yi and Sr = (SX ² + SY ² ) ^1/2 in a predetermined number of samplings are calculated.

Next, in step 302, the environment measuring unit 20
In 3, the running environment of the vehicle is required. here,
The traveling speed v is measured by the vehicle speed sensor 206, the steering angle θs is measured by the steering angle sensor 207, and the distance U to the obstacle in front of the vehicle by the obstacle sensor 208, the vertical and horizontal directions of the obstacle (ψ, φ) Also, obstacle information such as relative speed to the obstacle is measured. Further, the vehicle's vehicle position detection device 210 and the vehicle-mounted database 211 determine the curvature RN and the gradient change rate NT of the road ahead of the vehicle. At this time, it is also possible to obtain more detailed road information such as curvature and gradient change rate from the vehicle exterior database 212 referred to via the road-vehicle communication device 211. The azimuth (φ, φ) of the obstacle is converted into coordinates (Xf, Yf) on the virtual plane 352 of FIG.

After that, in the information processing unit 202, in step 303, the value of the elapsed time LT from the start of traveling of the vehicle is obtained from the time measuring unit 204. And step 30
In 4, the driver checks the foreground region, that is, checks whether or not the driver is looking ahead. Here, when the line-of-sight direction (Xi, Yi) measured by the line-of-sight measurement unit 201 remains outside the gaze area 350 on the virtual plane 352 of FIG. 12 for a predetermined reference time T1 or more, the forward gaze degree decreases. If it is determined that there is one, the process proceeds to step 306. Otherwise, go to step 305.

As shown in FIG. 12, the above-mentioned gaze area 35
0 is vertical 2Va and horizontal 2 with the point of (Xc, Yc) as the center.
It is defined by the rectangle of Ha. The parameters that determine these gaze areas 350 will be described below. The Xc at the center of the gaze area is calculated using the equation (1). Xc = A1.v / RN + A2..theta.s (1) (where A1 and A2 are constants) The area where a normal driver gazes moves left and right when traveling on a curved road. This amount of travel depends on the curvature of the road and how far the driver is looking at the road,
The latter can be approximated as being proportional to the speed of the vehicle. The first term of the equation (1) expresses this, and the second term corresponds to steering at the time of changing lanes within the road width.

Yc at the center of the gazing area is calculated by using the equation (2). Yc = Y0 + A3.v + A4.NT (2) (where Y0, A3, and A4 are constants) The normal driver's gazing area is approximately horizontal when driving at high speed, but the center is more stable when driving at low speed. It goes low (ie close), and the second term represents this.
Further, the gaze area moves upward when approaching an uphill and downward when descending due to the gradient of the road. Therefore, as in the third term, it is considered that the vertical movement of the gaze area due to the gradient is proportional to the rate of change NT of the gradient.

The width Ha of the gazing area is calculated using the equation (3). Ha = H0 + A5 • | v / RN | + A6 • | θs | + A7 • Sr (3) (however, H0, A5, and A6 are constants) When driving on a curved road, make a right or left turn from the road edge immediately before your vehicle. Since it is necessary to visually recognize the road ahead, the right and left width of the area where a normal driver gazes becomes wide. The same applies when changing lanes within the road width, and the second and third terms correspond to this. Further, it is known that the variance Sr in the line-of-sight direction has a large value because vehicles and various targets are distributed on the left, right, up, and down on a congested road or a complicated road. In response to this, the area width Ha is adjusted by the fourth term.

The gazing area height Va is calculated by the equation (4) in consideration of the influence of the congested road. Va = V0 + A8.Sr (4) (where V0 and A8 are constants) Then, the inattentive judgment reference time T1 is calculated by the equation (5). T1 = T10 -B1 · v -B2 · {(Xi-Xc) 2 + (Yi-Yc) 2} 1/2 -B3 · Sr (5) ( where, T10, B1, B2, B3 is a constant)

That is, since there is a high possibility that a short look-ahead operation will fall into a serious danger during high-speed traveling, the reference time T1 is shortened by the second term. Further, as the line-of-sight direction deviates greatly from the original gaze area when looking aside, the more difficult it is to obtain information by peripheral vision, so an alarm is issued in a shorter time according to the third term. Further, it is considered that instantaneous situation changes frequently on a congested road, so the reference time T1 is shortened in the fourth term.

The information processing unit 202 uses the parameters described above to judge that the state of | Xi-Xc |> Ha or | Yi-Yc |> Va continues for T1 or more and is a look aside.

In step 305, it is checked whether the driver is gazing at the obstacle. The driver's line-of-sight direction (Xi, Yi) near the obstacle direction (Xf, Yf)
If the state where does not exist continues for a certain time T2 or more,
When it is determined that the degree of obscuration is low, the process proceeds to step 306, and if not, the process proceeds to step 307. Here, the region near the direction (Xf, Yf) of the obstacle is set by the equation (6),
The reference time T2 for determining the degree of obstruction gaze reduction is calculated by the equation (7). ^{(Xi-Xf) 2 + (} Yi-Yf) 2> C1 (6) ( where, C1 is a constant) T2 = T20 -C2 · {( Xi-Xf) 2 + (Yi-Yf) 2} 1/2 - C3.v + C4.U-C5.dv (7) (however, T20, C2, C3, C4, C5 are constants)

The second term of the equation (7) indicates that the more the direction of the obstacle and the direction of the line of sight are greatly different, the more difficult it is to obtain information by peripheral vision, and therefore the alarm is issued in a shorter time. In addition, T2 is adjusted so that a warning is issued in a short time when the speed v of the vehicle is large, the distance U to the obstacle is short, or when the relative speed dv to the obstacle is large. Item 5 corresponds to this.

In step 306, since the front gaze degree is lowered or the obstacle gaze degree is lowered, the alarm output unit 20
After outputting a sound such as a warning buzzer, a light such as a warning light, or a signal for issuing a front gaze reduction warning by vibrating a steering wheel or a seat back from the first warning output unit 205a of No. 5, the process proceeds to step 307. .

From step 307 onward, it is checked whether or not the driver's awakening level has decreased. That is, first, step 307
Then, whether or not the line-of-sight direction remains in the lower field of view for a certain time or more, that is, Yc-Yi> Vs is a certain time T3
It is checked whether the above continues. Here, Vs representing the lower field of view is calculated using Expression (8) according to the variance Sr of the line-of-sight direction that changes according to the congestion status of the road, and the time T3 is calculated using Expression (9). It Vs = V0 + D1.Sr (8) T3 = T30-D2.v-D3.LT (9) (where V0, D1, T30, D2 and D3 are constants)

In the second and third terms of the equation (9), the time T3 is
This indicates that the travel speed v is reduced when the traveling speed v of the host vehicle is high, and the awakening degree is likely to decrease when traveling for a long time. Therefore, it is adjusted according to the elapsed time LT from the start of traveling. If the line-of-sight direction remains in the lower visual field set in this way for a certain period of time T3 or more, the awakening degree is lowered and the process proceeds to step 310. If not, the process proceeds to step 308.

In step 308, the line of sight is checked. This is to detect that the movement of the line of sight becomes smaller due to the decrease in arousal level, and it is checked whether or not Sr <E1 continues for a predetermined time T4 or longer. Here, the constant time T4 is calculated by the equation (10) using the traveling speed v and the elapsed time LT from the start of traveling. T4 = T40-E2.v-E3.LT (10) (where T40, E2, and E3 are constants) If T4 or more is continued, the awakening degree is lowered, and the process proceeds to step 310. Otherwise, the process proceeds to step 309. move on.

In step 309, it is checked whether or not there is a peculiar line-of-sight movement pattern that repeats a dramatic change in the line-of-sight direction after the line-of-sight has been stopped for a while. Specifically, S
r <G1 (However, G1 is a constant) after continues to be a certain period of time T5 or more, (Xi-Xi-1) 2 + (Yi-Yi-1) 2>
It is detected that the jump of G2 occurs G3 times / minute or more, and when this detection is detected, it is determined that the awakening degree has decreased. Here, the constant time T5 is calculated by the equation (11) including the traveling speed v and the elapsed time LT from the start of traveling. T5 = T50-G4.v-G5.LT (11) (However, T50, G4, and G5 are constants)

The line-of-sight jump G2 is determined according to the deviation from the center of the gazing area or the deviation from the front obstacle, and is expressed by the equation (12). G2 = MIN [G20 + G6 · {(Xi-Xc) 2 + (Yi-Yc) 2} 1/2, G20 + G7 · {(Xi-Xf) 2 + (Yi-Yf) 2} 1/2] (12 (However, G20, G6, and G7 are constants.) The stop / jump frequency G3 is determined by the traveling speed v and the elapsed time LT from the start of traveling, and the equation (1
3) is calculated. G3 = G30-G8.v-G9.LT (13) (However, G30, G8, and G9 are constants)

From the above, it is checked whether or not the driver's awakening degree is lowered, and if there is no peculiar line-of-sight movement pattern due to the lowered awakening degree, the process returns to step 301 and moves to the next cycle. When it is determined that the awakening level is lowered due to the unique eye movement pattern, in step 310, the alarm output unit 20
5, and the second alarm output unit 205b outputs a low alertness alarm output such as sound, light, and vibration. Then, the process returns to step 301. The output mode of the alertness lowering alarm is set so as to draw more attention than the alertness lowering warning in step 306.

In step 304, the gaze area as the foreground area is defined as a rectangle, but the gaze area is not limited to this and may be an elliptical shape having equivalent major and minor axes. Further, some or all of the equations for setting the parameters may be adjustable according to the skill of the driver, the physical and mental condition, the traffic situation of the surroundings, and the like.
Further, in the present embodiment, each equation is a linear function, but by expressing the traveling speed of the host vehicle, the steering angle of the vehicle, the deviation between the reference position and the line-of-sight direction, the elapsed travel time, etc., by using a higher-order equation, More accurate determination can be performed.

[0050]

As described above, according to the present invention, the line-of-sight direction detecting means for detecting the line-of-sight direction of the driver and the time measurement for measuring the time when the line-of-sight direction is not facing the predetermined gaze area and comparing it with the predetermined reference time. Since the device has the means, the decrease in the degree of forward gaze can be easily detected by continuing the reference time or more. Since the line-of-sight direction detecting means obtains the line-of-sight direction using the image data of the eyeball portion, it is affected by the incidence of direct light from the lateral direction and other light ray states as compared with those in which the entire face area is subjected to feature extraction. The effect is that the line-of-sight direction can be easily obtained without being received, and the gaze reduction state can be detected stably and reliably.

Further, when a pattern comparing means for detecting a movement pattern in the line-of-sight direction instead of deviating from the gaze area and comparing the movement pattern with a reference pattern is provided, an awakening degree is detected from an abnormal pattern different from normal driving. The lowered state can be detected. Further, it has an environment detecting means for acquiring the traveling speed, the elapsed time from the start of traveling, the steering angle of the steering wheel, the obstacle information, and the environmental information consisting of at least one of the road shapes, and the gazing area or the reference time or the reference time. If the pattern is set based on the above environment information, the traveling environment is reflected in the detection level, the accuracy is higher, and more rapid detection is performed according to the degree of urgency. It is especially effective when used as an alarm.

[Brief description of drawings]

FIG. 1 is a diagram showing a first embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a visual line measuring device.

FIG. 3 is an explanatory diagram showing generation of a corneal reflection image.

FIG. 4 is a flowchart showing a flow of extracting a corneal reflection image position and a retinal reflection image position.

FIG. 5 is a flowchart showing a flow of extracting a corneal reflection image position and a retinal reflection image position.

FIG. 6 is an explanatory diagram showing a process of extracting a corneal reflection image position.

FIG. 7 is an explanatory diagram showing a process of extracting a retina reflection image position.

FIG. 8 is a diagram showing a corneal reflection image position and a retinal reflection image position extracted from image data.

FIG. 9 is an explanatory diagram showing a gaze direction calculation procedure.

FIG. 10 is a diagram showing a second embodiment of the present invention.

FIG. 11 is a flowchart showing a control flow in the second embodiment.

FIG. 12 is an explanatory diagram showing a gaze area.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Windshield 2 Line-of-sight measurement device 3 Gaze degree determination device 4 Alarm generator 5 Driver 21 Camera 22 First light source 23 Second light source 24 A / D converter 25 Feature extraction unit 26 Line-of-sight direction calculation unit 27 Illumination light emission control unit 28 Overall control unit 30 Gaze area 31 Eyeball 32 Memory 50 Corneal sphere 51 Corneal sphere surface 52 Corneal reflection image 201 Eye gaze measuring unit 202 Information processing unit 203 Environment measuring unit 204 Time measuring unit 205 Alarm output unit 206 Vehicle speed sensor 207 Steering angle sensor 208 Obstacle Object sensor 209 Vehicle position detection device 210 In-vehicle database 211 Roadside-vehicle communication device 212 Vehicle exterior database A Retinal reflection image position B, C Corneal reflection image position L Line of sight O Corneal sphere center Q Pupil center P Illumination light specular reflection point

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Hiroshi Saito 2 Takara-cho, Kanagawa-ku, Yokohama, Kanagawa Nissan Motor Co., Ltd. (72) Kazuo Arai, 2 Takara-cho, Kanagawa-ku, Yokohama, Kanagawa Nissan Motor Co., Ltd.

Claims (4)

[Claims] 1. An image input means provided toward a driver's eyeball area, and a line-of-sight direction detecting means for detecting the driver's line-of-sight direction based on image data of an eyeball portion obtained by the image input means, An area determination unit that determines whether or not the line-of-sight direction is within a predetermined gaze region, and a time measurement unit that measures a time during which the line-of-sight direction is outside the gaze region and compares the time with a predetermined reference time A front gaze degree detection device characterized by the above. 2. An image input means provided toward the eyeball region of the driver, and a visual line direction detection means for detecting the visual line direction of the driver based on the image data of the eyeball portion obtained by the image input means, At least 1 of running speed, elapsed time from start of running, steering angle of steering wheel, obstacle information and road shape
Environment detection means to obtain the environment information consisting of the above, the area determination means for determining whether the gaze direction is within a predetermined gaze area, and the time the gaze direction is outside the gaze area is measured, A forward gaze degree detection device, comprising: a time measuring means for comparing with a predetermined reference time, wherein the predetermined gaze area or reference time is set based on the environment information. 3. An image input means provided toward the eyeball region of the driver, and a line-of-sight direction detecting means for detecting the line-of-sight direction of the driver based on image data of the eyeball portion obtained by the image input means, A forward gaze degree detection device comprising: a pattern detection unit that detects the movement pattern in the line-of-sight direction; and a pattern comparison unit that compares the movement pattern with a predetermined reference pattern. 4. An image input means provided toward a driver's eyeball area, and a line-of-sight direction detecting means for detecting the driver's line-of-sight direction based on image data of the eyeball portion obtained by the image input means, At least 1 of running speed, elapsed time from start of running, steering angle of steering wheel, obstacle information and road shape
Environment reference means for acquiring the environment information consisting of the above, pattern detection means for detecting the movement pattern in the line-of-sight direction, and pattern comparison means for comparing the movement pattern with a predetermined reference pattern, wherein the reference pattern is A forward gaze degree detection device, which is set based on the environment information.