JPH03533A - Detecting device for driver's operating condition - Google Patents

Abstract

PURPOSE:To judge the operating condition of a driver by extracting the existence of the driver's eyes from inputted image the information of which has been processed based on the reference point of a reference point indicating means, and thereby detecting a section corresponding to the erises of the driver. CONSTITUTION:It is caught by an infra-red image pick-up means 2 that 4 reference points in the center section out of those reference points 51 through 5n are hidden by the head of a driver, and 3 each of reference points 51 through 53, and 58 through 5n at the right and the left side are lighted. Infra-red rays from an infra-red ray irradiation means 1 are irradiated onto the driver's face by means of synchronizing signals Q3 from a synchronizing signal output means 3, the image of its reflection pattern and the light from the reference points 51 through 5n is picked up by an infra-red image pick-up means 2, and the image of luminescent points are extracted, which is formed by the respective 3 reference points 51 through 53, 58 through 5n at the right and left sides projected in an inputted image I (x, y) by an eye-ball existing range specifying means 6, so that the existing range of the driver's eyes is thereby specified. An eris detecting means 7 detects the section corresponding to the erises of the driver, and an operation judgement means 8 judges whether or not the driver's eyes are opened so that a warning command is thereby outputted to a warning output means 9.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of the Invention The present invention relates to a device for detecting driving conditions, such as whether a driver's eyes are facing forward, open, or closed.

As a conventional technical driver's driving state detection device, for example,
-158303, JP 60-158304, JP 61-77705, JP 61-77
Some of these are disclosed in Publication No. 706 and the like. These systems irradiate infrared rays onto the driver's face, including both eyes, from an infrared irradiation device installed inside the vehicle, and image the reflection pattern of this infrared rays with an infrared imaging device installed inside the vehicle to create images in bright and dark areas. It is structured to process.

Problems to be Solved by the Invention However, although the above-mentioned device can recognize the position of the driver's eyes from the shape of the bright and dark areas in the image, it cannot detect whether the driver's eyes are open or closed. In some cases, the positions of the eyes and eyebrows may be misidentified.

Means for Solving the Problem In a device that detects the driving condition of a driver by image processing a reflection pattern of infrared rays irradiated onto the driver's face including both eyes, there is provided a reference point for detecting the position of the driver's eyes. a reference point specifying means for instructing the reference point, an eyeball presence region defining means for extracting the driver's eye region from the image-processed input image based on the reference point of the reference point indicating means; and the eyeball presence region defining means The present invention includes an iris detection means for detecting a portion corresponding to the driver's iris within the eyeball existing region extracted in step 1, and a driving state determining means for determining the driving state of the driver from the detection result of the iris detection means.

Since the eyeball presence area can be extracted based on reference points from the input image that has been subjected to action image processing, it is possible to accurately measure the eyeball position, and the area corresponding to the iris can also be detected, and from this detection result, the driver's It determines the driving state, such as whether the eyes are facing forward, open, or closed.

Embodiment As shown in FIG. 1-10, this embodiment generally includes an infrared irradiation means 1, an infrared imaging means 2, a synchronization signal output means 3, an image memory 4, a reference point indicating means 5, and an eyeball presence area. Regulation means 6, iris detection means 7, and driving state determination means 8
and alarm output means 9.

The infrared irradiation means 1 irradiates the driver's face including both eyes with infrared rays, and is configured as a so-called infrared strobe. It is attached to a part that does not interfere with sex.

The infrared imaging means 2 images the reflection pattern of the infrared rays irradiated on the driver and outputs an image signal, and is configured as a so-called infrared camera. It is attached to a part where the pattern can be imaged and does not obstruct the driver's visibility to the front and sides.

The synchronization signal output means 3 outputs a synchronization signal Q to the infrared irradiation means 1 and the infrared imaging means 2, which synchronizes the timing of the infrared irradiation of the infrared ray irradiation means 1 and the reflection pattern imaging of the infrared imaging means 2.

The image memory 4 stores an image 1 (x, y) as an image signal output from the infrared imaging means 2 at - time. For example, this image 1 (x, y) has M pixels in the horizontal direction (X direction),
It is composed of N pixels in the vertical Y direction.

The reference point indicating means 5 is a reference point 5 for detecting the position of the driver's eyes.
1,5t...5, which indicates
It is composed of a plurality of light emitters such as LEDs or reflectors such as mirrors, and these light emitters or reflectors illuminate the front and sides of the driver within the infrared irradiation field emitted from the infrared irradiation means l. For example, ten head supports of the headrest 10, which are one of the parts in the vehicle interior that do not obstruct the visibility of the vehicle, are mounted horizontally in a horizontal row at equal intervals.

The eyeball existing area defining means 6 inputs the image 1 (x, y) temporarily stored in the image memory 4, and inputs this input image I (x
, y) to a reference point 5. by a light emitter or a reflector serving as a reference point indicating means 5. ．． 5. , . . . 51, the region where the driver's eyes exist is extracted.

The iris detection means 7 detects a portion corresponding to the driver's iris within the eyeball existence region extracted by the eyeball existence region defining means 6.

The driving state determining means 8 determines the driving state of the driver, such as a normal driving state, a distracted driving state, or a drowsy driving state, from the detection result of the iris detecting means 7.

The alarm output means 9 is installed in the vehicle interior, and generates an alarm by a buzzer, chime, voice, etc. when the driving state determining means 8 determines that the vehicle is in a distracted driving state or a drowsy driving state.

Note that the synchronizing signal output means 39 image memory 4. Eyeball presence area defining means 6. Iris detection means7. The driving state determining means 8 and the like are integrated into one control device configured in a microcomputer and attached to the vehicle body.

According to the structure of the above embodiment, as shown in FIG. 10 reference points 5. ．． 5. ,...
...Four points in the center of 5o are hidden by the driver's head, and there are three reference points on each side, 51°5, . 5. ．． 5. ．． 5. ．． 5
In a state where the infrared imaging means 2 captures that I is lit, the synchronization signal Q from the synchronization signal output means 3
3, infrared rays are irradiated from the infrared ray irradiation means l to the driver's face including both eyes, and the reflection pattern of this infrared rays and the reference point 5. ．． 5. ．． 5. ．． 5. ．． 5. ．． 5. is imaged by the infrared imaging means 2, and the image 1
(x, y) is temporarily stored in the image memory 4. Then, the eyeball existence area defining means 6 manually inputs the image 1 (x, y) temporarily stored in the image memory 4, and inputs this input image ■(x, y).
5. Three reference points on the left and right shown in ). ．． 5. ．．
5. ．． 5. ．． 5. ．． A bright spot image based on 5° is extracted, and this extracted reference point 5. .

5. 53,5. ．． 5. ．． 5n defines the region where the driver's eyes exist. Here, the reference points 51 to 5° can be used as reference points since they always emit light of a predetermined wavelength at a predetermined position when performing image processing.

Next, the operation of the eyeball existing area defining means 6 will be explained in detail based on the flowchart in FIG. 3 and the operation explanatory diagrams in FIGS. x, y).

Next, in step 102, reference point 5. ．． 5. ．． 53°5,
．． 5. ．． 5. Input image 1 (x, y) in step 101 with a threshold set to a level that allows extraction of .
is binarized and only the bright spot image is extracted.

As shown in FIG. 5, the image thus generated is J (
x, y).

l... Reference point (bright spot image) J (x, y) = 0... In addition, in step 103, by labeling the image J (x, y), the image shown in Fig. 6 is obtained. As shown, arithmetic numbers 1, 2, 3, 4, 5.6 indicating the number of areas are sequentially attached to the bright spot images from the one with the smaller X coordinate to the bright spot image with the larger X coordinate, and the image K (x, y
), and temporarily stores pixel values corresponding to the number of areas 1 to 6.

Specifically, when the number of regions is i, a pixel value i corresponding to the i-th region number is temporarily stored.

In step 104, the maximum number of areas is 1 and waax is the reference number n.
o For example, it is determined whether or not it is 7 or less, that is, whether or not the driver's head partially blocks the reference points 51 to 5□ of the headrest 10. When the maximum number of regions i, □ is less than or equal to the reference number n0, that is, 7, the driver's head is in the image K(x, y).
The process proceeds to step 105 assuming that the normal operating position is completely within the frame. Maximum number of regions i. 3 is the standard value n
. If it is larger than 8, that is, 8 or more, the driver's head does not fit into the frame of the image K(x, y) and the process returns to step 101.

In step 105, initial values i and h are set to 1.0.

In step 106, the area quantity is the maximum area number 1 waax
Determine whether or not. The number of regions i is the maximum number of regions (@a
If it is less than or equal to x, proceed to step 107, and if the number of areas i is the maximum number of areas f waax, proceed to step 111.
Proceed to.

In step 107, the labeled adjacent bright spot images i
The interval between the bright spot image 1+1 and the bright spot image 1+1 is determined. In other words, since the reference points 51 to 5n on the headrest 10 are arranged at equal intervals, if the reference points 5 and 5n are not blocked by the driver's head, the distance between the bright spot image i and the bright spot image i+l is becomes the default value. If the interval between the bright spot image i and the bright spot image 1+1 is the specified value, proceed to step 108; if it is not the specified value, proceed to step 10.
Proceed to step 9.

In step 108, 1 is added to i in step 105 (
i=i+1) and return to step 106.

In step 109, the bright spot image i and bright spot image i+1 in step 107 are temporarily stored. The process then proceeds to step 11O.

In step 110, 1 is added to h in step 105, and the process proceeds to step 108.

Step on the other hand! 11, adjacent bright spot images i and bright spot images i+
1. It is determined whether there is one location where the interval between the two locations is equal to or greater than a specified value. Specifically, it is determined whether h=1 or h≧2. If h≧2 and there are two locations where the interval between adjacent bright spot images i and bright spot images i+1 is equal to or greater than the specified value, it is determined that some noise has been mixed in and the process returns to step 101. h-1, and if there is one location where the interval between the adjacent bright spot images i and bright spot images 1+1 is equal to or greater than the specified value, the process proceeds to step 112.

In step 112, using predetermined values r, p, -q,
As shown in FIG. 6, the position r in the X direction from the reference point i, y
Two points, pixels A and H, located at positions p and -q in the direction are determined. The process then proceeds to step 113.

In step 113, two points, pixels C and D, which are located at the position r in the X direction and the positions p and -q in the y direction from the reference point i+1, are determined using the predetermined values -r and p+q, as shown in FIG. decide. The process then proceeds to step l14.

In step 114, pixels A, B, C, and D obtained in steps 112 and 113 are connected to generate an image L (x, y) as shown in FIG.
, D is filled in. As a result of this filling process, an image M(x,y) shown in FIG. 8 is generated.

By the way, input image 1 (
Since the portion corresponding to the iris is generally observed as a dark circular area in Set a rectangular area in each direction that intersects with , and calculate δ = (sum of brightness values of the shaded part in Figure 9 in the rectangular area) - (sum of brightness values of the white background part of Figure 9 in the rectangular area) Then, δ will output a prominent value at the center of the true circular region.

By utilizing such a principle, the iris detection means 7
As shown in the flowchart of FIG. 8, an iris-corresponding portion corresponding to the driver's iris is detected from an image M (x, y) obtained by converting input image 1 (x, y). In the flowchart shown in FIG. 8, since the radius of the iris to be detected varies depending on the individual or the distance between the camera and the passenger, zones within the detection radius are provided (R111tn to R,
,, ), and Δ= in the eyeball existing region as the final output.
It is designed to output δwax.

Here, when comparing when the eyes are open and when the eyes are closed, the maximum value δ, δ, , 8 within the previously set area M(x, y) becomes larger when the eyes are open, so this maximum value δwax is subjected to threshold processing. This allows you to determine whether the eyes are open or closed.

For this reason, the driving state discriminating means 8 compares Δ=δ, . If it is determined that the driver's eyes are closed, it is assumed that the driver is dozing, and an alarm command is output to the alarm output means 9, and the alarm output means 9 issues an alarm to call the driver's attention. In this case, if it is determined that the driver is dozing off by just one eye closure determination in the driving state determining means 8, there is a high possibility of a false alarm, so the same process is repeated multiple times. When the eyes closed determination is observed a number of times, for example, three or more times in a row, it is determined that the user is dozing, and an alarm is issued. Also, if only one eye is determined to be closed,
Input screen 1 (x, y
), it is thought that one eye is missing. Therefore, similarly to the case of dozing off, when it is determined that one eye is closed three times in a row, it is determined that the user is looking away.

It should be noted that the present invention is not limited to the above-mentioned embodiments, and a reference point indicating means 5A may be provided on the seat belt 11, for example, as shown in FIGS. 11 and 12. Specifically, reference point 5. ．． 5. ,...,5, seat belt 1
1, and a reference point on the back side of the seat belt 2.5. ．． 5
．． ,..., pressure sensor 1 is placed in the part corresponding to 5n.
2. ．． 12. ．． . . . , 12n are provided, and the pressure sensor 12. ．． 5. Reference point corresponding to the highest pressure part within 122.12n. ．． 5. .

. . ., a light emission control unit 13 is provided to light up 5a,
Light emission control section! 3 is inputted to the eyeball existing region defining means as the bright spot image in the embodiment described above.

Furthermore, in the embodiment described above, the iris-corresponding portion was detected by the iris detection means 7 using only rectangular areas in four directions, but the rectangular areas may be formed in more directions. In this case, the threshold value Th for determining whether the eyes are open or closed is different from that of the embodiment described above.

Effects of the Invention As described above, according to the present invention, the part corresponding to the driver's iris can be detected by simply searching a narrow area with reference to a reference point.
It is possible to reliably and quickly monitor the driving status of the driver, such as whether the driver's eyes are open or closed.

Further, according to this embodiment, it is possible to accurately detect dangerous driving conditions such as drowsiness or inattentiveness of the driver, and issue a detection warning.

[Brief explanation of drawings]

FIG. 1 is a configuration diagram showing one embodiment of the present invention, and FIG. 2 is a perspective view of a headrest provided with reference point indicating means of the same embodiment.
FIG. 3 is a flowchart of the eyeball presence area defining means of the same embodiment, and FIGS. 4 to 8 are explanatory diagrams of the main parts of the same flowchart,
FIG. 9 is an explanatory diagram showing the principle of iris detection in the same embodiment;
Figure O is a flowchart of the iris detection means of the same embodiment, the first
FIG. 1 is a perspective view showing a different example of the reference point indicating means of the present invention, and FIG. 12 is a sectional view taken along the line -XI in FIG. 11. l... Infrared irradiation means, 2... Infrared imaging means, 4
... Image memory, 5... Reference point indicating means, 6...
Eyeball presence area defining means, 7...iris detection means, 8...
・Driving status determination means. 3 people Figure 1 Figure 11 Figure 12 Figure Figure 1 Figure Figure Figure Figure Figure

Claims (1)

[Claims] (1) In a device that detects the driving condition of a driver by image processing a reflection pattern of infrared rays irradiated onto the driver's face including both eyes, a reference point for indicating a reference point for detecting the position of the driver's eyes. a point indicating means; an eyeball existence region defining means for extracting the driver's eye existence region from the input image subjected to the image processing based on the reference point of the reference point indicating means; and an eyeball extracted by the eyeball existence region defining means. A driver comprising: iris detection means for detecting a portion corresponding to the driver's iris within a presence area; and driving state determination means for determining the driving state of the driver from the detection result of the iris detection means. operating status detection device.