JP3111645B2 - Eye position detector - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an eye position detecting device for detecting the position of an eye, and more particularly, to detecting a driver's dozing or inattentive driving from an image of a driver's eye of a vehicle. The present invention relates to an eye position detecting device capable of detecting an eye position.

[0002]

2. Description of the Related Art Conventionally, an apparatus for detecting a position of a vehicle driver (including a partial determination of a face) by image processing is disclosed in, for example, JP-A-60-166807 and JP-A-60-168005. Something like that. These control the amount of light emitted from the light emitting means or the amount of light incident on the image pickup device, so that a clear image can always be obtained regardless of the brightness of the vehicle interior. (Including binarization), it is possible to accurately detect the position of the driver's eyes and the like.

[0003]

In the above-described conventional position detecting apparatus for a vehicle driver based on image processing, the control of the light amount of the light-emitting means and the control of the image pickup device are performed so as to correct the grayscale image information of the entire input image. Since the amount of incident light is not controlled, for example, when natural light having a much larger amount of light than the light emitting means enters a part of the input image, appropriate image processing is performed in the processing for the entire grayscale image. There is a problem that it may not be possible.

There are various processing methods such as a differential histogram method and a discriminant analysis method for binarizing image processing. All of these methods perform mathematical calculations on the grayscale information of the input image, set thresholds for binarization, convert eyes, eyebrows, hair, etc. into black pixels, forehead, cheeks, nose, Although the binarization method is originally used to convert the chin and the like to white pixels, as described above, in the binarization using the threshold value that is mathematically determined for all pixels, a portion of the eye There is a problem that may be converted to white pixels, or the part of the shadow that can be formed on the face may be converted to black pixels, making it impossible to distinguish from the black pixels of the real eye,
As a result, in an ever-changing illuminance environment such as in a vehicle cabin,
It is not always possible to accurately detect the driver's state due to dozing detection or the like.

[0005] The present invention has been made in view of the above,
An object of the present invention is to provide an eye position detection device that can accurately detect the position of an eye without being affected by the brightness of the surrounding environment.

[0006]

In order to achieve the above object, an eye position detecting apparatus according to the present invention, as shown in FIG. 1, converts at least a face image including an eye into a first predetermined area comprising a plurality of unit pixels. And image input means CL1 for inputting density information for each unit pixel of the first predetermined area; and dividing at least a part of the first predetermined area into a plurality of second predetermined areas, Area measuring means C for measuring the number of unit pixels each having density information equal to or more than a predetermined value
L2, correction means CL3 for correcting a threshold value for each of the plurality of second predetermined areas based on the measurement value of the area measurement means, and the image input means based on the threshold value corrected by the correction means. A binarizing unit CL4 for binarizing the density information of each unit pixel input by the above-mentioned method, and an eye existing region specifying unit CL5 for specifying an eye existing region from the information binarized by the binarizing unit. It is the gist to have.

[0007]

According to the eye position detecting device of the present invention, at least a part of the face image including the eyes is divided into a plurality of second predetermined areas, and each of the second predetermined areas has density information equal to or more than a predetermined value. The number of pixels is measured, the threshold value is corrected for each second predetermined area based on the measured value, and the density information for each unit pixel of the input image is binarized based on the corrected threshold value. The eye existence region is specified from the conversion information.

[0008]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 2 is a block diagram showing a configuration of an eye position detecting apparatus according to one embodiment of the present invention. The eye position detection device shown in FIG. 1 is applied to detect the position of the driver's eye of the vehicle, and includes an image input unit CL1 for capturing the driver's face and a background portion around the face. It has a CCD camera 1 to configure. The image information captured by the CCD camera 1 is converted into a digital signal via the A / D converter 3 and stored in the image memory 5.

The image of the driver's face taken by the camera 1 and the image of the background surrounding the face are, for example, as shown in FIGS. Is adjusted to 520 pixels in the horizontal (X) direction,
It is configured as an image composed of 500 pixels in the vertical (Y) direction.

The image signal stored in the image memory 5 is supplied to an image processing circuit 7 constituting the area measuring means CL2, the correcting means CL3 and the binarizing means CL4. A unit having an image of a background portion as a first predetermined region including a plurality of unit pixels, at least a part of the region is divided into a plurality of second predetermined regions, and each of the second predetermined regions has density information equal to or more than a predetermined value. Area measuring means CL2 for measuring the number of pixels, correcting means CL3 for correcting a threshold value for each of a plurality of second predetermined areas based on the measured values, and unit pixels of the input image based on the corrected threshold values Binarize density information for each 2
Each function of the value conversion means CL4 is achieved.

Then, the output signal of the image processing circuit 7 is supplied to an eye existence area specifying section 9 constituting the eye existence area specifying means CL5. Here, an eye signal is obtained from the information binarized by the binarization means. The existence area of is specified.

Next, the overall operation will be described with reference to the flowcharts shown in FIGS.

In FIG. 3, the face of the driver of the vehicle and
An image of a background portion around the face is picked up by the CCD camera 1, converted into a digital signal via the A / D converter 3, and stored in the image memory 5 as density gradation information (step 101).

The gradation J (x,
y) indicates that the black (dark) portion has J (x, y) = 0,
25 where the white (bright) part is J (x, y) = 255
Expressed in six gradations. The input image stored in the image memory 5 is configured as an image composed of 520 pixels in the horizontal (X) direction and 500 pixels in the vertical (Y) direction as described above with reference to FIGS. Is adjusted so that is almost full in the vertical direction.

The following step 10 shown in FIGS.
In steps 2 to 203, as a first process, a process of detecting the position of the driver's eyes in a light environment in which the sun is on the driver's seat side and the right half of the driver's face is illuminated by light will be described. .

First, at step 102, a bright pixel counter N for counting the number of very bright pixels exceeding a predetermined density gradation, J (x, y) = 250, is provided. N is first reset, and in the image shown in FIG.
From 0 to 250, the Y direction is scanned from Y = 0 to 200, and Y = 0 is set to check the density of each pixel in the left region of the upper half of the same image (step 102).
0 is also set (step 103).

First, the density gradation J (x, y) of the pixel at the position of X = 0, Y = 0, that is, J (0,0) is 2
Check whether it is greater than 50 (Step 10)
4). This density gradation J (x, y) is larger than 250,
If it is very bright, the bright pixel counter N is incremented by one (step 105).

Then, it is checked whether or not X becomes 250 (step 106). If X is less than 250, X is incremented by +1 (step 107), and this incremented (X, Y) = ( Similarly, it is checked whether or not the density gradation J (x, y) of the (1, 0) pixel is greater than 250 (step 104). If it is, the bright pixel counter N is similarly counted up (step 1).
05). The same processing of steps 104 to 107 is performed by X
Is repeatedly executed until Y becomes 250, and X when Y = 0
= 0 to X = 250, the density gradation J (x,
Check y).

When X = 250, it is checked whether or not Y becomes 200 (step 108). If Y is less than 200, Y is incremented by +1 (step 1).
09), and this incremented (X, Y) = (0,
It is similarly checked whether the density gradation J (x, y) of the pixel of 1) is greater than 250 (step 104), and if it is, the bright pixel counter N is counted up (step 105). The same processing of steps 104 to 107 is repeatedly executed until X becomes 250, and the density gradation J of each pixel from X = 0 to X = 250 when Y = 1
Check (x, y). Hereinafter, the same operation is performed with Y = 2.
00, so that X = 0 to 250,
The density gradation J (x, y) of each pixel of Y = 0 to 200 is checked, and the number of very bright pixels whose density gradation J (x, y) is larger than 250 is counted by the bright pixel counter N. Is done.

The following steps 110 to 117 correspond to FIG.
The density gradation J of each pixel in the upper right region of the image shown in FIG.
Check if (x, y) is greater than 250,
A process similar to the above-described steps 102 to 109 for counting the number of very bright pixels larger than 250 is performed, and another bright pixel counter M is set for this purpose.
First, the bright pixel counter M is reset to 0, Y = 0 is set (step 110), and X = 2
60 (step 111), whereby X = 26
Each pixel of 0 to 520 and Y = 0 to 200 is scanned. This process is similar to steps 103 to 109 described above.
The density gradation J (x, y) of each pixel in this scanning range is 250
It is checked in step 112 whether the value is greater than
If it is larger, the bright pixel counter M is counted up by +1 (step 113), X is incremented by +1 (step 115) until X = 520 (step 114), and Y is further increased until Y = 200 (step 11).
6), Y is incremented by +1 (step 11)
7).

X = 260-520 and Y = 0-200
, The density gradation J (x,
When the check in y) is completed, the process proceeds from step 116 in FIG. 3 to step 201 in FIG.

In step 201, it is determined whether or not the difference (N−M) between the count values N and M of the bright pixel counters N and M counted in steps 105 and 113 is larger than a predetermined pixel number P. To check. In this embodiment, since a right-hand drive vehicle is assumed, NM is taken as the difference between the count values of the bright pixel counter. Since the directions are reversed, it is sufficient to check whether or not the difference of the inverse MN is larger than a predetermined number of pixels P.

If the difference NM is less than the predetermined number of pixels P, it is determined that there is no difference between the brightness of the left and right images of the driver's face, and as shown by the jump symbol B, as shown in FIG. When the difference NM is larger than the predetermined number of pixels P, it is determined that the light environment is such that the sun is on the driver's seat side and the right half of the driver's face is illuminated by light. Judge, step 2
Go to 02.

In step 202, when the image signal is binarized, the driver's left eye not exposed to direct sunlight is exposed to the binarization threshold automatically set from the image density information. In order to enable detection in an appropriate state, a binarization threshold value taking into account the correction value α1 is set. Thereafter, in step 203, a one-eye detection process is performed.

This one-eye detection process is performed in the manner described in Japanese Patent Application No. 2-4025.
As described in No. 12, the density change point is detected, and the eyeball detection area is set on the left side of the density change point. Then, the input image is binarized using the threshold value set in step 202. An eye existence area is set in the eyeball search area set last. This processing is the same as the processing described in Japanese Patent Application No. 2-402512, and if only the search start point is the end point of the eyeball search area by the density change point, search for only one eye is possible.

In the check in step 201,
When it is determined that the difference NM is less than the predetermined number of pixels P, there is no difference between the brightness of the left and right images, and it is determined that the environment is not a positive light environment, then it is shown in FIGS. In the processing, it is determined whether or not the front is positive.

That is, in the processing of FIGS. 5 and 6, the sun is in front of the vehicle, the altitude is high, or a sun visor is used, and the light shines on the lower half of the driver's face. This is to determine the environment.

More specifically, it is checked whether or not the density gradation J (x, y) of each pixel in the upper half area of the image shown in FIG. 13 is larger than 250, and it is larger than 250 and very bright. The same processing as described above for counting the number of pixels is performed. For this purpose, a bright pixel counter N is set,
The bright pixel counter N is reset to 0, and Y =
0 (step 301), and further sets X = 0 (step 302), whereby X = 0 to 520, Y =
Each pixel of 0 to 200 is scanned. This processing is similar to the above-described processing, and the density gradation J of each pixel in this scanning range is
Step 303 determines whether (x, y) is greater than 250.
And if it is large, the bright pixel counter N
Is incremented by +1 (step 304), and X = 5
Until 20 (step 305), X is incremented by +1 (step 306), and Y is incremented by +1 (step 308) until Y = 200 (step 307).

When the density gradation J (x, y) of each pixel in the upper half area of the image of X = 0 to 520 and Y = 0 to 200 is completed, the lower half area of the image shown in FIG. Is checked whether or not the density gradation J (x, y) of each pixel is larger than 250, and the same processing as described above for counting the number of very bright pixels larger than 250 is performed. For this purpose, a bright pixel counter M is set, the bright pixel counter M is reset to 0, and Y = 20.
1 (step 309), and further sets X = 0 (step 310), whereby X = 0 to 520, Y =
Each pixel 201 to 500 is scanned. This processing is the same as the processing described above, and it is determined in step 31 whether or not the density gradation J (x, y) of each pixel in this scanning range is greater than 250.
If it is checked at 1 and it is larger, the bright pixel counter M is incremented by +1 (step 312), and X =
Until 520 (step 313), X is incremented by +1 (step 314), and Y is incremented by +1 (step 316) until Y = 500 (step 315).

X = 0 to 520 and Y = 201 to 500
When the check of the density gradation J (x, y) of each pixel in the lower half area of the image is completed, the process proceeds from step 315 in FIG. 5 to step 401 in FIG.

In step 401, it is determined whether or not the difference (M-N) between the count values N and M of the bright pixel counters N and M counted in steps 304 and 312 is greater than a predetermined number Q of pixels. To check.

If the difference M−N is less than the predetermined number of pixels Q, it is determined that there is no difference in brightness between the upper and lower images of the driver's face. When the difference M−N is larger than the predetermined number of pixels Q, the sun is in front of the vehicle and the altitude is high or the sun visor is used to light the lower half of the driver's face. It is determined that the light environment is such that the light is applied, and the process proceeds to step 402.

In step 402, when the image signal is binarized, the driver's eyes that are not exposed to direct sunlight are appropriately adjusted to the binarization threshold value automatically set from the image density information. In order to enable detection in a proper state, a binarization threshold value taking into account the correction value α2 is set. Thereafter, in step 403, a binocular detection process is performed on the entire face image in the eyeball detection area.

In the check in step 401,
The difference NM is less than the predetermined number of pixels Q, there is no difference in brightness between the upper and lower images, the front altitude where the altitude of the sun is high or the altitude of the sun is low, and it is not a light environment using the sun visor. If it is determined, the process shown in FIG. 7 then determines whether the light environment is such that the sun altitude is low and the sun visor is not used in the front sun.

That is, the processing in FIG. 7 determines a light environment in which the sun is in front of the vehicle, the altitude is low, and the entire driver's face is illuminated with no sun visor. It is.

More specifically, X = 160 to 3 which is a substantially central area in the driver's face image shown in FIG.
It is checked whether or not the density gradation J (x, y) of each pixel in the range of 59 and Y = 101 to 400 is larger than 250, and the number of pixels that are larger than 250 and very bright is counted. A bright pixel counter N is set, and the bright pixel counter N is set to 0.
And set Y = 101 (step 501), and further set X = 160 (step 50).
2), whereby X = 160-359, Y = 101-4
Each pixel in the range of 00 is scanned. This processing is similar to the above-described processing, and the density gradation J of each pixel in this scanning range is
Step 503 determines whether (x, y) is greater than 250.
And if it is large, the bright pixel counter N
Is incremented by +1 (step 504), and X = 3
Until 59 (Step 505), X is incremented by +1 (Step 506), and Y is incremented by +1 (Step 508) until Y = 400 (Step 507).

X = 160-359 and Y = 101-4
When the check of the density gradation J (x, y) of each pixel in the area of 00 is completed, in step 509, the bright pixel counter N
It is checked whether the count value N is larger than a predetermined number R of pixels.

If the count value N is less than the predetermined number R of pixels, it is determined that the entire face of the driver is not exposed to direct sunlight, and the process proceeds to the processes shown in FIG. However, when the count value N is larger than the predetermined number R of pixels, the sun is ahead of the vehicle, the altitude is low, the sun visor is not used, and the entire face of the driver is illuminated. It is determined that the lighting environment is
Proceed to.

In this step 512, when the image signal is subjected to the binarization processing, the driver's eyes exposed to direct sunlight are appropriately adjusted to the binarization threshold value automatically set from the image density information. In order to enable detection in a proper state, a binarization threshold value taking into account the correction value α3 is set. Thereafter, in step 511, a binocular detection process is performed on the entire face image in the eyeball detection area.

In the check in step 509,
If the count value N is less than the predetermined number of pixels R and it is determined that the light environment is low and the altitude of the sun is low, the sun is behind the vehicle in the processing of FIGS.
A determination is made as to whether or not the light environment causes backlight.

That is, the processes in FIGS. 8 to 10 are for judging a light environment in which the sun is behind the vehicle, the light is backlit, and the driver's face is shadowed.

More specifically, in the image shown in FIG. 15, it is checked whether or not the density gradation J (x, y) of each pixel in a region surrounding the face in a downward concave shape is larger than 250. The number of very bright pixels larger than 250 is counted by the bright pixel counter N, and the density gradation J (x, y) of each pixel in the downward concave internal region is 250.
The number of very bright pixels larger than 250 is counted by the bright pixel counter M. From the count values of both bright pixel counters, the light environment, that is, when the sun is behind the vehicle, , And performs the same processing as described above to determine whether or not there is a light environment in which the face of the driver is shadowed.

For this purpose, first, the bright pixel counter N is reset to 0, and Y = 0 is set (step 60).
1) Further, X = 0 is set (step 602), whereby each pixel in the range of X = 0 to 159 and Y = 0 to 500 is scanned. This processing is the same as the above-described processing. It is checked in step 603 whether or not the density gradation J (x, y) of each pixel in this scanning range is greater than 250. N is incremented by +1 (step 604), and X is incremented by +1 until X = 159 (step 605) (step 6).
06), and until Y = 500 (step 60).
7), Y is incremented by +1 (step 60)
8).

When the check of the density gradation J (x, y) of each pixel in the region on the left side of the image of X = 0 to 159 and Y = 0 to 500 is completed, in the next steps 609 to 616, the image shown in FIG. The density gradation J (x, x) of each pixel in a part of the upper part, that is, the area of the hair part substantially at the center of the head
Check if y) is greater than 250,
In order to perform a process of continuously counting the number of pixels that are larger and very bright by the bright pixel counter N, first, Y = 0
Is set (step 609), and X is set to 160 (step 610), whereby X = 160 to 359,
Each pixel in the range of Y = 0 to 100 is scanned. This processing is the same as the above-described processing. It is checked in step 611 whether or not the density gradation J (x, y) of each pixel in this scanning range is greater than 250. N is incremented by +1 (step 61)
2) Until X = 359 (step 613), X is incremented by +1 (step 614), and Y =
Until the value reaches 100 (step 615), Y is incremented by +1 (step 616).

X = 160-359 and Y = 0-100
When the check of the density gradation J (x, y) of each pixel in the region is completed, the process proceeds to FIG. 9 with a jump symbol F. In steps 701 to 708, each of the regions on the right side of the image shown in FIG. Pixel density gradation J (x, y) is 25
It is checked whether or not the number is greater than 0, and the number of very bright pixels greater than 250 is counted by the bright pixel counter N continuously. For this purpose, first, Y = 0 is set (step 701), and further, X is set to 360 (step 702), whereby X = 360 to 520 and Y = 0.
Each pixel in the range of ~ 500 is scanned. This processing is the same as the above-described processing, and it is determined in step 70 whether or not the density gradation J (x, y) of each pixel in this scanning range is greater than 250.
If it is checked in step 3 that it is larger, the bright pixel counter N is incremented by 1 (step 704), and X =
Until 520 (step 705), X is incremented by +1 (step 706), and Y is incremented by +1 (step 708) until Y = 500 (step 707).

X = 360-520 and Y = 0-500
Density gradation J of each pixel in the area on the right side of the image of FIG.
When the check of (x, y) is completed, similarly, it is determined whether or not the density gradation J (x, y) of each pixel in the above-mentioned downward concave inner region of the image shown in FIG. Is checked, and the same processing as described above for counting the number of very bright pixels larger than 250 is performed. For this purpose, the bright pixel counter M is set, and the bright pixel counter M is reset to 0. At the same time, Y = 101 is set (step 709), and further, X is set to 160 (step 710), whereby X = 160 to 359 and Y = 10
Each pixel in the range of 1 to 500 is scanned. This processing is the same as the processing described above, and it is determined whether or not the density gradation J (x, y) of each pixel in this scanning range is greater than 250 (step 7).
If it is checked in step 11 that it is larger, the bright pixel counter M is incremented by +1 (step 712), and X
X is incremented by +1 (step 714) until Y = 359 (step 713), and Y is incremented by +1 (step 716) until Y = 500 (step 715).

X = 160-359 and Y = 101-5
When the check of the density gradation J (x, y) of each pixel in the downward concave inner region of the image 00 is completed, the process proceeds from step 715 in FIG. 9 to step 801 in FIG.

In step 801, the count values N and M of the bright pixel counters N and M counted in steps 601 to 708 and steps 709 to 716 described above, respectively.
It is checked whether or not the difference (N−M) is larger than a predetermined number S of pixels.

If the difference NM is less than the predetermined number of pixels S, it is determined that there is no difference between the brightness of the background and the image of the face of the driver, and as shown by the jump symbol H in FIG. When the difference NM is larger than the predetermined number of pixels S, the process is determined to be in a light environment in which the sun is behind the vehicle and the face of the driver is shaded by the backlight. , Step 80
Proceed to 2.

In step 802, when the image signal is binarized, the driver's eyes, which are shadowed by the binarization threshold value automatically set from the image density information, are kept in an appropriate state. In order to enable detection, a binarization threshold value taking into account the correction value α4 is set. Then, step 80
In step 3, the eyeball detection area performs a binocular detection process on the entire face image.

In the check of the step 801,
If it is determined that the difference NM is less than the predetermined number S of pixels and there is no difference between the brightness of the background and the face, and the light environment is not backlighting, then it is shown in FIG. Proceeding to the processing, it is determined whether or not the light environment has sufficient brightness.

That is, the processing in FIG. 11 is for determining a light environment in which sufficient brightness is maintained.
X = 160 to 359 and Y = 101 to 40, which are the substantially central face areas in the driver's face image shown in FIG.
It is checked whether or not the density gradation J (x, y) of each pixel in the range of 0 is larger than 150 which is a bright pixel of the intermediate gradation.
The same processing as described above for counting the number of bright pixels larger than 150 is performed. For this purpose, a bright pixel counter N for an intermediate gradation is set, and the bright pixel counter N is reset to 0. , Y = 101 (step 901), and further set X = 160 (step 90).
2), whereby X = 160-359, Y = 101-5
Each pixel in the range of 00 is scanned. This processing is similar to the above-described processing, and the density gradation J of each pixel in this scanning range is
Step 903 determines whether (x, y) is greater than 150.
And if it is large, the bright pixel counter N
Is incremented by +1 (step 904), and X = 3
Until 59 (Step 905), X is incremented by +1 (Step 906), and Y is incremented by +1 until Y = 500 (Step 907).

X = 160-359 and Y = 101-5
When the check of the density gradation J (x, y) of each pixel in the area of 00 is completed, in step 909, the bright pixel counter N
It is checked whether the count value N is larger than a predetermined number T of pixels.

If the count value N is larger than the predetermined number of pixels T, it is determined that sufficient brightness is maintained.
In step 910, the binocular detection processing is performed on the entire face image in the eyeball detection area, and the processing ends. If the count value N is less than the predetermined number of pixels T, it is determined that the light environment does not maintain sufficient brightness of the driver, and the process proceeds to step 911. In this step 911, when the image signal is subjected to the binarization processing, the eyes of the driver from the face which is entirely dark with respect to the binarization threshold value which is automatically set from the density information of the image are appropriately adjusted. In order to enable detection, a binarization threshold value taking into account the correction value α5 is set. Then, step 9
In step 12, the eyeball detection region performs the binocular detection process on the entire face image, and the process ends.

FIG. 16 is a flowchart showing the operation of the eye position detecting apparatus according to another embodiment of the present invention.

In the embodiment shown in FIG. 16, a plurality of reference patterns are prepared by patterning the density information of the image obtained by capturing the driver's face in accordance with each light environment as described with reference to FIGS. In advance, it is determined which reference pattern the captured image corresponds to, the light environment is estimated according to the determined reference pattern, and the binarization correction and the eyeball position search method are determined.

That is, in FIG. 16, first, the entire image area is set in advance as shown in FIG.

The density information of each area is obtained by dividing into the divided areas and it is determined to which preset reference pattern the density information corresponds (step 140).
1).

FIGS. 18A to 18F show six types of reference patterns as an example.

However, six types of reference patterns are created using a map indicating whether the number of pixels whose density gradation J (x, y) exceeds 250 is large or small, for example, in each of the above areas (1) to (4). ing. In step 1401, it is determined which of the six types of reference patterns the image density information corresponds to.

When the corresponding reference pattern is determined, the light environment state in the vehicle cabin is estimated from the reference pattern, and the binarization correction value and the eyeball position search method are determined (step 1402). Then, after the correction, the threshold value set by the automatic binarization for all the images is subjected to the binarization process after the correction using the value determined by the reference pattern (step 1403). And finally, step 1
An eyeball is searched by the eyeball position search method determined in 402 (step 1404).

FIGS. 19A to 19E are explanatory diagrams showing the relationship between the above-described plurality of main light environment conditions and the method of correcting the binarization threshold value for the light environment conditions in an easy-to-understand manner.

First, FIG. 19A shows the light environment when the position of the sun is ahead of the vehicle in fine weather and the height of the sun is high (a) and when the sun is low (b). In the figure, the hatched portion is a shade.

In the case where the height of the sun is high in FIG.
In this case, light hits the vicinity of the driver's chin, and the eyes are shaded. In extracting the eye part in the shade, the threshold value by the automatic binarization is set to be large due to the influence of the part not illuminated. Becomes large, and the identification of the shadow and the eye portion becomes inappropriate. Therefore, under such light environment conditions, automatic 2
It is necessary to correct the threshold value to a smaller value than the threshold value for binarization.

In the case where the height of the sun is low in FIG.
Illuminates the entire driver's face. In other words, the light directly hits the eye, and the part of the eye is crushed in the state of the current image,
It becomes whitish. Further, the threshold value of the automatic binarization does not become as large as possible to extract the eyes due to the influence of the background portion which is not illuminated, so that the eyes disappear. Therefore, under such light environment conditions, the threshold value needs to be largely corrected with respect to the threshold value of automatic binarization.

FIG. 9B shows a case where the sun is positioned behind the vehicle and the height of the sun is high in fine weather (a).
(B) is the light environment.

(B) In the case where the height of the sun is high in FIG.
In both cases (b), the driver's background becomes bright and the entire face becomes a shadow. By brightening the background,
As a result, the binarization threshold value is set to a large value. Therefore, the area of the face portion that is to be a shadow is determined to be a black pixel, and the discrimination between the shadow and the eye becomes inappropriate. Therefore, under such light environment conditions, the threshold for automatic binarization is
The threshold value needs to be corrected smaller.

FIG. 3C shows a case where the sun is on the driver's seat side and the height of the sun is high in fine weather (a).
(B) is the light environment.

In the case where the height of the sun is high in FIG.
In both cases (b), light strikes the right half of the driver's face, and the right eye is crushed by the light in the current image, so that it is difficult to extract both eyes. Therefore, in order to extract the part of the eye in the shade inside the vehicle, the threshold value by the automatic binarization is set to be large due to the influence of the part that is illuminated. Becomes large, and the identification of the shadow and the eye part becomes inappropriate. Therefore,
Under such light environment conditions, the threshold value needs to be corrected to be smaller than the threshold value of the automatic binarization.

FIG. 9D shows the light environment when the sun is in the passenger seat in fine weather and the height of the sun is high (a) and low (b).

In the case where the height of the sun is high in FIG.
In both cases (b) and (b), the entire face of the driver is shaded, but since it is uniformly and relatively bright, an appropriate binarized image is obtained by setting the threshold by automatic binarization. No changes are required.

FIG. 9E shows the light environment when the weather is cloudy. In this case, although the condition is close to the condition when the sun is on the passenger seat side, the image data is low in contrast because the entire screen is dark and the amount of light is insufficient. In addition, since the face part is darker than the background, the area of the face part that is determined as a black pixel increases, and the identification of the shadow and the eye part becomes inappropriate.
Therefore, under such light environment conditions, the threshold value needs to be corrected to be smaller than the threshold value of the automatic binarization.

[0072]

As described above, according to the present invention,
At least, at least a part of the face image including the eyes is divided into a plurality of second predetermined regions, and the number of unit pixels having density information equal to or higher than a predetermined value is measured in each of the second predetermined regions,
The threshold value is corrected for each second predetermined area based on the measured value, and the density information for each unit pixel of the input image is binarized based on the corrected threshold value. Since the region is specified, the position of the eyeball can be accurately detected even when the external light condition changes complicatedly in the running vehicle.

[Brief description of the drawings]

FIG. 1 is a diagram corresponding to claims of the present invention.

FIG. 2 is a block diagram illustrating a configuration of an eye position detection device according to an embodiment of the present invention.

FIG. 3 is a flowchart showing an operation of the eye position detecting device shown in FIG.

FIG. 4 is a flowchart showing an operation of the eye position detection device shown in FIG.

FIG. 5 is a flowchart showing the operation of the eye position detecting device shown in FIG.

6 is a flowchart showing the operation of the eye position detecting device shown in FIG.

FIG. 7 is a flowchart showing the operation of the eye position detecting device shown in FIG.

8 is a flowchart showing the operation of the eye position detecting device shown in FIG.

FIG. 9 is a flowchart showing the operation of the eye position detecting device shown in FIG.

FIG. 10 is a flowchart showing the operation of the eye position detection device shown in FIG.

11 is a flowchart showing the operation of the eye position detecting device shown in FIG.

FIG. 12 is an explanatory diagram showing an image and a coordinate system used for explaining the operation of the eye position detecting device shown in FIG. 2;

FIG. 13 is an explanatory diagram showing an image and a coordinate system used for explaining the operation of the eye position detecting device shown in FIG. 2;

FIG. 14 is an explanatory diagram showing an image and a coordinate system used for explaining the operation of the eye position detecting device shown in FIG. 2;

FIG. 15 is an explanatory diagram showing an image and a coordinate system used for explaining the operation of the eye position detecting device shown in FIG. 2;

FIG. 16 is a flowchart illustrating an operation of an eye position detection device according to another embodiment of the present invention.

FIG. 17 is an explanatory diagram showing an image, a coordinate system, and a plurality of divided regions used for explaining the operation of the eye position detecting device shown in FIG.

18 is a diagram showing a reference pattern used in the eye position detecting device shown in FIG.

FIG. 19 is an explanatory diagram showing a relationship between a plurality of main light environment conditions and a method of correcting a binarization threshold value with respect to the light environment conditions for easy understanding.

[Explanation of symbols]

 Reference Signs List 1 CCD camera 3 A / D converter 5 Image memory 7 Image processing circuit 9 Eye existence area specifying unit

Claims (2)

(57) [Claims] At least a face image including an eye is defined as a first predetermined area including a plurality of unit pixels, and the first predetermined area includes a plurality of unit pixels.
Image input means for inputting density information for each unit pixel of the predetermined area; at least a part of the first predetermined area is divided into a plurality of second predetermined areas, and density information having a predetermined value or more in each of the second predetermined areas. Area measuring means for measuring the number of unit pixels having the following formula:
Correction means for correcting a threshold value for each area of the predetermined area; density information for each unit pixel input by the image input means based on the threshold value corrected by the correction means;
An eye position detecting device comprising: a binarizing unit for converting a value; and an eye existing region specifying unit for specifying an eye existing region from information binarized by the binarizing unit. 2. The method according to claim 1, wherein when the number of unit pixels having density information equal to or more than a predetermined value in each of the second predetermined areas is equal to or less than a first predetermined number, at least a part of the first predetermined area is determined. Dividing the image into a plurality of third predetermined areas different from the second predetermined area, and measuring the number of unit pixels having density information equal to or more than a predetermined value in each of the third predetermined areas; the third time unit number of pixels having density information of a predetermined value or more in each of the predetermined regions under the predetermined number or less, characterized in that it has means for correcting the threshold value for each of each of the plurality of third predetermined region The eye position detecting device according to claim 1, wherein