JPH0757072A - Driver's state detector - Google Patents

Abstract

PURPOSE:To provide a driver's state detector which is capable of improving the detection performance of the area of the eyes of a driver for detecting a doze by detecting the opening/closing states of the eyes of the driver. CONSTITUTION:A timer is started and an infrared ray lamp A causing the imprinting of glasses is turned on. By a presence or absence of glasses decision circuit 6, the presence or absence of a high luminance point of an infrared ray lamp A imprinted on the lens part of the glasses is decided. When a light luminance point exists, the eyes existence area is set. When the high luminance point does not exist, the retrieval of the high luminance point by the presence or absence of glasses decision circuit 6 is repeated for a preliminarily set prescribed time. In this case, when the high luminance point does not exist even if the prescribed time elapses, the timer is stopped and a timer counter is made to clear. Subsequently, the eyes area is set by an eyeball existence location designation circuit 7. After the eyes existence area is set, the infrared ray lamp A for the presence or absence of glasses is turned off. Subsequently, the decision of the opening and closing of the eyes is performed by an opening/closing eyes decision circuit 8 in the set eyes existence area. Based on the decision result of time series decision result of these opening and closing eyes, a doze decision is performed by a doze decision circuit 9.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a driver's state detecting device for detecting whether the driver's eyes are open or closed to detect dozing.

[0002]

2. Description of the Related Art Conventionally, as a device capable of detecting the condition of a vehicle driver, there is one disclosed in, for example, Japanese Patent Application Laid-Open No. 62-247410. This is to binarize the captured image data, recognize the position of the driver's eyes as a region based on the continuity of white pixels and black pixels, determine the open / closed eyes in the region, and detect the driver's state. It is configured so that it can be used to detect the driver's dozing.

[0003]

However, in the above-described vehicle driver state detection device, since the eye area is set only by the continuity of the white pixels and the black pixels in the binarized image data, the glasses are used. When targeting the person wearing the glasses, the frames of glasses etc. remain as noise in the binarized data,
There is a problem that the eye area cannot be set accurately.

The present invention has been made in view of the above problems, and it is an object of the present invention to provide a driver's state detection device capable of further improving the detection performance of the driver's eye region.

[0005]

In order to achieve the above object, the driver state detecting device of the first aspect of the present invention includes image input means for inputting a driver's face image as face image data, and driver's face image. Light emitting means for irradiating the face portion with invisible light rays, eyeglasses presence / absence determining means for determining presence / absence of eyeglasses by image recognition of reflection points of the light emitting means reflected in the eyeglasses in the grayscale image sent from the image input means, Eye presence area setting means for determining an area including eyes, open / closed eye detection means for detecting the open / closed eyes of the driver within the area, and judgment of the driver's state by the open / closed eye signal from the open / closed eye detection means And a driver state determining means for performing the operation.

According to a second aspect of the present invention, in the driver state detecting apparatus according to the first aspect of the present invention, when the eyeglasses presence / absence determining means determines that the eyeglasses are present, the eye presence area setting means wears the eyeglasses of the driver. It is characterized in that the presence region of the eye is set with reference to the reflection point of the light emitting means reflected in the lens.

According to a third aspect of the present invention, in the driver state detecting apparatus according to the first aspect of the present invention, when the eyeglasses presence / absence determining means determines that no glasses are present, the eye presence area setting means is sent from the image inputting means. Binarizing means for binarizing the input image, white pixel number counting means for scanning the left and right directions of the image in the binarizing means to count the number of continuous white pixels, and the face image based on the change amount of the pixel number. Stable portion detecting means for detecting a stable portion at the end in the width direction, contour line continuity determining means for determining the continuity of the contour line of the face image based on the presence or absence of the stable portion and the amount of change in the number of continuous pixels, and based on the determination Face edge specifying means for specifying the edge of the face, left and right eye area determining means for determining the lateral position of the eye existing area from the edge of the face specified by the face edge specifying means, and the determined eye Area in the horizontal direction of the A black area detecting means for detecting, and a vertical area determining means for the eyeball that determines the vertical position of the eye existing area based on the detected black area, and the eye existing area is set by To do.

[0008]

With the above construction, the first aspect of the present invention inputs the face image of the driver as face image data by the image input means, irradiates the driver's face portion with an invisible light beam by the light emitting means, and the glasses presence / absence determining means makes an image. The presence or absence of eyeglasses is determined by image recognition of the reflection points of the light emitting means reflected on the eyeglasses in the grayscale image sent from the input means. Then, the area including the eyes of the face image data is determined by the eye presence area setting means, the opening / closing eye detecting means detects the opening / closing eyes of the driver in the area, and outputs the opening / closing eye signal to determine the driver state. Means determines the driver's condition from the open / close eye signal.

In the second aspect of the invention, in the first aspect of the invention, when the eyeglasses presence / absence determining means determines that the eyeglasses are present, the eye presence area setting means is reflected on the lens of the eyeglasses worn by the driver. The presence region of the eye is set based on the reflection point of the light emitting means.

According to a third aspect of the present invention, in the first aspect of the present invention, when the eyeglasses presence / absence determining means determines that the eyeglasses are not present, the eye presence area setting means is sent from the image inputting means by the binarizing means. The input image is binarized, the white pixel number counting means scans the image in the left-right direction in the binarizing means to count the number of continuous white pixels, and the stable part detecting means calculates the width of the face image from the change amount of the pixel number. Detecting the stable part at the edge of the direction, the continuity of the contour is judged by the continuity judgment means of the contour line, and the continuity of the contour line of the face image is judged from the amount of change in the number of continuous pixels. The edge of the face is specified based on the judgment. Further, the lateral position of the eye existing region is determined from the edge of the face identified by the edge identifying unit of the face by the left and right region determining unit of the eyeball, and the lateral region of the eye determined by the black region detecting unit. The black area is detected from below in the vertical direction, and the eye existing area is set by determining the vertical position of the eye existing area with reference to the black area detected by the eyeball vertical area determining means.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a block diagram of an embodiment of a vehicle driver state detecting device of the present invention, which is an example applied to an automobile.
FIG. 2 is a flowchart showing the operation of the vehicle driver state detection device based on the configuration of FIG.

In FIG. 1, the state detecting device is an infrared lamp 1,
2 (infrared lamp A, B), camera for image input 3, A /
It has a D converter 4, an image memory 5, a presence / absence determination circuit 6 for eyeglasses, an eyeball existing position defining circuit 7, an open / closed eye determination circuit 8, and a drowsiness determination circuit 9.

In FIG. 1, the image input camera 3 is installed in front of the driver in the instrument panel in order to photograph the face of the driver.

When the driver wears the infrared lamp A at the position of the image input camera 3, the reflected image of the infrared lamp A itself reflected on the lens of the eyeglass is taken by the image input camera 3. It is installed in such a position (hereinafter referred to as “imprint”). In this case, infrared lamp A
At the position of, the incident light of the image of the infrared lamp A is the lens S of the glasses.
It is provided in consideration of the reflection angle / incident angle so that the light is reflected by and enters the image input camera 3.

On the other hand, the infrared lamp B is provided at a position where it does not appear in the eyeglasses, and in this embodiment, an infrared lamp for night illumination is used as the infrared lamp B.

In the present embodiment, the input image of the image input camera 3 is composed of 520 pixels in the horizontal (X) direction and 500 pixels in the vertical (Y) direction as shown in FIG. 3 (a window showing an area photographed by the camera 3). The angle of view is adjusted so that the face part is almost full in the vertical direction.

The image memory 5 is connected to the image input camera 3 via the A / D converter image 4, and inputs and stores the input image data obtained by the image input camera 3. In addition, A
The / D converter 4 converts the input image captured by the image input camera 3 into a digital amount and supplies it to the image memory 5.

The spectacle presence / absence determining circuit 6 is connected to the image memory 5 and determines the presence or absence of a high brightness point of the infrared lamp A reflected on the lens portion of the spectacles based on the input image data stored in the image memory 5. .

The eyeball existence position defining circuit 7 is connected to the eyeglasses presence / absence determining circuit 6 and defines the eyeballs presence position area in accordance with the determination result of the presence / absence of glasses by the eyeglasses presence / absence determining circuit 6.

The open / closed eye determining circuit 8 is connected to the eyeball existing position defining circuit 7 and processes the image data of the image memory 5 in the area defined by the eyeball existing position defining circuit 7 to judge the open / closed eye.

The drowsiness determination circuit 9 is connected to the open / closed eye determination circuit 8 and determines from the determination result of the open / closed eye determination circuit 8 whether or not the driver is asleep.

Next, the operation of the entire driver state detection device will be described with reference to the flowchart of FIG.

First, in step S21, the seating of the driver is confirmed by a seat sensor or the like. When it is confirmed that the driver is seated, the process proceeds to step S22, and the infrared lamp A that starts the timer and causes the reflection of the glasses is turned on.

After that, the process proceeds to step S23, and the presence / absence determination circuit 6 of the eyeglasses determines whether or not there is a high brightness point of the infrared lamp A reflected on the lens portion of the eyeglasses. If there is a high-intensity point, the process moves to step S28, and the eye presence position defining circuit 7 sets the eye presence region by the eye presence region setting method B (described later).

On the other hand, if there is no high-brightness point, the process proceeds to step S24, and the search for a high-brightness point by the eyeglasses presence / absence determination circuit 6 is repeated for a predetermined time set in advance.

In this case, if there is no high-intensity point even after the lapse of a predetermined time, the process proceeds to step S25, the timer is stopped, and the timer counter is cleared. After that, the process proceeds to step S26, and the eyeball existing position defining circuit 7
The eye area is set by the method A for setting the eye existing area (described later).

The processing from step S26 and step S28 is performed on the basis of the binarized image obtained by binarizing the grayscale image which is the current image with a certain threshold value.

After the presence region of the eyes is set in steps S26 and S28, the process proceeds to steps S27 and S29, and the infrared lamp A for judging the presence / absence of glasses is turned off. After this, step S2
In step 10, the open / closed eye determination circuit 8 determines whether the eye is open / closed in the eye presence region set in steps S26 and S28. Based on the time-series determination result of the open / closed eyes, the dozing determination circuit 9 performs the dozing determination in step S211.

Details of the processing for setting A of the eye presence region in step S26 will be described with reference to FIGS. Details of the processing of setting B of the eye presence region in step S28 will be described with reference to FIG.
This will be described with reference to FIG. Details of the open / closed eye determination processing in step S210 will be described with reference to FIGS.

FIG. 4 is a flow chart for determining the horizontal position of each of the left and right windows related to the setting A of the eye existence area in step S26 of FIG.

In steps S101 and S102 of this figure, as shown in FIG. 7, the center of the image (X coordinate = 250)
The search start line is set to, and the white continuous pixel numbers WXCL and WXCR are separately counted from the left and right, and the left and right X coordinates XLM and XR when the white continuous pixel number is maximum.
Remember M. This processing will be described later with reference to FIG.

Then, in step S103, it is checked whether or not the sum of the numbers of continuous pixels on the left and right is larger than 200, and the hair,
If the number of eyebrows, eyeballs, etc. is 200 or less, step S104
Left stability counter WLCON, right stability counter WRCO
N, left stability flag STFLGL, right stability flag STFL
GR is cleared and the process returns to step S101 to scan the next line.

If the sum of the numbers of continuous pixels on the left and right is larger than 200 in step S103, it is checked in step S105 whether or not the left end point detection flag OKFGL is set. If it is set, the right end search after step S120 is performed. Move to processing.

If the left end point detection flag OKFCL is not set (OKFCL = 0), step S10
If the difference between the white continuous pixel number WXCL and the continuous number MAEL in the immediately preceding scanning line is less than 10 in 6 and S107, the right stability counter WCON is incremented.

In steps S108 and S109, since the current scanning line is a stable candidate portion having a small difference from the number of consecutive immediately preceding scanning lines, XL when the number of consecutive pixels is maximum is X1, and when it is minimum. Is stored as X2.

In steps S110 to S112, it is checked whether or not the left stability counter WLCON exceeds 10 and the difference between X1 and X2 is less than 30, and if this condition is satisfied, the left stability flag STFGL is set and the left stability flag STFGL is set. White continuous pixel count W
Make XCL a new MAEL.

In step S106, the number of white continuous pixels WXC
If the difference between L and the number of consecutive MAELs in the previous scan is 10 or more, it is determined that the continuity of the contour line may have been lost, and the process proceeds to steps S113 to S119.

In this portion, it is determined whether the number of continuous pixels has greatly changed due to eyebrows or the like, or whether the contour line is interrupted, based on the change in the number of continuous pixels and the existence of a stable portion.

First, in step S113, the left stability flag S
If the stable portion exists and STFLGL is set before checking the TFLGL and the continuous white pixel number greatly changes, it is checked in step S114 whether the continuous pixel number has increased.

When the number of continuous pixels is increasing, it is determined that the contour line is interrupted, and the X coordinate X1 stored in step S108 stored in step S108 is set as the left edge of the face.

If there is no stable portion before the number of continuous white pixels changes significantly, or if the number of continuous pixels decreases even if there is a stable portion, the portion from the portion where the contour line is interrupted to the portion where the contour line is present It is conceivable that the scanning has been shifted to, or the scanning of the eyebrows, eyes, spectacles, or the like.

Therefore, in step S116, when the number of white continuous pixels of the current scanning is smaller than the number of white continuous pixels of the previous time by 50 or more, it is determined that the portion where the contour line is interrupted is shifted to the scanning of the portion where the contour line is present. Then, the leftmost point X of this scan
Let L be the left edge of the face.

When the left edge of the face is set in step S115 or step S118 as described above, step S11
After the left end point detection flag OKGLG is set at 9, the process proceeds to step S120.

If the number of consecutive times of this time is not much smaller than the number of consecutive times of last time, it is considered to be an eyebrow, an eye, a spectacle portion, or a portion having a large shadow, so that X1 and X in step S117.
Clear 2, WLCON, STFLGL and step S
Proceed to 112.

In steps S120 and S121, similarly to the above steps S105 to S109, the contour line break determination on the right side of the face and the right end setting at the time of the contour line break are performed.

The above processing is performed in steps S122 and S12.
As shown in FIG. 3, both the right and left edge detection flags of the face are set, or otherwise the process is continued until the preset search range in the Y direction ends.

When both end point detection flags are set, the width setting is completed. Further, when the search range in the Y (vertical) direction is finished, steps S124- shown in FIG.
The process moves to S127.

Here, it is checked whether or not the left and right end point detection flags are set, and if they are not set, the left and right when the number of continuous white pixels on the left and right is the maximum among all the scans stored in step S102. Edge X coordinate XLM, XR
Let M be the left and right edges of the face.

If the left and right edges of the face can be detected as described above, the horizontal position of the window can be determined according to the following equation.

X-axis center = Xc = XLM + ((XR
M-XLM) / 2) -Left X coordinate of left eye window = X1 = XLM-Right X coordinate of left eye window = X2 = Xc-25-Left X coordinate of right eye window = XX1 = Xc + 25-Right eye window Right X coordinate = XX2 = XRM As described above, even if the contour line of the face image loses continuity due to external light shining from one side of the face, the presence / absence of a stable portion at the end in the width direction of the face and the number of continuous pixels The left and right edges of the face can be specified by judging the continuity of the contour line from the change.

Further, in the binarized image, the left and right edges of the face are detected by scanning from the center of the image, that is, from the approximate center of the face to the left and right, so that the face is always detected even if the background is not white. Therefore, it is possible to perform accurate detection.
In addition, the left and right edges of the face can be detected by the cheeks and the like without being affected by the hairstyle and glasses. Furthermore, since the left and right ends of the face are independently detected, the opposite end can be detected even if one end of the face cannot be detected due to the influence of outside light.

The process of storing the left and right edges XLM and XRM when the number of continuous white pixels is maximum in steps S101 and S102 will be described below with reference to the flowchart shown in FIG.

First, in step S201, the vertical scanning Y
The coordinates are 40. This Y = 40 is the processing speeded up on the assumption that the maximum value of the face does not exist within this range. In step S202, the X coordinate value 250 of the horizontal search scan start line is set to the left search X coordinate XL and the right search X coordinate XR.
Set to. As shown in FIG. 7, if the vehicle driver is within the angle of view of the camera, this X coordinate value surely specifies a line that exists in the face portion.

Next, in step S203, it is checked whether or not the right scanning end flag OKR is set, and if it is set, the face left end search in step S211 and thereafter is performed. If ORK is not set, step S20.
In step 4, it is checked whether or not the pixel J (XR, Y) is white. If it is white, in step S205, the right white pixel continuous counter WXCR
In step S206, the search X coordinate XR is incremented.

In step S204, the pixel J (XR,
If Y) is not white, OKR is set in step S207, and then the maximum value XRM of the right end point that has been stored so far is compared with the end point XR of this time in step S208 to determine XR.
Is larger (when it is on the right side), step S
At 209, XR is set as a new end point XRM.

Next, in steps S211 to S217, the same processing as above is performed on the left side. The difference from the search on the right side is that the search X coordinate XL is counted down in step S214, and the smaller of the left end point XLM and the end point XL of this time stored in steps S216 and S217 (the one on the left side). The left end point is XLM.

If the left and right ends of one scanning line are detected and it is determined in step S221 that both scanning end flags OKL and OKR have been set, these flags are cleared (= 0) in step S222, and step S223 is performed. Then, the search line Y is counted up and this process is completed.

FIG. 6 detects the Y-direction coordinates of the area where the eyes are present, which is performed for each of the left and right eyes.

With respect to the left eye, as shown in FIG. 8, 10 dots from the right side X coordinate X2 of the left eye window, that is, X2-10 is used as the starting point (this is to avoid detection of the black part of the nose). X from the position in the lateral direction (X direction)
The range is from 2 to 90, and 0 from Y coordinate YL of the search start
The search is performed upward in the vertical direction (Y direction) within this range, and this is performed in the horizontal direction at intervals of 4 dots. The Y coordinate YL is set below the scan line whose left and right ends are determined.

With respect to the right eye, 10 dots to the right of the left X coordinate XX1 of the right eye window, that is, XX1 + 10 is set as the starting point, and from this position in the lateral direction (X direction), XX1 + 9.
The range from 0 to Y is searched, and the Y coordinate YL is searched in the upward direction in the vertical direction (Y direction), and this is performed in the horizontal direction at intervals of 4 dots.

In FIG. 6, first, in step S61, the memory variable BY1M of the maximum value (bottom point) of the Y coordinate of the black area is set.
AX is cleared, the detection range regulation counter XC in the X direction
HECK is initialized to X2-10, and Y-direction search range counter YCHECK is initialized to YL.

Next, in step S62, it is determined whether or not the search range defining counter XCHECK in the X direction is X2-90 or less. This determination is to determine whether or not all the searches in the X direction have been performed. If the search in all the X directions has not been completed, the process proceeds to step S63.
Move to 74.

A step S63 decides whether or not all the Y directions on the X axis have been searched. If the search in all the Y directions has not been completed, the process proceeds to step S64. If it has been completed, the process proceeds to step S73.

In step S64, the maximum Y in which a black area appears
The purpose is to detect the coordinates, and it is determined whether or not the pixel that started the search is a black pixel. If it is not a black pixel, the process proceeds to step S72, the black pixel counter BLACK is cleared, and the process proceeds to step S71. When the Y coordinate is counted down by one and the determination in step S63 is obtained and the search in all Y directions has not been completed, the next pixel is determined in step S64.

This process is repeated from the start of the search until the black pixel appears. When it is determined in step S64 that the pixel is a black pixel, the process proceeds to step S65 and the black pixel counter BLACK is incremented.

Next, in step S66, it is determined whether the black pixel appears for the first time. Judgment is counter B
It is assumed that the counter appears for the first time if the counter BLACK = 1.

If the black pixel appears for the first time, step S6
7, the Y coordinate is assigned to SETY, and step S6 is performed.
Go to 8. If the black pixel does not appear for the first time in step S66, the process proceeds to step S68.

In step S68, the black pixel counter BL
It is determined whether or not the value of ACK exceeds 5. This process takes into consideration that when the number of consecutive black pixels is 5 or less, it is unlikely that the pixel is a black pixel of the eye. When the number of consecutive black pixels is less than 6, the process proceeds to step S72, the black pixel counter BLACK is cleared, and then step S71.
And the Y coordinate is counted down by one, and the search in the Y direction on the X axis is continued (that is, step S63).
~ S68 is repeated).

If there are five or more continuous black pixels in step S68, the process proceeds to step S69. Step S69 is a process for updating the maximum value Y1MAX of the Y coordinate where the black pixel appears for each X axis, and the Y coordinate SETY where the black pixel area appears on the X axis is the maximum value BY1M of the Y coordinate so far.
It is checked whether or not it exceeds AX. If SETY exceeds BY1MAX, the process proceeds to step S70, BY1MAX is updated, the process proceeds to step S73, and the same processing (steps S62 to 69) is repeated for the next X axis. SETY is BY1M
If it does not exceed AX, the process proceeds to step S73 without updating BY1MAX.

When it is confirmed in step 62 that all X-axes have been searched, the process proceeds to step S74, and the maximum value YT and the minimum value YB on the Y-coordinate of the eye existence region are set.

FIGS. 9 and 10 are explanatory views showing a method of determining the eye presence area relating to the setting B (see FIG. 2) of the eye presence area in step S28, that is, when the driver wears glasses. Is.

FIG. 9 shows a case where a high-brightness reflection point of an infrared lamp is present on the lens of the glasses worn by the driver. The high-brightness reflection point can be freely set according to the arrangement and number of the infrared lamps. Since it can be set, it is possible to distinguish even if there is a reflection of another high brightness point.
Further, since it is reflected in the lens portion of the eyeglasses, the presence region of the eye can be determined by using the high brightness point as a reference. FIG. 10 shows an example of that,
The specularly reflected light on the corneal surface of the eyeball illuminated by the infrared lamp is captured by the camera 3 and observed as a bright spot (corneal reflection image) 18, and the infrared lamp height is increased by the lens of the glasses worn by the driver. The brightness reflection point 19 is shown.

As described above, according to this embodiment, it is possible to accurately cut out the area where the eyes are present regardless of whether or not the driver wears glasses, and it is possible to detect the drowsiness with higher accuracy.

Further, assuming that the sunlight is directly on the face, it is possible to make the area of the eye shade by linking a sun visor or the like with the lighting of the infrared lamp for presence / absence determination of glasses.

FIG. 11 is a flow chart for explaining an example of determining whether the eye is open or closed by the coefficient of the number of continuous black pixels in the eyeball portion in the area where the eye is present.

This processing is to detect the coordinates in the Y direction of the window, and is performed for each of the left and right eyes. First, the coefficient processing of the number of continuous black pixels will be described with reference to FIG. 12, taking the case of the left eye as an example.

In FIG. 12, the X coordinate of the right X coordinate of the automatically searched left eye window is set as the starting point, and the Y coordinate Y of the search start is set in the range from this position to X1 in the lateral direction (X direction).
YB-50 is searched from B upward in the vertical direction (Y direction), and this is performed at intervals of 1 dot in the horizontal direction.

With respect to the right eye, starting from XX1 of the left X coordinate of the window of the right eye, the range from this position to XX2 in the lateral direction (X direction), and the Y coordinate of the search start is the window coordinate of the right eye. From YB 'obtained in the setting process to YB'-50 in the vertical direction upward (Y direction),
This is performed at intervals of 1 dot in the horizontal direction.

In FIG. 11, first, in step S401, the X-direction detection range defining counter XECHECK is set to X.
2 and the specified counter YECH for the detection range in the Y direction
ECK is initialized to YB and the number of continuous black pixels is cleared.

Next, in step S402, it is determined whether or not the X-direction search range defining counter XECHECK is equal to or less than YB-50. This determination is to determine whether or not all the search has been performed in the X direction. If all search has been performed in the X direction, the processing ends.

If it is determined in step S402 that all search has not been performed in the X direction, the process proceeds to step S403.

Then, in step S403, the flag E which has detected the number of continuous black pixels when the new X-axis is searched.
BFL, the flag EWFL for detecting the number of continuous white pixels, the black pixel continuous counter BLACK, and the white pixel continuous counter WHITEE are cleared, and the process proceeds to step S421.

In step S421, it is determined whether the Y-axis search initial value is a white pixel. If the search initial value is not a white pixel, that is, if it is a black pixel, it is considered that hair or the like is in the window, the search in the Y direction on the X axis is stopped, and the process proceeds to step S418 to move to the next X axis. Move on to search. If the Y-axis search initial value is a white pixel, step S4
Move to 22.

In step S422, it is determined whether or not all the Y-axis directions on the X-axis have been searched. Even when the search in the Y-axis direction has been completed, the process proceeds to step S418 and the search for the next X-axis is performed.

With respect to the flow charts after step S404, a specific example of the search in the Y-axis direction will be described separately for the scanning lines (a) to (f) in FIG.

(1) Pattern (a) In step S404, it is determined whether or not the flag EWFL of continuous white pixels is set (set if EWFL = 1). If the flag EWFL is set, step S41
However, in the pattern (a), the flag EWFL is set.
Is not set, it moves to step S405 and it is determined whether the corresponding dot is a black pixel or a white pixel. Figure 1
As is clear from 2, since the pixel is a white pixel in the case of the pattern (a), the process proceeds to step S412.

Whether or not the flag EBFL of continuous black pixels is set in step S412 (set if EBFL = 1)
To judge. In pattern (a), since the flag EBFL is not set, the process proceeds to step S416, and the count of the white pixel counter WWHITE is started.

Next, in step S417, it is determined whether or not the number of continuous white pixels exceeds 30, and if the number is 30 or less, 3 is determined.
Until it exceeds 0, the process proceeds to step S419, the Y coordinate is counted down, and the same procedure is repeated from step S422 to count up the number of white pixels.

When the number of white pixels exceeds 30 in step 417, the process proceeds to step 418 even if the Y coordinate search range is not satisfied, and the process proceeds to the next X axis search (step S4).
The reason why the maximum number of consecutive white pixels is 30 in 17 is that black pixels appearing after 30 or more white pixels continue have a low probability of being an eye and have a high probability of eyebrows, hair, etc. This is to prevent counting the black pixels that represent).

No black pixel appears in the search on the X-axis in the case of the pattern (a).
Since it does not move to 420, the value of the maximum black pixel number BLACKEMAX used for the judgment of the open / closed eye is 0,
The numbers are never updated.

(2) In the case of pattern (b): Return to step S418, move to search for a new X axis, and flag EBF for detecting the number of continuous black pixels in step S403.
L, a flag EWFL that detects the number of continuous white pixels, a black pixel continuous counter BLACK, a white pixel continuous counter W
After clearing HITEE, the process moves to step S404.

In step S404, since the flag EBFL and the flag EWFL have been cleared in step S403, the steps S405 and S405 are the same as in the case of the pattern (a).
S412, S416, S417, S420, S419,
From S404 to S405, the Y coordinate is counted down until a black pixel appears, and the white pixel continuous counter WHITE
Count up E.

In the case of the pattern (b), black pixels appear before at least 30 white pixels. Therefore, when black pixels appear, the process moves to step S406, the continuous black pixel counter BLACK is started, and step S
Move to 408.

Step S408 is to remove noise from black pixels of 2 dots or less. If black pixels appear continuously with 2 dots or less, step S41 is performed.
The process proceeds from 2, S416, S417 to S420, clears the black pixel continuous counter BLACK, which may have been counted as noise, and continues the search.

When three or more black pixels are continuous in step S408, the process proceeds to step S409 and the flag EBFK is set. As long as black pixels continue after the flag EBFK is set, steps S419, S404 to S404
409 is repeated to count up the black pixels. Incidentally. Here, if the continuity of black pixels is interrupted, the process proceeds to step S412.

In step S412, it is determined whether the flag EBFL is set. In the case of pattern (b), EBFL has already been set, so step S
At 413, the white pixel continuous counter WWHITE starts counting.

Next, in step S414, the number of consecutive white pixels is checked. If the continuous number is less than 5, step S
419, S404, S405, S412, S413 to S414 are repeated to count down the Y coordinate and count up the white pixels.

If the number of consecutive white pixels is 5 or more in step S414, the flag EWFL is detected in step S415.
Set. After setting the flag EWFL, step S
In step S419, the Y coordinate is similarly counted down, and after step S422, the process proceeds to step S404.

In the judgment of step S404, the flag EWF is set.
Since L is set, the process proceeds to step S410, and in step S410, the continuous black pixel number BLACK and the maximum continuous black pixel number BLAC counted in step S407.
Compare with KEMAX.

As a result of the comparison, the continuous black pixel number BLACK counted on the X axis is the maximum continuous black pixel number BLA.
If there is more than CKEMAX, BLA in step S411
CKE is replaced by BLACKEMAX, and the process proceeds to step S418.

In step S410, the continuous black pixel number BLACKE is the maximum continuous black pixel number BLACKEMAX.
If the number is smaller, the next X-axis is searched as it is (step S
418).

In the flow of the pattern (b) on the X-axis, the continuity of white pixels from the initial value on the Y-axis is confirmed, and then the continuity of black pixels corresponding to the eye part is judged to count the number of black pixels. . After that, the size of the eye in the vertical direction is recognized by the continuous black pixels by determining the white pixels on the eyes.

(3) In case of pattern (c) In pattern (c), the iris part of the eyeball becomes white when the eye part is binarized (at night, in an image of infrared light only in a tunnel, etc. Which is often the case).

In the case of pattern (c), the basic processing is the same as in the case of pattern (b). In pattern (C), the number of consecutive pixels is confirmed, and if white pixels in the iris part of the eye appear while counting up black pixels, the black pixels are counted even if the black pixels in the eye part are still present. Then, the count is started for white pixels in the iris part.

Here, in step S414, the number of white pixels is 5
If black pixels appear again before the number of black pixels is exceeded, the flag EWFL has not been set, and thus the counting up of the black pixels that have been stopped is restarted. After that, when the white pixel on the eye is confirmed, the same processing as the pattern (b) is performed.

(4) In the case of pattern (d) In the case of pattern (d), it is exactly the same as in the case of pattern (b), but if the value of black pixel number BLACKE counted up in step S407 is larger, After confirming the continuous white pixels on the above, the maximum continuous black pixel number BLACKEMAX is set to the black pixel number B in steps S410 and S411.
Replace with LACKE and store the maximum number of consecutive black pixels in that window.

(5) In the case of pattern (e) The pattern (e) is a case where, for example, continuous black pixels other than the eyes, such as eyebrows and hair, are counted. Since black pixels such as eyebrows and hair do not exist under the eyes at least, Y
If the number of continuous white pixels exceeds 30 in step S417 by repeating the search on the axis and counting up the white pixels (the black pixels formed thereafter are not considered to belong to the eye), the X axis Stop the search in and move to the next X-axis search.

(6) In the case of pattern (f) The pattern (to) is a case where the continuous black pixels of the hair at the left end of the window are counted. The number of continuous black pixels of the hair on the left edge of the window may be very large, but no white pixel appears on the continuous black pixel on the left edge of the window, and the flag EWFL is not set. Even if the number of pixels increases, the maximum continuous black pixel number BLACKEMAX is not replaced.

Therefore, when all the X-axis searches are completed, the maximum number of continuous black pixels in the vertical direction of the eye (Y-axis direction) can be obtained. These processes can be detected regardless of whether the eye is open or closed, and the open / closed eye can be recognized by the maximum number of consecutive black pixels.

Although one embodiment of the present invention has been described above, it is needless to say that the present invention is not limited to the above embodiment and various modifications can be made.

[0111]

As described above, according to the first aspect of the invention, whether or not the driver is wearing glasses is determined in advance by the reflection point of the infrared lamp, and even if the driver is wearing glasses. Since the configuration is such that the eye presence region can be determined, it is possible to accurately detect the eye presence region whether or not the driver wears the glasses.

Further, according to the second and third aspects of the invention, the method for determining the area where the eye is present is configured separately for the case where the driver wears the glasses and the case where the area where the eye is present is further divided. It has become possible to detect accurately.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an embodiment of a vehicle driver state detection device of the present invention.

FIG. 2 is a flowchart showing an operation of the vehicle driver state detection device of FIG.

FIG. 3 is an explanatory diagram of a window showing a region photographed by a camera.

FIG. 4 is a flowchart for determining the horizontal position of each of the left and right windows related to the setting A of the existing area.

FIG. 5 is a detailed flowchart of a part of the processing shown in FIG.

FIG. 6 is a flowchart of a window vertical range setting process.

FIG. 7 is an explanatory diagram of vertical range setting processing of a window.

FIG. 8 is an explanatory diagram of determining coordinates in the Y-axis direction of an eye existence region.

FIG. 9 is an explanatory diagram showing a method of determining an eye existence region when a driver wears glasses.

FIG. 10 is an explanatory diagram showing a method of determining an eye existence region when a driver wears glasses.

FIG. 11 is a flowchart of open / closed eye detection processing.

FIG. 12 is an explanatory diagram of the flowchart of FIG. 11.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Infrared lamp A (light emitting means) 2 Infrared lamp B 3 Camera (image input means) 4 A / D converter (binarization means) 5 Image memory 6 Glasses presence / absence determination circuit (glasses presence / absence determination means) 7 Eyeball existence position Regulation circuit (eye presence region setting means) 8 Open / closed eye determination circuit 9 Doze determination circuit

Claims (3)

[Claims] 1. An image input means for inputting a driver's face image as face image data, a light emitting means for irradiating a driver's face portion with an invisible light beam, and a grayscale image sent from the image input means for wearing on eyeglasses. Eyeglasses presence / absence determining means for determining presence / absence of glasses by image recognition of reflection points of light emitting means, eye presence area setting means for determining an area including eyes of the face image data, and opening / closing of a driver in the area A driver's state detection device comprising: an open / closed eye detecting means for detecting an eye; and a driver state determining means for determining a driver's state based on an open / closed eye signal from the open / closed eye detecting means. 2. When the spectacles presence / absence determining unit determines that spectacles are present, the eye presence region setting unit sets the eye based on the reflection point of the light emitting unit reflected in the lens of the spectacles worn by the driver. The driver's state detection device according to claim 1, wherein the presence region is set. 3. When the spectacles presence / absence determining unit determines that there is no spectacles, the eye presence region setting unit binarizes the input image sent from the image input unit 2
A binarizing unit, a white pixel number counting unit that scans the number of consecutive white pixels by scanning in the left-right direction of the image in the binarizing unit, and a stabilizing unit at the widthwise end of the face image based on the change amount of the pixel number. Stable portion detecting means for detecting, contour line continuity determining means for determining the continuity of the contour line of the face image from the presence or absence of the stable portion and the amount of change in the number of continuous pixels, and the edge of the face based on the determination. Face edge specifying means for specifying, lateral eye area determining means for determining the lateral position of the existing area of the eye from the edge of the face specified by the face edge specifying means, and the side of the determined eye A black area detection unit that detects a black area from below in the vertical direction in the existence area of the direction, and a vertical area determination unit of the eyeball that determines the vertical position of the existence area of the eye based on the detected black area, A contract that is characterized by setting the area where the eye is present by The driver of the state detecting device of claim 1, wherein.