JP3858534B2 - Eye state detection device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an eye state detection device for detecting and alerting to a dozing state or a side-viewing state of a driver of a vehicle, a ship operator, an operator of a plant or the like.
[0002]
[Prior art]
An example of this type of apparatus using conventional image processing is described in Japanese Patent Laid-Open No. 6-251273. This detects the position of the eyes from the driver's face image, and then tracks the position of the eyes to determine the state of the driver.
[0003]
[Problems to be solved by the invention]
However, in the above-described conventional device, when the driver turns around or looks at the door mirror, the face moves greatly and the eye cannot be tracked, the eye position is detected again from the face image, There is a limit to the judgment in situations where it is necessary to track the situation and a judgment is required in a short time such as a driver's side-viewing condition.
[0004]
An object of the present invention is to provide an eye state detection device that can perform a fast process even when a state in which the eye cannot be tracked completely occurs.
[0005]
[Means for Solving the Problems]
  According to a first aspect of the present invention, there is provided an image output means for outputting occupant face image data, a feature amount detection means for detecting a face feature amount by processing the face image data output from the image output means, and the detection First eye position detecting means for detecting the position of the eye based on the detected facial feature amount, and a predetermined region including the detected eye position is determined for each of the left and right eyes one by one. Area determining means for tracking the eye according to movement of the eye in the predetermined area; second eye position detecting means for detecting positions of left and right eyes in the predetermined area; With eye position detection meansWhen the detected relative position of the left and right eyes exceeds the set range,One of the left and right eyesAs eyesNo longer detectedAs, The size that is defined as the smallest size that can simultaneously include the left and right eyes when the vehicle vibratesThe predetermined areaAn area expanding means for expanding and a third eye position detecting means for detecting the positions of the left and right eyes in the expanded predetermined area are provided.
[0006]
A second aspect of the present invention is the eye state detection apparatus according to the first aspect, wherein the region expanding means expands a predetermined region including the position of the other eye being detected.
[0007]
A third aspect of the invention is the eye state detection device according to the first aspect, wherein the region expanding means detects the eye position detected by the first eye position detecting means or the second eye position detecting means. At least one of the positions is stored, and the predetermined area is expanded to a size including the stored eye position.
[0009]
  Claim4The invention of claim 1 to claim 13The eye state detection device according to any one of the above, wherein the area determination unit determines the left and right predetermined areas again when the relative positions of the left and right eyes are within a set range in the expanded predetermined area. It is characterized by doing.
[0010]
  Claim5The invention of claim4The eye state detection device according to claim 1, wherein the relative position is a left-right distance between left and right eyes.
[0011]
  Claim6The invention of claim4The eye state detection device according to claim 1, wherein the relative position is a vertical distance between left and right eyes.
[0012]
  Claim7The invention according to claim 3 is the eye state detection device according to claim 3, wherein the region expanding means has a set time within which the relative positions of the left and right eyes detected by the second eye position detecting means are within a set range. , Storing the positions of the left and right eyes, storing the positions of the left and right eyes detected by the first eye position detecting means when the relative position is not within a set range, and storing the stored eye positions The predetermined area is expanded from the above.
[0013]
【The invention's effect】
According to the first aspect of the present invention, the image data of the occupant output from the image output means is processed, the face feature quantity is detected by the feature quantity detection means, and the first eye based on the detected face feature quantity is detected. The position of the eye can be detected by the position detection means. The area determining means can determine one predetermined area including the detected eye position for each of the left and right eyes, and can track the eye according to the predetermined area according to the movement of the eyes. In each of the predetermined areas, the positions of the left and right eyes can be detected by the second eye position detecting means.
[0014]
  In such detection, when a situation occurs in which the occupant turns around or looks at the door mirror and the face is moved so much that the eyes cannot be tracked, one of the left and right eyes is detected by the second eye position detection means. Eyes are not detected, but the predetermined area isWhen the vehicle vibratesPossible to include left and right eyes simultaneouslyDefined as minimum sizeThe position of the left and right eyes can be detected by the third eye position detection means in the expanded predetermined area. Therefore, it is not necessary to perform the processing again from the processing of the image data, enabling a quick processing and sufficiently supporting a side-viewing operation.
[0015]
In the invention of claim 2, in addition to the effect of the invention of claim 1, when a predetermined area including the position of the other eye detected by the area expanding means can be expanded, and one eye disappears, By using the predetermined area of the other eye, a fast process can be performed.
[0016]
In the invention of claim 3, in addition to the effect of the invention of claim 1, at least one of the eye positions detected by the first eye position detecting means or the second eye position detecting means is stored by the area expanding means. The predetermined area can be expanded to a size including the stored eye position, and even when both eyes disappear, the stored predetermined area of the eye can be used to perform fast processing.
[0018]
  Claim4In the invention of claim 1,3In addition to the effect of any one of the inventions, the region determination means can determine the left and right predetermined regions again when the relative positions of the left and right eyes are within the set range within the expanded predetermined regions. Eye detection can be performed again in a range with less noise compared to the region, and eye detection can be performed quickly and accurately.
[0019]
  Claim5In the invention of claim4In addition to the effects of the invention described above, the relative position is set to the left-right distance between the left and right eyes, so that even if a non-eye is erroneously detected in the left-right direction of the occupant's face, the predetermined area can be expanded, and quick and accurate Eye detection can be performed.
[0020]
  Claim6In the invention of claim4 descriptionIn addition to the effect of the present invention, by setting the relative position to be the vertical distance between the left and right eyes, even if a non-eye is erroneously detected in the vertical direction of the occupant's face, the predetermined area is expanded, and a quick and accurate eye Can be detected.
[0021]
  Claim7In the invention of claim 3, in addition to the effect of the invention of claim 3, when the relative position of the left and right eyes detected by the second eye position detecting means is within a set range by the area expanding means, The positions of the eyes are stored, and the positions of the left and right eyes detected by the first eye position detecting means when the relative position is not within the set range are stored, and the predetermined area is determined from the stored positions of the eyes. The predetermined area can be expanded by a simple process, and the position of the eye can be reliably detected.
[0022]
DETAILED DESCRIPTION OF THE INVENTION
In addition to automobiles, this device can be used as a drowsy driving alarm device or an armpit driving alarm device for railway vehicles, ships, plant operators, etc., but in this embodiment, it is described when applied to an automobile as an armpit driving alarm device To do.
[0023]
FIG. 1 is a block diagram illustrating a configuration of an eye state detection apparatus to which the present invention is applied. The apparatus includes an image input unit CL1, a facial feature amount detection unit CL2, a first eye detection unit CL3, A region determining unit CL4, a second eye position detecting unit CL5, a region expanding unit CL6, and a third eye position detecting unit CL7 are provided.
[0024]
The image input means CL1 inputs an occupant's face image and outputs image data. The face feature quantity detection means CL2 detects the face feature quantity by processing the face image data output by the image input means CL1. The first eye position detecting means CL3 detects the position of the eye based on the detected facial feature amount. The region determining means CL4 determines one predetermined region including the detected eye position for each of the left and right eyes and tracks the eye according to the movement of the eye using the predetermined region. The second eye position detection means CL5 detects the positions of the left and right eyes in each of the predetermined areas.
[0025]
The area expanding means CL6 expands the predetermined area to a size capable of including left and right eyes simultaneously. The third eye position detection means CL7 detects the positions of the left and right eyes in the expanded predetermined area.
[0026]
From the detected eye position, for example, a side-by-side determination is performed using a reference value, and a warning such as a buzzer is issued if the side-by-side driving state is detected.
[0027]
FIG. 2 is a configuration block diagram according to an embodiment of the present invention.
[0028]
As shown in the figure, the TV camera 1 is installed in an automobile instrument and photographs the driver's face from the front. In the present embodiment, the input image of the TV camera 1 includes, for example, a horizontal direction (X) 512 pixels and a vertical direction (Y) 480 pixels.
[0029]
An input image captured by the TV camera 1 is stored in the image memory 3 as digital input image data via the A-D converter 2. The output of the image memory 3 is input to the image data calculation circuit 4.
[0030]
The image data calculation circuit 4 detects the density of the pixel column in the vertical direction of the face based on the input image data, extracts points based on the increase in density in the pixel column and its change state (extraction point), and adjacent Continuous data in the lateral direction of the face is extracted continuously at points close to the pixel column direction of the pixel column to be output and output as face image data.
[0031]
Therefore, the TV camera 1, the A / D converter 2, the image memory 3, and the image data calculation circuit 4 constitute the image output means CL1 in this embodiment. The output of the image data calculation circuit 24 is input to the face feature quantity detection circuit 5.
[0032]
The face feature quantity detection circuit 5 constitutes the face feature quantity detection means CL2, and specifies an eye from a plurality of extracted feature quantities of the face and obtains coordinates thereof. The output of the face feature quantity detection circuit 5 is input to the eye position detection circuit 6.
[0033]
The eye position detection circuit 6 detects an eye position by selecting an eye from the continuous data, and determines a predetermined region including the eye position from the eye position for each of the left and right eyes. In addition, the predetermined area is made to track the eye according to the movement of the eye, and in each predetermined area where the eye is tracked, a point is extracted from an increase in density in the vertical direction and a change state thereof. Continuous data in the horizontal direction of the face is extracted continuously from points close to each other in the pixel column direction, and the positions of the left and right eyes are detected. Accordingly, in the present embodiment, the eye position detection circuit 6 constitutes the first eye position detection means CL3, the region determination means CL4, and the second eye position detection means CL5.
[0034]
The output of the eye position detection circuit 6 is input to the area expansion circuit 7. The area expansion circuit 7 constitutes the area expansion means CL6. When one of the left and right eyes is not detected by the predetermined area for tracking the eyes, the predetermined area is included in the left and right eyes simultaneously. Expand to the size possible. The output of the area expansion circuit 7 is input to the eye position detection circuit 8. The eye position detection circuit 8 constitutes the third eye position detection means CL7, and detects the positions of the left and right eyes in the expanded predetermined region. The eye position detection circuits 6 and 8 may be configured as one circuit.
[0035]
Although not shown, the eye position data detected by the eye position detection circuits 6 and 8 is used to determine the state of the eye, to determine whether the occupant is asleep or to look aside, and to perform a doze or look aside. A warning is given by a buzzer etc.
[0036]
Next, a method for detecting the feature amount of the face from the entire face image and specifying the eye position will be described.
[0037]
FIG. 3 shows the occupant's face projected by the TV camera 1, wherein 9 is a screen frame and 11 is the occupant's face. Although it is possible to specify the eye feature amount from the entire face image, since the calculation time becomes long if the search area is wide, in this embodiment, the screen is divided into four areas to reduce the area, and the calculation time is reduced. The process is shortened to speed up processing.
[0038]
Reference numeral 13 denotes a search area for dividing the screen into four parts for specifying the eyes. If the eyes can be specified in (1) of FIG. 3, the initial detection of the eyes ends here. If no eye is found at position 1, (2), (3), and (4) are sequentially changed to the search area.
[0039]
Next, FIG. 4 is an explanatory diagram for detecting the facial feature quantity in the search area 13. In the detection area 13, for example, a pixel row for every 10 pixels is set in the vertical direction, and a point having a large density change is extracted for each pixel row. Further, the extracted points facing in the horizontal direction are grouped to be converted into data as feature amounts as indicated by reference numerals 15 and 17 in FIG. Then, 15 is identified as an eye by eye template matching. Here, template matching will be described with reference to FIGS.
[0040]
FIG. 5 is an explanatory diagram for examining the density value of the eye, and X arranged in the vertical direction in the region 13 of FIG._a, X_b, X_c, ..., X_nIs a pixel column for examining density values. This pixel row X_a, X_b, X_c, ..., X_nThe upper density values are shown in FIG. 6, FIG. 7, and FIG.
[0041]
FIG. 6 shows a pixel row X_aThe upper density value is shown. Where the density change is large (a₁, A₂). FIG. 7 shows a pixel row X_aAnd X_cPixel row X in the middle position of_bUpper concentration value and extraction point (b₁, B₂). FIG. 8 shows the right pixel row X_cUpper concentration value and extraction point (c₁, C₂).
[0042]
Whether or not it is an eye is determined by the pixel row X on both sides_a, X_cAnd center pixel row X_bDistance Y between extraction points_a, Y_b, Y_cconnection of
Y_a<Y_b> Y_c
If is established, it is determined as an eye.
[0043]
Both eyes are identified from the position of one eye thus identified. First, as shown in FIG. 9, a horizontally long search area 19 is set on the assumption that the other eye exists at a symmetrical position with reference to the coordinate value (x1, y1) of one eye. Similarly to the above, a point having a large density change is extracted for each pixel column at a position corresponding to the other eye, and the extracted point is grouped to be converted into data as a feature value 21. The coordinates of the other eye The value (x2, y2) is specified, and the position of both eyes is detected. Then, as shown in FIG. 10, windows 23 and 25 as predetermined areas are set in the coordinates of both eyes, and the movement of both eyes is tracked by the windows 23 and 25.
[0044]
(How to determine the size and position of the eye window)
Window size
The size of the window is determined as shown in FIGS. FIG. 11 shows the size of the window, and FIGS. 12 and 13 show the statistical data of the horizontal and vertical lengths obtained by examining the sizes of the eyes of several people. Here, the size of the window should be as small as possible in order to reduce noise (extract facial wrinkles, brightness and darkness) and not reduce the processing speed. The size used in the processing such as the current look-ahead detection is a size obtained by examining the size of several people's eyes and adding a margin (for example, x1.5 times). As a method for statistically obtaining the size of the eyes of several people, as shown in FIGS. 12 and 13, data on the vertical and horizontal dimensions of the eyes are collected, and the size covering, for example, 95% of the distribution is determined with a margin. A way to do this is considered. The dimension covering 95%, that is, the lateral dimension x_a, Vertical dimension y_aAs shown in FIG. 11, the margin is determined by looking at the margin (× 1.5). A method of estimating the eye width and height by image processing and adding a margin to the vertical and horizontal sizes is also conceivable.
[0045]
How to determine the position
FIG. 14 shows a method for positioning the right-eye window, for example.
[0046]
Based on the eye coordinate values (x2, y2), a reference point P for drawing a window is determined at positions of distances x3, y3, and predetermined window dimensions x4, y4 are drawn from the P point to determine the position. . x3 and y3 are ½ of x4 and y4, and have a length such that the window is in the center of the eye in advance.
[0047]
The above explains the process from identifying the position of both eyes from the face image to tracking both eyes.
FIG. 15 shows a flowchart of processing in the area expansion circuit 7 of FIG. First, in S1, the coordinate values of both eyes detected by the eye position detection circuit 6 are input. Next, in S2, it is determined whether or not the coordinate value of one eye has disappeared. In this case, if both eyes can be tracked, two coordinate values of both eyes are recognized, and if only one eye is tracked, only one coordinate value of the eye that has not disappeared is obtained. Whether the coordinate value of one eye has disappeared is determined based on the number of coordinate values. An example of the disappearance of one eye may occur when an operation is performed so that one eye disappears from the image, such as confirmation of a door mirror or safety confirmation at the rear.
[0048]
If it is determined in S2 that the coordinate value of one eye has disappeared, the process proceeds to S2 'to determine whether the coordinate value of both eyes has disappeared. If it is determined that the coordinate values of both eyes have disappeared, the process returns to S1. If the coordinate value of the eye that has not disappeared is confirmed, the predetermined area, that is, the window is expanded in S3.
[0049]
16 to 18 show the progress of the processed image. FIG. 16 shows the position of the face when looking at the right door mirror, for example. In this case, the binocular tracking windows 23 and 25 may overlap each other at a reduced distance, and when the line of sight returns from the door mirror in the forward direction, tracking of one eye cannot be performed as shown in FIG. In such a case, the window 27 is expanded as a predetermined area as shown in FIG. 18 to a size that allows the left and right eyes to be included simultaneously. In the expanded window 27, the positions of the left and right eyes are detected again. Here, how to determine the size and position of the search area will be described.
[0050]
(How to determine the size and position of the search area)
How to determine the size
FIG. 19 shows how to determine the size of the search area, that is, the window. The size of the window 27 is preferably a small area in order to reduce noise (extract facial wrinkles, brightness and darkness) and not to reduce the processing speed. However, if the eye frequently comes out of the area due to vertical and horizontal vibrations while traveling, the position of the eye cannot be determined. Therefore, the vertical direction is the distance moved by vertical vibration, and the horizontal direction is the distance obtained by adding a margin to the distance moved by left-right vibration to the distance of both eyes. For example, as shown in FIG. 19, the size x11 in the horizontal direction is a value obtained by multiplying a distance L0 (L0 = x1-x2) between both eyes by a predetermined constant K. The constant K includes a margin in the amount moved by the left and right vibration of the vehicle, and is set to K = 2.0, for example. The size of the vertical direction Y11 is determined, for example, by multiplying x11 by 0.3.
How to determine the position
In order to set the position of the window when one eye disappears, the position of both eyes has been detected in the past in FIG. 9 and is set based on the coordinates.
[0051]
  Here, when an example based on the coordinates of the right eye is shown, a reference point Q for drawing at a distance x10, y10 from the right eye coordinates (x2, y2) is created. From this Q point (x11, Y11) To set the area. This (x11, Y11) Is set so that the window 27 is positioned at the center of both eyes..Even if one eye disappears by looking at the door mirror in this way, and one eye cannot be traced as shown in FIGS. 16 and 17 when facing the front again, both eyes are immediately tracked by expanding the window. can do. Therefore, there is no need to start from the first image data capture, and even if one eye disappears once, tracking of both eyes can be resumed quickly, even if fast processing such as detection of a driver's side-by-side driving is required It can respond sufficiently.
[0052]
In the above embodiment, when one eye disappears, the window is expanded from the coordinate values of the eyes detected in the past. For example, only the position of the other detected eye is used as a reference. The window may be expanded.
[0053]
FIG. 21 is a flowchart of an embodiment according to a modification of the first embodiment, and corresponds to FIG. In this embodiment, S1 and S3 are the same as those in FIG. 15, and the predetermined area of S3 is expanded when the relative distance between both eyes is larger than the predetermined range in the determination of S4. . This is an example of processing when the window 23 mistakenly catches eyebrows and the relative position becomes abnormal as shown in FIG. 22, for example.
[0054]
In such a case, the binocular detection is performed by expanding the window from the binocular tracking as described above. When the relative distance between both eyes is greater than the predetermined distance, for example, as shown in FIG. 23, the left / right distance x20 of the left and right eyes is not abnormal, but if the value of the vertical distance y20 is clearly large, either It is determined that it is not an eye, and then the eye is specified by template matching. Template matching is the same as described above.
[0055]
In this case, the right eye is identified as the eye, and the window is expanded in the same manner as described above based on the coordinate value of the right eye to detect both eyes. Further, although the vertical distance y20 between the left and right eyes is small, when the left / right distance x20 changes to 1.5 times, for example, it is regarded as abnormal, and the eyes are similarly identified by template matching to expand the window. Accordingly, in any case, when the left and right windows 23 and 25 are tracking an object that is not an eye, this can be immediately regarded as abnormal and the process can proceed to detection of the right and left eyes. Moreover, in this case as well, it is not necessary to start from taking in image data, it is possible to perform a quick process, and it is possible to sufficiently cope with a case where a quick process such as a side-viewing operation is required.
[0056]
Next, with reference to FIG. 24, a case will be described in which the eye position detection circuit 8 returns to the eye tracking by the left and right windows in the eye position detection circuit 6 from the eye detection in the expanded region. This processing is performed by the eye position detection circuit 8.
[0057]
First, in step S5, eye coordinate values corresponding to both eyes are input. In step S6, it is determined whether or not the relative distance between both eyes is within a predetermined range. If the relative distance is within the predetermined range, the left and right windows are set again based on the coordinate values corresponding to both eyes in step S7. Detect and track both left and right eyes. Therefore, the eye can be detected again in a range with less noise as compared with the expanded window, and quick eye detection can be accurately performed.
[0058]
As described above, in the first embodiment of the present invention, the positions of both eyes are independently detected and tracked, and even if one eye disappears or is erroneously detected, the presence or absence of both eyes is detected by immediately expanding the window. The eye position can be searched in a short time even when the occupant turns around or moves the face greatly, and the eye can be tracked continuously.
[0059]
(Second Embodiment)
FIG. 25 shows a flowchart according to the second embodiment of the present invention. The basic configuration in this embodiment is the same as that shown in FIG. The flowchart of FIG. 25 also executes the area expansion circuit 7. Accordingly, steps corresponding to those in FIG. 15 are described with the same reference numerals, and redundant description is omitted.
[0060]
On the other hand, in FIG. 25 of the present embodiment, after the coordinate value of one eye disappears, it is determined in S8 whether or not the distance between both eyes before the movement is within a predetermined range and for a predetermined time or more. Here, “within a predetermined range” refers to taking, for example, 1000 coordinate values that are within ± 20% from the coordinate value at the time of eye position detection with reference to the distance L0 between both eyes (see FIG. 26). Coordinate values with the highest probability that the eye position exists are calculated and stored from the distribution of coordinate values of 1000 data. When 1000 data is converted into time, if the time of one loop is 0.1 sec, it becomes 100 sec. This is a case where the face is still in good condition, and since the face is actually moving, 1000 data is collected. Takes a long time (for example, 200 sec). If the eye data is collected under the condition that the eye stays as much as possible, the accuracy of the eye position is improved.
[0061]
In S8, when the distance between both eyes before the movement continues within a predetermined range and for a predetermined time or longer, the predetermined area, that is, the window is expanded to include the coordinates of both eyes before the movement in S9. Next, in S10, it is determined whether or not the distance between both eyes in the window is stable. That is, it is determined whether or not data corresponding to both eyes is detected in the expanded window, and if it is detected, the process returns to S1 again.
[0062]
In S8, if it is determined that the distance between both eyes before the movement is within a predetermined range and does not continue for a predetermined time or more, or if the distance has remained for a relatively short time, there is a possibility that the occupant is moving. is there. Here, the case of staying for a relatively short time is a case where the coordinate value of the distance L0 ± 20% of both eyes before the movement is less than 1000 data even after 200 seconds, and the driver's face is moving. Therefore, in this case, the eye position coordinates searched in the initial stage are used as a reference. That is, in S11, the predetermined area, that is, the window is expanded to include the initial coordinate values of both eyes. As the initial coordinate values of both eyes, the coordinate values of both eyes detected and stored in the eye position detection circuit 6 are used.
[0063]
Since the initial data is used here, the face is not always in the center of the screen frame 13, so it is ideal to search by expanding the window over the entire image. However, if it is expanded to the whole, for example, noise such as recognition of a vehicle interior pattern is recognized as an eye, the processing speed becomes slow, and it becomes impossible to follow the movement of the face. Therefore, the range in which the eye moves with the vertical and horizontal vibrations of the vehicle while the occupant is looking forward is set as the window size, and this is experimentally obtained. Since the size of the face changes depending on the driver, it is variable by proportional calculation from the distance of both eyes based on the obtained reference window.
[0064]
Next, in S12, it is determined whether or not the distance between both eyes is stable.
[0065]
This determination is the same as in S10.
[0066]
26 to 28 show processed images. Note that the processed image from the input of the images up to S1 and S2 and the one-eye coordinate value disappearing until one of the windows disappears and cannot be traced is the same as that in the first embodiment. Refer to FIG. FIGS. 27 and 28 are diagrams corresponding to FIGS.
[0067]
When one window disappears in this way, the coordinates of both eyes when looking forward and the position is determined to be stable before disappearing, that is, (x10, y10) ( The window 29 is expanded to include x20, y20) as shown in FIG. In addition, when the initial binocular coordinates, for example, 33 and 35, are used as shown in FIG. 30, the window 31 is expanded to a larger size.
[0068]
Therefore, even when the occupant's face 11 moves greatly to the opposite side, the eyes can be captured continuously. In this way, the present embodiment can achieve substantially the same operational effects as the first embodiment. On the other hand, in this embodiment, since the window is expanded to include the pre-stored eye position, the eye can be tracked quickly and accurately even when both eyes disappear.
[Brief description of the drawings]
FIG. 1 is a block diagram of the present invention.
FIG. 2 is a block diagram according to the first embodiment of the present invention.
FIG. 3 is a processed image according to the first embodiment.
FIG. 4 is a processed image of the first embodiment.
FIG. 5 is an explanatory diagram of template matching.
FIG. 6 is an explanatory diagram of pixel columns and density values.
FIG. 7 is an explanatory diagram of pixel columns and density values.
FIG. 8 is an explanatory diagram of a pixel column and a density column.
FIG. 9 is an explanatory diagram illustrating a processed image according to the first embodiment.
FIG. 10 is an explanatory diagram illustrating a processed image according to the first embodiment.
FIG. 11 is an explanatory diagram showing the size of a window.
FIG. 12 is a statistical diagram of the lateral length of the eye.
FIG. 13 is a statistical diagram of the length of the eye in the vertical direction.
FIG. 14 is an explanatory diagram for determining the position of a window.
FIG. 15 is a flowchart of the first embodiment.
FIG. 16 is a processed image of the first embodiment.
FIG. 17 is a processed image of the first embodiment.
FIG. 18 is a processed image of the first embodiment.
FIG. 19 is an explanatory diagram of how to determine the size of a window.
FIG. 20 is an explanatory diagram of how to determine the position of a window.
FIG. 21 is a flowchart of the first embodiment.
FIG. 22 is a processed image of an embodiment according to a modification of the first embodiment.
FIG. 23 is an explanatory diagram showing the relationship between the vertical distance and the horizontal distance of the left and right feature points.
FIG. 24 is a flowchart of the first embodiment.
FIG. 25 is a flowchart according to the second embodiment of the present invention.
FIG. 26 is a processed image of the second embodiment.
FIG. 27 is a processed image of the second embodiment.
FIG. 28 is a processed image of the second embodiment.
FIG. 29 is a processed image of the second embodiment.
FIG. 30 is a processed image of the second embodiment.
[Explanation of symbols]
CL1 image output means
CL2 facial feature amount detection means
CL3 first eye position detecting means
CL4 area determination means
CL5 second eye position detection means
CL6 area expansion means
CL7 Third eye position detection means

Claims (7)

Image output means for outputting occupant face image data;
Feature amount detection means for processing the face image data output from the image output means to detect a face feature amount;
First eye position detecting means for detecting an eye position based on the detected facial feature amount;
Area determining means for determining one predetermined area including the detected eye position for each of the left and right eyes, and for causing the predetermined area to track the eye according to the movement of the eye;
Second eye position detecting means for detecting the positions of the left and right eyes in each of the predetermined areas;
Left and right when the relative position of said second left and right eyes detected by the position detecting means of the eye when exceeding the set range, as one eye of the left and right eyes is not detected as the eye, the vehicle is vibrated Area expanding means for expanding the predetermined area to a size determined as a minimum size capable of simultaneously including the eyes of
An eye state detecting device comprising: third eye position detecting means for detecting positions of left and right eyes in the expanded predetermined region. The eye state detection device according to claim 1,
The eye state detection device, wherein the region expanding means extends a predetermined region including the position of the detected other eye. The eye state detection device according to claim 1,
The region expanding means stores at least one of the eye positions detected by the first eye position detecting means or the second eye position detecting means, and has a size including the stored eye position. An eye state detection device characterized by expanding an area. The eye state detection device according to any one of claims 1 to 3,
  The area determining means determines the left and right predetermined areas again when the relative positions of the left and right eyes are within a set range in the expanded predetermined area.
An eye state detection device characterized by that. The eye state detection device according to claim 4,
2. The eye state detection apparatus according to claim 1, wherein the relative position is a distance between left and right eyes. The eye state detection device according to claim 4,
2. The eye state detection apparatus according to claim 1, wherein the relative position is a vertical distance between the left and right eyes. The eye state detection device according to claim 3,
When the relative position of the left and right eyes detected by the second eye position detection unit is within a set range, the area expanding unit stores the position of the left and right eyes, and the relative position is set in the set range. The left and right eye positions detected by the first eye position detecting means are stored when the set time is not reached, and the predetermined area is expanded from the stored eye positions.
An eye state detection device characterized by that.

Images