JP3582324B2 - Eye position detector - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an eye position detecting device that can be used for automatically controlling a mirror and a seat, and for specifying a position of an eye necessary for detecting a drowsy driving state from an open / closed state of a driver's eyes.
[0002]
[Prior art]
As a conventional device for detecting the eye position of a driver or the like, there is, for example, a device described in JP-A-7-181012.
[0003]
This eye position detection device performs a binarization process on grayscale image data captured by a camera installed in front of a driver, and detects a position of an eye by performing a density projection, a labeling process, and the like on the binarized image. It has become.
[0004]
Also, as an eye position detection device for a grayscale image, Wish There is one described in JP-A-8-101904.
[0005]
This eye position detection device detects the density of pixels along a vertical pixel row of the face, determines one element for each local increase in density in the pixel row, sets an element as an extraction point, Extraction points adjacent to each other in the pixel column direction of the pixel column are connected to detect the position of the eye from curve data extending in the horizontal direction of the face.
[0006]
[Problems to be solved by the invention]
However, in the above-described eye position detecting device, as shown in FIG. 23, when there is a pattern (for example, a pattern that can show a view seen from the rear window) in a portion corresponding to the background of the eye position detection target person, In addition to the data of the nostrils and the like, a large number of data based on the background pattern appear, so that not only the selection formula for detecting the eyes becomes very complicated, but also the accuracy of detecting the position of the eyes decreases.
[0007]
Further, even in an eye position detecting apparatus for a gray-scale image, as shown in FIG. 24, a large number of curve data based on the background pattern still appear, and there is still the same problem.
[0008]
SUMMARY OF THE INVENTION It is an object of the present invention to provide an apparatus that can maintain detection accuracy of an eye position even when a lot of noise patterns and the like are present in a background portion of an eye position detection target person.
[0009]
[Means for Solving the Problems]
The invention according to claim 1, wherein image input means for inputting image data including a face, and eye search area setting means for setting the plurality of eye search areas by dividing the input image data into a plurality of pieces. And an eye search area changing means for sequentially switching the search areas of each eye, and detecting a pixel density along a vertical pixel row of the face in each of the eye search areas to be sequentially switched. , When the dark side of the light and dark in the image is defined so that the density value is lower than the light side, Of the density value on the pixel row Lower side Of the peak Immediately before The derivative of the density change is Judgment criteria value Is below On the condition that Lower side Point extraction means for identifying a peak pixel of the image as an extraction point; curve data extraction means capable of extracting curve data in which the extraction point extends in the pixel row direction in the horizontal direction of the face; A curve data confirming unit for confirming the presence or absence of curve data in an area; and searching for the eye to change to the plurality of divided eye search areas when the curve data is not confirmed by the curve data confirming unit. A search area change instructing means to be output to an area changing means; and an eye position for detecting an eye position from a relative positional relationship of the curve data in the eye search area when the curve data is confirmed by the curve data checking means. It is characterized by comprising detection means.
[0010]
According to a second aspect of the present invention, in the eye position detecting apparatus according to the first aspect, when the curve data is confirmed by the curve data confirming means, the topmost curve data of the topmost search area of the eye is detected. Curve data detection means, first stability condition determination means for the top curve data which is determined to be stable on condition that the top curve data appears at the same position a plurality of times, and first stability condition determination means. Reduced first area setting means for setting a reduced first area based on the uppermost curve data when the appearance of the uppermost curve data is stabilized by the determination; and curve data appearing in the reduced first area. And eye position detecting means for detecting the position of the eye from the eye.
[0011]
A third aspect of the present invention is the eye position detecting device according to the first or second aspect, wherein the appearance of the uppermost curve data is not stable as determined by the first stability condition determination means. First eye search area cutting means for cutting the upper part of the eye search area with reference to the above, and the uppermost curve data detecting means detects the uppermost curve data again in the eye search area after the cut. It is characterized by the following.
[0012]
According to a fourth aspect of the present invention, in the eye position detecting apparatus according to the second or third aspect, when the appearance of the uppermost curve data is stabilized by the determination of the first stability condition determining means, the uppermost curve data is obtained. A reduced second region setting means for reducing a region in the vertical direction and setting a reduced second region expanded in the left and right direction with reference to the one or two curve data appearing in the reduced second region from above. Curve data extraction means to be extracted, curve data feature quantity determination means for determining at least one of the positional relationship and length of the curve data extracted by the curve data extraction means, and the curve extracted by the curve data extraction means Second stability condition determination means for curve data that is determined to be stable on the condition that the data appears multiple times at the same position and the same length; With The reduced first area setting means sets the reduced first area based on the curve data when the appearance of the curve data is stable as determined by the second stability condition determining means.
[0013]
According to a fifth aspect of the present invention, in the eye position detecting apparatus according to the fourth aspect, when the appearance of the curve data in the reduced second area is not stable by the second stability condition determination means, the eye position detection apparatus is based on the curve data. A second search area cutting means for cutting the upper part of the eye search area, wherein the uppermost curve data detecting means detects the uppermost curve data again in the eye search area after the cut. And
[0014]
According to a sixth aspect of the present invention, in the eye position detecting apparatus according to the fourth or fifth aspect, the setting of the first reduced area is corrected in accordance with a state of curve data satisfying a stability condition in the second reduced area. Characterized in that it has a reduced first area setting correction unit that performs the reduction.
[0015]
According to a seventh aspect of the present invention, in the eye position detecting apparatus according to the sixth aspect, the reduced first area setting correction unit determines that the length of the curve data satisfying the stability condition in the reduced second area is a predetermined value. The setting of the reduced first area is corrected when the value exceeds.
[0016]
The invention of claim 8 provides any one of claims 4 to 7 1 item Wherein the reduced second area setting means sets the reduced second area based on the center coordinates of the uppermost curve data.
[0017]
The invention according to claim 9 is any one of claims 4 to 8. 1 item 3. The eye position detecting device according to claim 1, further comprising a curve data determining unit configured to determine that the curve data has not been confirmed when an interval between two left and right curve data appearing in the reduced second area is smaller than a predetermined value. The search area change instructing means, when the curve data determining means determines that the curve data is not confirmed, changes the eye search area to change to the plurality of divided eye search areas. Output to the means.
[0018]
【The invention's effect】
According to the first aspect of the invention, the detection area of the eye is divided into a plurality of small areas in advance, and the continuity in the horizontal direction of points extracted from the density change state serving as eye candidate data in one of the areas. By determining whether or not the curve data extracted in the above is confirmed, and by detecting the position of the eye when the curve data is confirmed in the area, it is possible to simplify the selection formula for detecting the eye, Can be improved in position detection accuracy.
[0019]
In other words, the entire image is divided, the stable appearance of the curve data appearing in the upper part in the area is determined, and the search area of the eyes is narrowed down sequentially, so that the face of the face due to individual differences of the driver or the like is determined. It is possible to cope with the difference in the position, and it is possible to detect the position of the eye with high accuracy even if a lot of noise data other than the facial components is included.
[0020]
According to the invention of claim 2, in addition to the effect of the invention of claim 1, a reduced first area is set on condition that curve data appearing at the uppermost end of the eye detection area is stable, and By detecting the position of the eye on the appearing curve data, the selection formula for detecting the eye can be simplified, and the accuracy of detecting the position of the eye can be further improved.
[0021]
According to the third aspect of the present invention, in addition to the effects of the first or second aspect, when the uppermost curve data does not appear, the upper part of the search area is cut, so that the uppermost stable curve data is reliably obtained. Can be detected.
[0022]
According to the invention of claim 4, in addition to the effect of claim 2 or 3, the reduced second area is set on condition that the curve data appearing at the uppermost end of the eye search area is stable. Since the reduced first area is set on condition that the state of the curve data appearing in the area is stable, the eye position detection accuracy can be further improved.
[0023]
According to the invention of claim 5, in addition to the effect of the invention of claim 4, when the appearance of the curve data is not stable, the upper part of the search area is cut again, and the stable curve data can be reliably detected.
[0024]
According to the sixth aspect of the invention, in addition to the effects of the fourth or fifth aspect, the set position of the reduced first area is corrected according to the state of the curve data, and the eye position detection accuracy is further improved. it can.
[0025]
According to the invention of claim 7, in addition to the effect of the invention of claim 6, when the length of the curve data exceeds a predetermined value, the set position of the reduced first area is corrected, for example, the user wears glasses. Even in such a case, it is possible to set an accurate reduced first region and further improve the eye position detection accuracy.
[0026]
According to the eighth aspect of the present invention, in addition to the effect of any one of the fourth to seventh aspects, by setting the reduced second area based on the center coordinates of the uppermost curve data, the position detection accuracy of the eye can be improved. Can be further improved.
[0027]
According to the ninth aspect of the present invention, in addition to the effects of any one of the fourth to eighth aspects, when the interval between the two left and right curve data appearing in the reduced second area is smaller than a predetermined value, the curve data is Since it is determined that no confirmation has been made, it is possible to further improve the eye position detection accuracy.
[0028]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 1 is a functional block diagram of an eye position detection device according to an embodiment of the present invention. The eye position detection device includes an eye search area setting and changing unit CL1, an image input unit CL2, and a density detection unit. CL3, point extraction means CL4, curve data extraction means CL5, confirmation means CL6, top end curve data detection means CL7, first stability condition determination means CL8, reduced second area setting means CL9, curve data The apparatus includes a determination unit CL10, a reduced first region setting unit CL11, a reduced first region setting correction unit CL12, and an eye position detecting unit CL13.
[0029]
The eye search area setting and changing means CL1 sets or changes the eye search area as a search area setting means, a search area change means, a search area change command means, and a first and second search area cut means. The image input means CL2 inputs a face image. The pixel row density detecting means CL3 detects the density of the vertical pixel row of the face image input from the image input means CL2. The point extracting means CL4 extracts points based on the increase in the density in the pixel row and its change state. The curve data extraction means CL5 extracts a curve group in the horizontal direction of the face by continuously connecting points adjacent to each other in the pixel column direction. The curve data checking unit CL6 checks the number of curve data in the set eye search area as a curve data checking unit and a curve data determining unit. The uppermost curve data detecting means CL7 detects the uppermost curve data in the eye search area. The uppermost curve data stability condition determination means CL8 determines a stable appearance of the uppermost curve data. The reduced second area setting means CL9 sets a reduced second area based on the curve data when the appearance of the uppermost curve data is stabilized. The curve data determination unit CL10 determines the appearance state of the curve data in the reduced second area as a curve data extraction unit, a very small amount determination unit, and a second stability condition determination unit. The first reduced area setting means CL11 sets the first reduced area based on the curve group specified in the second reduced area. The first reduced area setting correction means CL12 corrects the setting of the first reduced area in accordance with the state of the curve data specified in the second reduced area. The eye position detecting means CL13 detects the position of the eye in the reduced first area.
[0030]
FIG. 2 is a configuration block diagram according to an embodiment of the present invention.
[0031]
As shown in FIG. 2, a camera 21 is provided as an image input unit CL2 that is installed on the instrument and captures the driver's face from the front. In this embodiment, the input image from the camera 21 is horizontal as shown in FIG. It consists of 512 pixels in the (X) direction and 480 pixels in the vertical (Y) direction.
[0032]
The input image taken by the camera 21 is stored in the image memory 23 via the AD converter 22 as digital image input image data.
[0033]
An image data operation circuit 24 is connected to the image memory 23.
[0034]
The image data calculation circuit 24 includes a setting and changing unit CL1 for the eye search area, a density detection unit CL3 for the pixel row, a point extraction unit CL4, the curve data extraction unit CL5, Checking means CL6, the uppermost curve data detecting means CL7, the first stability condition determining means CL8, the reduced second area setting means CL9, the curve data determining means CL10, and the reduced first area setting means. CL11 and the setting reduction means CL12 for the reduced first area.
[0035]
The image data arithmetic circuit 24 is connected to an eye position detection circuit 25 as eye position detection means CL13 for detecting the position of the eye in the reduced eye search area.
[0036]
Next, the flow of operation in the above configuration will be described with reference to the flowcharts of FIGS. 3 to 6 and the explanatory diagrams of FIGS.
[0037]
First, in step S301, an eye search area is set. The set value of the eye search area in the initial state is one obtained by dividing the entire screen. An example of division in this embodiment will be described with reference to FIG. In FIG. 10, the entire screen is divided into four by I to IV, and the area is I in the initial state.
[0038]
In step S302, each variable such as a counter, a flag, and a memory used for the subsequent determination is initialized. At this time, the initial value of the variable gksyflg that controls the switching of the eye search area is 1.
[0039]
In step S303, it is determined whether or not the eye search area change flag is set. After the initialization of each variable in step S302, since the eye search area change flag has not been set, the process proceeds to step S304.
[0040]
In step S304, variables that need to be initialized each time one image data is determined are initialized. In step S305, pixel data in the eye search area is read out, and in step S306, eye candidate data serving as curve data (in the transition step, the curve data is referred to as continuous data).
[0041]
The processing contents in steps S305 and S306 will be described with reference to the flowchart of FIG. 6 and the explanatory diagrams of FIGS.
[0042]
First, in step S601, the face of the driver is photographed by the camera, and one frame of the input image is converted into a digital signal and stored in the image memory. In step S602, point extraction processing is performed on the eye search area I as shown in FIG. 7, and after the completion of one line, processing is shifted to the next adjacent line, and point extraction is performed on all lines in a predetermined direction. It is determined whether or not has been completed.
[0043]
If it is determined in step S602 that point extraction has not been performed on all lines, the process proceeds to step S603. In step S603, arithmetic mean averaging of the density values of one line in a predetermined direction is performed. This processing is intended to eliminate small variations in the change in density value when capturing image data, and to capture a global change in density value. FIG. 8A shows a processing result of the arithmetic averaging operation of the line data of Xa in FIG.
[0044]
In step S604 of FIG. 6, a differential operation is performed on the arithmetic mean value that is the operation result of step S603. This processing result is shown in FIG.
[0045]
In step S605 in FIG. 6, point extraction is performed using the differential value that is the result of the operation in step S604. In the point extraction method, points (p1 to p10) at which the differential value changes from negative to positive, and in FIG. 8A, points at which the graph is convex downward are extracted. This point becomes a one-way peak of the density value on the pixel row. Next, whether or not the change in the density value (q1 to q10) until reaching the point is equal to or less than a predetermined value, is equal to or less than the reference value for the differential value in FIG. Then, the Y coordinate values (a3, a4) are extracted for points having a density change less than or equal to a predetermined value. After this process is completed for one line, the process is switched to the process for the next line in step S606.
[0046]
If it is determined in step S602 that point extraction for all lines has been completed, point extraction as shown in FIG. 9A is performed. That is, in the Xa = X1 line shape in FIG. 7, three points A1, A2, and A3 are extracted. Thereafter, the process proceeds to step S607, in which the Y coordinate values of the extraction points (A1, A2, A3,...) Of the adjacent lines are compared, and if the Y coordinate value is within a predetermined value, (1) continuous data And (2) the continuous start line number and (3) the number of continuous data.
[0047]
Here, since the detection target is an eye, the feature amount thereof can be data that continues relatively long in the horizontal direction, so that continuous data can be extracted on the condition that the data continues in the horizontal direction by a predetermined value or more. Therefore, as a result of the processing in step S607, continuous data of G1 and G2 as shown in FIG. 9B can be recognized as curve data.
[0048]
After extracting the continuous data in step S306, the process proceeds to step S307. In step S307, the state of the variable gksyflg that controls the switching of the eye search area is determined. Since the initial value of gksyflg is 1 as described in step S302, the process proceeds to step S308. In step S308, it is determined whether or not continuous data exists in the eye search area. If there is one or more continuous data, the process proceeds to step S312. If there is no data, the process proceeds to step S309, and the undetected counter 1 of the continuous data is incremented. In step S310, it is determined whether or not the undetected counter 1 of the continuous data has exceeded 20. That is, after the undetected counter 1 is incremented, the process returns to step S304 again. If the continuous data is not extracted in the same eye search area, the undetected counter is incremented. Then, the process proceeds to step S311.
[0049]
In step S311, a search area change flag 1 is set. After that, the process shifts to step S303 to determine whether or not the search area change flag is set. At this time, since the search area change flag is set, the flow shifts to step S301 to change the eye search area. This processing will be described with reference to the explanatory diagram of FIG.
[0050]
When the driver's face is almost at the center of the screen as shown in I of FIG. 11, in step S306, continuous data of eyebrows and eyes is extracted as shown in I of FIG. However, when the driver's face comes to a position as shown in II in FIG. 11 depending on the driving posture or physique, continuous data does not exist in the eye search area I. This situation can be reliably grasped by repeating the process a plurality of times. This state is determined in step S310, and a search area change flag 1 is set as a trigger for switching to area II. In addition, even when the face is shifted leftward or the face is lower leftward, as shown in the example of III and IV in FIG. 11, it is possible to sequentially switch the trigger by the absence of continuous data in the eye search area. it can.
[0051]
The initial positions of the divided eye search areas and the switching order thereof are intended to detect the position of the driver's left eye earlier and more reliably. The reason is that in a normal driving posture, the left eye of the driver is most stochastically included in the region I, followed by the region II in which the position of the face is lowered due to the physique difference. Because. In addition, the reason why the driver's left eye is focused on is that in the case of a right-hand drive car, the image of the driver's right eye may be significantly deteriorated by the direct sunlight from the right side in consideration of the light environment in the passenger compartment. is there. Therefore, the setting and switching order of this area is left-right symmetric when the vehicle is a left-hand drive vehicle.
[0052]
As shown in FIG. 9B, the number of pieces of continuous data in the divided eye search area is limited as described above. Can be simplified, and the eye position detection accuracy can be improved.
[0053]
Next, a method for further improving the eye position detection accuracy after step S312 in the flowchart of FIG. 3 will be described.
[0054]
In step S312, the judgment value of each continuous data is read. In step S313, the slope of the continuous data is calculated from the vertical position of the continuous start point and the vertical position of the continuous end point, which are one of the determination values. This is a step of calculating and deleting continuous data having a large inclination. If it is assumed that the data is continuous data of eyes, data that is equal to or more than the inclination of the face can be deleted. This is to improve eye detection accuracy by reducing at least one candidate data from the eye selection targets.
[0055]
In step S314, similarly to step S307, the state of the variable gksyflg that controls switching of the eye search area is determined. Also here, since the initial value of gksyflg is 1, the flow shifts to step S315. In step S315, the same determination as in step S308 is performed after deleting the continuous data having a large inclination in step S313.
[0056]
Also in step S315, it is determined whether or not continuous data exists after deletion in the eye search area. If there is one or more continuous data, the process proceeds to step S401 in FIG. If there is no data, the process proceeds to step S316, and the undetected counter 2 of the continuous data is incremented.
[0057]
In step S317, it is determined whether or not the undetected counter 2 of the continuous data has exceeded 20. That is, after the undetected counter 2 is incremented, the process returns to step S304 again, and if no continuous data is extracted in the same eye search area, the undetected counter is incremented. , To step S318.
[0058]
In step S318, a search area change flag 1 is set. Thereafter, the flow shifts to step S303 to determine whether or not the search area change flag is set. At this time, since the search area change flag is set, the flow shifts to step S301 to change the eye search area. The method of changing the eye search area is described above with reference to FIG.
[0059]
In step S401 in FIG. 4, as shown in FIG. 12, the uppermost continuous data in the eye search area is detected. After that, the process shifts to step S403 to determine whether or not the appearance of the uppermost continuous data is stable. The condition for stabilizing the appearance of the uppermost continuous data is that it is determined that the uppermost continuous data continuously appears a predetermined number of times at an equivalent position allowing an extraction error of the continuous data in the eye search area. If this condition is not satisfied, the process moves to step S419, and it is determined whether or not the appearance state of the uppermost continuous data satisfies the unstable condition. The instability condition is a state in which the position of the topmost continuous data varies for each processing of each frame, and is counted up each time the continuous data position is different. The condition has been met. Therefore, the process until the unstable condition is satisfied returns to step S304, and the process proceeds to extraction of continuous data of the next frame.
[0060]
If the unstable condition is satisfied in step S419 before the appearance of the uppermost continuous data is stabilized in step S403, the process proceeds to step S404. In step S404, the search area change flag 3 is set, and the process returns to step S303. In step S301, the eye search area is changed.
[0061]
As a method of changing the eye search area, as shown in FIG. 13, the upper part of the search area is cut and reduced based on the top position of the continuous data that appears in the search area of the same eye. By this processing, a background pattern which tends to frequently appear in an unstable state or an area based on continuous data of splashed hair is deleted. When the continuous state at the uppermost end is unstable as shown in FIG. 13, the search area of the eye is reduced stepwise, and the continuous data does not appear in the reduced area as shown in FIG. In this case, it is possible to switch from the eye search area I to the eye search area II in step S308 or step S315 in FIG. 3, and to switch the eye search area to a position of the eye in the image data shown in FIG.
[0062]
If the eyes or eyebrows can be captured as the uppermost continuous data in the eye search area in this way, the appearance of the uppermost continuous data is very stable, and the stability condition in step S403 should be easily satisfied. Can be. At this time, gksyflg is set to 0, and the flow shifts to step S405. In step S405, it is determined whether or not gksyflg is 0. If it is 0, the process proceeds to step S406, and the eye detection accuracy is further determined based on the topmost continuous data position that has stably appeared. In order to improve the image quality, a reduced second area which is reduced in the vertical direction and expanded in the horizontal direction is set as the determination area shown in FIG. In this area, it is determined whether the detected continuous data is an eye or an eyebrow based on the appearance state of one or two continuous data appearing from the upper part in the same area.
[0063]
After setting the reduced second area in step S406, gksyflg is set to 88. If gksyflg is 88 in step S407, the process proceeds to step S408.
[0064]
In step S408, one or two continuous data appearing in the reduced second area shown in FIG. 15 are detected, and the corresponding continuous data is determined to be the eyebrow or eye or eyeglass frame depending on the length and positional relationship of the continuous data. Is determined, and the flow advances to step S409.
[0065]
In step S409, the eye or eyebrow is detected by capturing that continuous data that appears stably in a symmetrical positional relationship appears. As shown in FIG. 15, when the uppermost continuous data stably appearing in the eye search area is one side of the eye or eyebrow, it is long horizontally in the left-right direction based on the continuous data. (Based on the center coordinates of the topmost curve data) By detecting up to two consecutive data from the top in the reduced second region, the other eye or eyebrow in the target relationship can be detected. Also in these two continuous data, as in the case of the uppermost continuous data, when they are eyes or eyebrows, they appear at the same position and at the same interval. Therefore, the stability condition can be easily satisfied as in the case of step S403.
[0066]
If the appearance of the corresponding continuous data is not stable in step S409, or if the interval between two continuous data as shown in FIG. Is not an eye or eyebrow, and the flow shifts to step S420. FIG. 16 shows an example in which the background pattern has been detected as stable topmost continuous data. Even in such a case, the corresponding noise data can be removed by the determinations in steps S408 and S409. The reason is that if the data is continuous data such as a background pattern or bouncing hair, it is difficult to stabilize even if it appears alone, and there is another continuous data at the same height and at a predetermined distance or more. Is extremely small.
[0067]
In step S420, it is determined whether or not the appearance state of the symmetric continuous data satisfies the unstable condition. As described above, the unstable condition is a state in which the position and length of continuous data that is symmetrical in the left and right directions are detected for each processing of each frame, and the state is counted up each time the continuous data and length are different. , The unstable condition is satisfied when the number of times exceeds a predetermined number. Therefore, the process until the unstable condition is satisfied returns to step S304 and shifts to extraction of continuous data of the next frame. When this process is repeated, since gksyflg is 88, the flow proceeds in the order of steps S307, S312, S313, S314, S407, S408, S409, and S420, and the uppermost end continuous data is not detected.
[0068]
If the unstable condition is satisfied in step S420 before the appearance of the symmetric continuous data is stabilized in step S409, the process proceeds to step S410. In step S410, the search area change flag 3 is set, the process returns to step S303, and the search area of the eye is changed in step S301. The method of changing the eye search area at this time is to reduce the area based on the set position of the reduced second area set as the area for determining the appearance of symmetric continuous data. By this processing, a background pattern which tends to appear frequently in an unstable state or an area based on continuous data of bouncing hair is deleted.
[0069]
Also, by initializing gksylfg to 1 in step S302, the stability condition determination of the topmost continuous data in the reduced eye search area is returned again. If the stable topmost continuous data is detected again in the reduced eye search area, the stability condition of the symmetrical continuous data is determined again based on the continuous data, and the stability of the symmetrical continuous data is determined. The same determination is repeated until the condition is satisfied. Therefore, in this case as well, as shown in FIG. 17, the eye search area is reduced stepwise, and if continuous data does not appear in the eye search area, the eye search is performed in step S308 or step S315. By switching from the region I to the region II, the eye search region can be switched to a position where the eye exists in the image data shown in FIG.
[0070]
Next, as shown in FIG. 19, when the driver wears glasses and only one continuous data in steps S408 and S409 appears, or when two continuous data appear, one of them is very long. Will be described.
[0071]
In the case of a driver wearing black frame glasses, continuous left and right data in the reduced second area may be integrated. In such a case, instead of determining the interval between two continuous data, the determination can be made based on whether or not the length of the target continuous data is equal to or longer than a predetermined value.
[0072]
If stable left-right symmetric continuous data is detected in step S409, the variable gksyflg that controls switching of the eye search area is set to 99. In step S411, after confirming that the state of gksyflg is 99, the process proceeds to step S412.
[0073]
The reduced area in step S412 is defined as a reduced first area, and is an area that is set to be small around the driver's left eye in the case of an eye inside the vehicle as shown in FIG. The center of the continuous data on the right side of the symmetric continuous data is used as a reference for setting the area.
[0074]
Next, correction when setting the reduced first area will be described with reference to FIG. When the driver is wearing glasses, the uppermost continuous data detected in the range of the eye search area is as shown in FIG. 21A, and the center coordinates in the horizontal direction of the uppermost continuous data are as follows. May not match eye position. If the reduced first area is set using these coordinate values as they are, the left eye of the driver may not be able to catch the central part of the area, as shown in FIG. In such a case, it is determined that the topmost continuous data is longer than a predetermined value, and it is recognized that the user is wearing glasses. As shown in FIG. , The entire length L2 of the corresponding continuous data is calculated, and L2 is divided by a predetermined ratio a: b to set the reduced second region after correction. With this correction, the section D is corrected, and the eye can be detected at the center of the reduced first area. By performing the eye detection in the reduced first region of FIG. 22 which is set small around the left eye of the driver in this way, the eye selection formula can be greatly simplified, and the detection accuracy can be improved. is there. In addition, in the detection before using this selection formula, since it is assumed that the frame is an eyebrow, eye, or eyeglass frame, one or two continuous data in a limited area is detected from the upper part. This is a repetition of the judgment in a very simple form.
[0075]
Finally, a method of specifying the position of the eye in the processing after step S413 will be described.
[0076]
In step S413, a determination formula is created based on the relative positional relationship of limited facial components such as frames of eyeglasses, eyebrows, and eyes, and continuous data is selected according to which one of the expressions. In this selection, if the pattern does not match any pattern, the process shifts from step S414 to step S416, increments the undetected counter 3 of the continuous data, and shifts to step S417.
[0077]
In step S417, the same determination is repeated until the undetected counter 3 exceeds 20. In the processing flow at this time, since gksyflg is 99, the determination of continuous data that is symmetric at the uppermost end is not performed. If the undetected counter 3 exceeds 20 in step S417, the search area change flag 4 is set in step S418, the process returns from step S303 to step S304, and the eye position detection is restarted from the initial state. It is rare in practice to enter this flow, and the probability that the detected object is not an eye is very high.
[0078]
If there is continuous data that matches the eye selection formula in step S414, the process moves to step S415. In step S415, the stability condition value is calculated on the condition that the appearance position of the continuous data specified as the eye is detected a plurality of times at the same position.
[0079]
Thereafter, the stability condition is determined in step S501 of FIG. If the stability condition is not satisfied, a determination is made in step S502 as to a variation in coordinate data from a change in position coordinates of each eye in each frame. If it is determined in step S502 that there is no variation, the process returns to step S304, and the processing when gksyflg is 99 is repeated. If it is determined in step S502 that the coordinates vary, the process proceeds to step S503, and the undetected counter 4 of the continuous data is incremented.
[0080]
In step S504, it is determined whether or not the variation counter for counting the variation state has exceeded 10, and the same processing is repeated until the variation counter has exceeded 10. If so, the search area change flag 5 is set in step S505, and step S303 is performed. Then, the process returns to step S304, and the eye position detection is performed again from the initial state. When entering this flow, it can be said that the driver's face position is largely moving at the time of entering the final eye detection stage, and since the eye position detection in this state includes many errors, This is because it is preferable to return to the state and wait for the timing at which the face position calms down.
[0081]
If the coordinate position of the continuous eye data is stable in step S501, the coordinates of the eye are output in step S506.
[0082]
The overall processing flow has been described above with reference to the flowcharts of FIGS. 3 to 5. In the present embodiment, the description has been made in the order of setting the reduced second area and then setting the reduced first area. Even if the setting of the reduced second region is omitted, it is possible to improve the eye detection accuracy as compared with the conventional example.
[Brief description of the drawings]
FIG. 1 is a functional block diagram of an embodiment of the present invention.
FIG. 2 is a block diagram showing a configuration of one embodiment of the present invention.
FIG. 3 is a flowchart illustrating an operation of the embodiment.
FIG. 4 is a flowchart illustrating the operation of the embodiment.
FIG. 5 is a flowchart showing the operation of the embodiment.
FIG. 6 is a flowchart illustrating the operation of the embodiment.
FIG. 7 is an explanatory diagram related to a process in a flowchart.
FIG. 8 is an explanatory diagram relating to processing in a flowchart.
FIG. 9 is an explanatory diagram related to a process in a flowchart.
FIG. 10 is an explanatory diagram related to a process in a flowchart.
FIG. 11 is an explanatory diagram related to a process in a flowchart.
FIG. 12 is an explanatory diagram related to a process in a flowchart.
FIG. 13 is an explanatory diagram related to a process in a flowchart.
FIG. 14 is an explanatory diagram related to a process in a flowchart.
FIG. 15 is an explanatory diagram related to a process in a flowchart.
FIG. 16 is an explanatory diagram related to a process in a flowchart.
FIG. 17 is an explanatory diagram related to a process in a flowchart.
FIG. 18 is an explanatory diagram related to a process in a flowchart.
FIG. 19 is an explanatory diagram related to a process in a flowchart.
FIG. 20 is an explanatory diagram related to a process in a flowchart.
FIG. 21 is an explanatory diagram related to a process in a flowchart.
FIG. 22 is an explanatory diagram related to a process in a flowchart.
FIG. 23 is an explanatory diagram according to a conventional example.
FIG. 24 is an explanatory diagram according to a conventional example.
[Explanation of symbols]
CL1 Eye search area setting and changing means (search area setting means, search area changing means, search area change command means, first and second search area cutting means)
CL2 Image input means
CL3 Density detection means for vertical pixel row of face
CL4 Point extraction means in each pixel column
CL5 Curve data extraction means
CL6 Curve data number confirmation means (curve data confirmation means, curve data judgment means)
CL7 Uppermost curve data detecting means in a predetermined area
CL8 First stability condition judging means for uppermost curve data
CL9 Reduction second area setting means
CL10 Curve data determination means in reduced second area (curve data extraction means, feature quantity determination means, second stability condition determination means)
CL11 Reduction first area setting means
CL12 Reduction first area setting correction means
CL13 Eye position detecting means
21 TV camera
22 AD converter
23 Image memory
24 Image data operation circuit
25 Eye position detection circuit

Claims (9)

Image input means for inputting image data including a face; eye search area setting means for dividing the input image data into a plurality of pieces to set a plurality of eye search areas; Eye search area changing means for sequentially switching the areas, detecting the density of pixels along the vertical pixel row of the face in each of the eye search areas to be sequentially switched, and setting the dark side of the light and dark in the image to the bright side when defined as the density value is lower than the lower side of the concentration values on condition that the differential value of the change in concentration of the immediately preceding peak of low density value side on the pixel row is less than the determination reference value Point extraction means for identifying a peak pixel of the image as an extraction point; curve data extraction means capable of extracting curve data in which the extraction point extends in the pixel row direction in the horizontal direction of the face; Check for the existence of curve data in the area Curve data checking means, and a search area change output to the eye search area changing means to change to a plurality of other eye search areas when curve data is not checked by the curve data checking means Command means; and an eye position detecting means for detecting an eye position from a relative positional relationship of the curve data in the eye search area when the curve data is confirmed by the curve data confirming means. Eye position detection device. The eye position detection device according to claim 1,
When the curve data is confirmed by the curve data confirmation means, the top curve data detection means for detecting the top curve data of the eye search area, and the top curve data appears at the same position a plurality of times. A first stability condition determining means for the uppermost curve data which is determined to be stable on the condition that the uppermost curve data is stable when the appearance of the uppermost curve data is stabilized by the determination by the first stability condition determining means. A first reduced area setting means for setting a first reduced area based on an eye position; and an eye position detecting means for detecting an eye position from curve data appearing in the first reduced area. Detection device. The eye position detection device according to claim 1 or 2,
When the appearance of the uppermost curve data is not stable as determined by the first stability condition determination means, first eye search area cutting means for cutting the upper part of the eye search area based on the uppermost curve data is provided, The eye position detecting device, wherein the uppermost curve data detecting means detects the uppermost curve data again in the eye search area after the cut. The eye position detection device according to claim 2 or 3,
When the appearance of the uppermost curve data is stabilized by the determination of the first stability condition determining means, the area is reduced in the vertical direction based on the uppermost curve data and the reduced second area expanded in the horizontal direction is set. A second reduced area setting means, a second curve data extracting means for extracting one or two pieces of curve data appearing in the second reduced area from above, and a positional relationship between the curve data extracted by the curve data extracting means. A curve data characteristic amount determining unit that determines at least one of the lengths, and a curve that is determined to be stable on condition that the curve data extracted by the curve data extracting unit appears a plurality of times at the same position and the same length. comprising a second stable condition judging means of the data, and the reduced first area setting means, if the appearance of the curve data is determined in the second stable condition judging means is stabilized Eye position detecting device and sets the reduced first region relative to the curve data. The eye position detection device according to claim 4,
When the appearance of the curve data in the reduced second area is not stable as determined by the second stability condition determining means, the image processing apparatus includes second eye search area cutting means for cutting an upper part of the eye search area based on the curve data. The eye position detecting device, wherein the uppermost curve data detecting means detects the uppermost curve data again in the search area of the eye after cutting. The eye position detection device according to claim 4 or 5,
An eye position detection device comprising: a reduced first area setting correction unit that corrects the setting of the reduced first area according to the state of the curve data satisfying the stability condition in the reduced second area. The eye position detecting device according to claim 6,
Wherein the first reduced area setting correction means corrects the setting of the first reduced area when the length of the curve data satisfying the stability condition in the second reduced area exceeds a predetermined value. Detection device. A eye position detecting device according to any one of claims 4-7,
The eye position detecting apparatus according to claim 1, wherein the reduced second area setting means sets the reduced second area based on the center coordinates of the uppermost curve data. A eye position detecting device according to any one of claims 4-8,
When the interval between the two left and right curve data appearing in the reduced second area is smaller than a predetermined value, the data processing device includes a curve data determination unit that determines that the curve data has not been confirmed
The search area change instructing means, when the curve data determining means determines that the curve data is not confirmed, the eye search area changing means to change to the plurality of divided eye search areas. An eye position detection device, which outputs an image to the eye.

Images