JP2010003232A - Eye detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an eye detection device for preventing a contour of eyeglasses from being erroneously detected as a contour of an eye in a face image of a person wearing the eyeglasses. <P>SOLUTION: The eye detection device comprises an image sensor for acquiring a driver's face image, and an image processing ECU. The image processing ECU first extracts a driver's upper and lower eyelid candidate edge on face image data and detects a driver's face orientation angle on the face image data. Then, the image processing ECU determines whether or not a driver's face is turned to the side, and extracts a face contour edge on the face image data when the face is turned to the side. The image processing ECU also determines whether there is a face contour edge shift at the top and bottom of each upper and lower eyelid candidate edge, determines the upper and lower eyelid candidate edge to be an upper and lower eyelid edge when there is no face contour edge shift, and determines the upper and lower eyelid candidate edge as an eyeglass edge when there is a face contour edge shift. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an eye detection device that detects eyes of a driver of a vehicle, for example.

As a conventional eye detection device, for example, as described in Patent Document 1, the driver's face is imaged, the curvature of the upper eyelid is detected by edge extraction from the driver's eye image, and the curvature of the upper eyelid is detected. Based on the above, there is known one that makes an open / closed eye determination.
JP 2007-257332 A

  However, in the above prior art, when the driver is wearing glasses, the edge of the glasses is erroneously detected as the edge of the eyelid, and as a result, the driver's opening / closing eye may be erroneously determined. There is.

  The objective of this invention is providing the eye detection apparatus which can prevent the misdetection with the outline of glasses and the outline of eyes in the face image of the subject who wears glasses.

  The present invention provides an eye contour candidate extraction unit that extracts a target person's eye contour candidate in a face image in an eye detection device that acquires the target person's face image and detects the subject's eye based on the face image. The face contour extracting means for extracting the face contour of the subject in the face image, the face direction detecting means for detecting whether the face of the subject in the face image is facing sideways, and the face orientation detecting means A determination means for determining whether or not the face contour is discontinuous above and below the eye contour candidate when it is detected facing sideways; and the face contour is discontinuous above and below the eye contour candidate by the determination means And determining means for determining that the eye contour candidate is not the eye contour of the subject person.

  The present invention is made by paying attention to a phenomenon in which the outline of a face shifts in the lens of the eyeglass when the subject wearing the eyeglasses turns his face sideways (left and right). That is, in the eye detection device of the present invention, the candidate's eye contour candidate and face contour are extracted based on the subject's face image, and whether the subject's face is facing sideways is detected. When the subject's face is facing sideways, it is determined whether the face contour is discontinuous above and below the eye contour candidate. At this time, when the face contour is shifted and discontinuous above and below the eye contour candidate, the eye contour candidate is not the eye contour of the subject but the contour of the glasses worn by the subject. Is determined. This prevents erroneous detection of the eyeglass contour as the eye contour.

  Preferably, the determination unit determines whether or not the face contour is discontinuous above and below the eye contour candidate based on whether or not the face contour is shifted above and below the virtual line obtained by extending the eye contour candidate. In this case, even if the eye contour candidate does not intersect the face contour depending on the face orientation angle of the target person, the virtual line obtained by extending the eye contour candidate intersects the face contour. It is possible to accurately determine whether or not the face contour is shifted up and down and is discontinuous.

  ADVANTAGE OF THE INVENTION According to this invention, the misdetection with the outline of spectacles and the outline of eyes can be prevented in the face image of the subject wearing spectacles. This makes it possible to accurately detect the eyes that are feature points of the subject's face.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of an eye detection device according to the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a schematic configuration diagram showing an embodiment of an eye detection apparatus according to the present invention. In the figure, an eye detection device 1 according to the present embodiment is a device that is mounted on a vehicle and detects the eyes of a driver D. The eye detection device 1 includes an image sensor 2 and an image processing ECU (Electronic Control Unit) 3 connected to the image sensor 2.

  The image sensor 2 is a sensor that is installed on, for example, the steering column cover 4 and captures the face of the driver D to acquire the face image of the driver D. For example, a CCD camera is used as the image sensor 2.

  The image processing ECU 3 includes a microcomputer including a CPU, a ROM, a RAM, and the like as main components. The image processing ECU 3 inputs the face image of the driver D acquired by the image sensor 2, performs predetermined processing including image processing, and detects the upper and lower eyelids (eyelids) of the driver D. Since the image processing ECU 3 processes only the image acquired by one image sensor 2, a very high-speed processing chip as in the case of processing a plurality of images becomes unnecessary.

  FIG. 2 is a flowchart showing details of the wrinkle detection processing procedure executed by the image processing ECU 3. In the figure, first, the face image data of the driver D obtained by the image sensor 2 is inputted (step S51).

Then, for example, a plurality of upper and lower eyelid candidate edges (see P _{1 to} P _{4 in} FIG. 4) of the driver D on the face image data of the driver D are extracted by using an edge-based method or a pattern recognition method (procedure) S52). At this time, for example, the likelihood is obtained by using the edge strength and the continuity of the edge as parameters, and those having a likelihood higher than a predetermined value are extracted as upper and lower eyelid candidate edges.

  Subsequently, the face direction angle of the driver D in the lateral direction (left and right direction) on the face image data of the driver D is detected (step S53). At this time, a roll angle (rotation angle), a pitch angle (vertical angle), and a yaw angle (horizontal angle) are calculated by image processing.

  Then, it is determined whether or not the face of the driver D faces in the lateral direction without facing the front direction (step S54). Specifically, it is determined whether or not the yaw angle of the driver D's face exceeds a preset threshold value. When it is determined that the face of the driver D is facing the front, the process returns to step S52.

  On the other hand, when it is determined that the face of the driver D is facing sideways, a face contour edge (see Q in FIG. 4) on the face image data of the driver D is extracted (step S55). Then, it is determined whether or not there is a step of the face contour edge above and below each of the upper and lower eyelid candidate edges extracted in step S52 (step S56).

  Here, when the driver D wears glasses, when the driver D turns his / her face in the left / right direction (lateral direction) for left / right confirmation or the like, the face seen through the lens of the glasses is displayed. The phenomenon that the contour line shifts occurs. Specifically, when the spectacle lens is a near-sighted (myopia) lens, when the driver D turns his face in the left-right direction, as shown in FIG. When the contour line h shifts inward and the spectacle lens is a presbyopia (hyperopia) lens, when the driver D turns his face in the left-right direction, as shown in FIG. The contour line h of the face is shifted outward in the lens.

Therefore, as shown in FIG. 4, virtual lines R _{1 to} R ₄ (see broken lines) obtained by extending the upper and lower eyelid candidate edges P _{1 to} P ₄ to the outside of the face are drawn. At this time, virtual lines R _{1 to} R ₄ are drawn from pixel coordinates (for example, five points) closest to the face contour edge Q. Thus, depending on the yaw angle of the face of the driver D, even if the upper and lower eyelid candidate edges P ₁ and P ₂ corresponding to the true upper and lower eyelid edges do not intersect the face contour edge Q, the upper and lower eyelid candidate edges P ₁ and virtual line _R 1 obtained by extending the P _{2, R 2} is intersect with the face edge rims Q. FIG. 4 shows a face image when the driver D wearing myopia glasses turns his face in the left-right direction.

Then, it is determined whether the upper and lower vertical eyelid edge P ₁ to P ₄ virtual line R ₁ to R ₄ which is extended shift the face contour edge Q (step) is not.

If the face contour edge is not shifted above and below the virtual line obtained by extending the upper and lower eyelid candidate edges (see R ₂ and R _{3 in} FIG. 4), the upper and lower eyelid candidate edges are determined to be the upper and lower eyelid edges (procedure) S57). On the other hand, when the face contour edge is shifted above and below the virtual line obtained by extending the upper / lower eyelid candidate edge (see R ₁ and R _{4 in} FIG. 4), the upper / lower eyelid candidate edge is determined as the eyeglass edge, Exclude from the upper and lower eyelid candidate edges (step S58).

  In the above, steps S51 and S52 constitute eye contour candidate extraction means for extracting the subject's eye contour candidate from the face image. Steps S51 and S55 constitute face contour extracting means for extracting the face contour of the subject in the face image. Steps S51, S53, and S54 constitute face orientation detection means for detecting whether the face of the subject in the face image is facing sideways. Step S56 constitutes determination means for determining whether or not the face contour is discontinuous above and below the eye contour candidate when the face orientation detection means detects that the subject's face is facing sideways. To do. Step S58 constitutes a determining unit that determines that the eye contour candidate is not the eye contour of the subject when the determining unit determines that the face contour is discontinuous above and below the eye contour candidate.

  As described above, in the present embodiment, the upper and lower eyelid candidate edges of the driver D are extracted on the basis of the face image data of the driver D, and the lateral direction angle of the driver D is detected, When is not facing the front direction, a face contour edge is further extracted based on the face image data. Then, the continuity of the face contour edge above and below the virtual line obtained by extending each of the upper and lower eyelid candidate edges is determined, and when it is determined that the face contour edge is not continuous above and below the virtual line, the upper and lower eyelid candidate The edge is determined to be an eyeglass edge so that it is not extracted as an eye contour edge. Accordingly, when the driver D is wearing glasses, it is possible to prevent erroneous detection of the edge of the glasses as the eye contour edge in the face image of the driver D.

  The present invention is not limited to the above embodiment. For example, the eye detection device 1 of the above-described embodiment detects the eyes of the driver D of the vehicle. However, the present invention is not limited to a vehicle, and is mounted on a robot, for example, for people around the robot. The present invention can also be applied to those that detect eyes.

1 is a schematic configuration diagram showing an embodiment of an eye detection device according to the present invention. It is a flowchart which shows the detail of the wrinkle detection process procedure performed by image processing ECU shown in FIG. It is a figure which shows notionally the phenomenon in which the outline of a face shifts within the lens of spectacles when the driver who wore spectacles turned his / her face in the left-right direction. It is a figure which shows an example of a face image when the driver | operator who wore spectacles turned his / her face in the left-right direction.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Eye detection apparatus, 2 ... Image sensor, 3 ... Image processing ECU (Eye outline candidate extraction means, face outline extraction means, face direction detection means, judgment means, discrimination means), D ... Driver (subject).

Claims (2)

In an eye detection device that acquires a face image of a subject and detects the eyes of the subject based on the face image.
Eye contour candidate extracting means for extracting eye contour candidates of the subject in the face image;
A face contour extracting means for extracting the face contour of the subject in the face image;
Face orientation detection means for detecting whether the face of the subject in the face image is facing sideways;
Determining means for determining whether the face contour is discontinuous above and below the eye contour candidate when the face orientation detecting means detects that the subject's face is facing sideways;
And determining means for determining that the eye contour candidate is not the eye contour of the subject when the determining means determines that the face contour is discontinuous above and below the eye contour candidate. A featured eye detection device. The determination unit determines whether the face contour is discontinuous above and below the eye contour candidate, depending on whether the face contour is shifted above and below a virtual line obtained by extending the eye contour candidate. The eye detection device according to claim 1.

Images