JP2757756B2 - Eye corner detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for detecting the structure of the outer corner of the eye from a human face image with high accuracy, and for measuring communication between a human and a machine, such as human recognition and facial expression recognition from a facial image. It is applied to.

[0002]

2. Description of the Related Art An example of a conventional method for detecting an eye structure from a human face image will be described with reference to a document [Japanese Patent Application No. 4-153128]. At this time, it is assumed that a partial image of the eye as shown in FIG. 2 has been input.

Before describing the conventional method, it is used here.
Each edge of Step (-) / Step (+) / Roof / V-Ditch and
The Step (-) to V-Ditch area sandwiched between the edges will be described with reference to FIG. 3 based on the literature [Japanese Patent Laid-Open No. Hei 5-189566].

It is assumed that the horizontal axis in FIG. 3 is a coordinate and the vertical axis is a luminance value. When the input signal 100 is input, a first derivative signal 101 and a second derivative signal 102 of the input signal 100 are calculated. The point of the roof edge 103 is a local maximum point, and the first differential signal 101 is a 0 / second differential signal 102
Can be obtained under the condition that is negative. Step
(−) Edge 104 is an inflection point, and the primary differential signal 1
01 can be obtained under the condition that the negative / secondary differential signal 102 is 0. The V-Ditch Edge 105 is located at a minimum point, and can be obtained under the condition that the primary differential signal is 0 and the secondary differential signal 102 is positive. Although not shown in FIG. 4, Step (+) Edge is also used as an inflection point in Step
(-) Defined in the same way as Edge, the primary differential signal 10 can be obtained under the condition that the positive and secondary differential signals 102 are 0.

[0005] An area sandwiched between two types of edges is defined below. The Roof to Step (−) region 106 sandwiched between the Roof Edge 103 and the Step (−) Edge 104 is the first differential signal 1
01 and the second derivative signal 102 can be obtained under the condition that both are negative. Step (-) Edge104 and V-Ditch Ed
Step (-) to V-Ditch area 107 sandwiched by ge 105
Is that the primary differential signal 101 is a negative / secondary differential signal 102
Can be obtained under the condition that is positive. Although not shown here, similarly, the first differential signal and the second differential signal are used to calculate Step (+) to Roof area, V-Ditch to Step
(+) Area-Step (+) to Step (+) area-Step (-)
~ Step (-) region can be defined.

An eye structure to be detected will be described with reference to FIG.
It is assumed that a large eye is photographed in the input image 50.
The structure of the eyes to be extracted includes the pupil 51, the upper eyelid 52, and the inner eye 5
4. Eye corner 56. The conventional method is the inner corner 54 and the outer corner 5
6. This method is a method of modeling and detecting the structure of a portion of the upper eyelid 52 that is close to the inner corner of the eye 54 and the outer corner of the eye 56 as “the deepest position at the boundary between the eyeball and other parts”.

The conventional method will be described with reference to FIG.

Step (-) to V-Ditch area detecting means 30
Are input from the input image signal 11 as steps (−) to V-Ditc.
h area is cut out, and Step (−) to V-Ditch area signal 33
Is output. The upper eyelid candidate region detecting means 31 performs the Step (−)
With the ~ V-Ditch region signal 33 as an input, all regions wider than a certain threshold are selected as upper eyelid candidate regions, and an upper eyelid candidate region signal 34 is output. The threshold is chosen to be wider than the typical pupil width and narrower than the eye width. The upper eyelid candidate area signal 34 is input, and the upper eyelid area detecting means 32 outputs the upper eyelid area signal 12 assuming that the area having the lowest average luminance value in the area is the upper eyelid.

The first V-ditch Edge detecting means 6 receives an input image signal 11 which is an image obtained by cutting out the vicinity of the eyes, detects a dark point locally in a certain direction, and detects the first V-ditch Edge. -
Output as a ditchEdge signal 21. Second V-ditch Edg
Similarly, the e-detecting means 7 receives the input image signal 11 which is an image obtained by cutting out the vicinity of the eyes, and detects a dark point locally in a direction different from that of the V-ditch Edge detecting means 6 to perform the second detection. Is output as the V-ditch Edge signal 22. The OR means 3 calculates the upper eyelid region signal 12 and the first V-ditch Edge
A logical sum is calculated by using the signal 21 and the second V-ditch Edge signal 22 as inputs, and an eye structure point group signal 24 including an inner corner point, an outer corner point, and an upper eyelid line is output. The eye corner point selecting means 5 selects a point which is the most outside of the face from the eye structure point group signal 24 based on the face direction signal 14 for determining the direction of the outer corner of the eye, which indicates the vertical relationship of the face. In this method, an eye corner point is output and the eye corner point signal 15 is output.

[0010]

The above-mentioned eye structure detection method has two parameters. One is a parameter for detecting the upper eyelid region, and the other is the size of the edge operator. The former only needs to be set between the typical eye width and pupil width, and this parameter only detects the upper eyelid area that roughly indicates where the structure of the outer or outer corner of the eye is located. The amount did not affect the detection result even if there was some deviation.

However, the size of the edge operator is an amount that determines the edge position indicating the actual position of the outer or outer corner of the eye, and largely depends on the structure of the outer or outer corner of the eye to be detected. Conventionally, it has not been known what criteria should be used. Here, applying an edge operator having an appropriate size to an image is equivalent to performing low-pass filtering on the image and then performing differential processing, so the problem of determining the size of the edge operator is the cutoff of the low-pass filter. It can be considered to be equivalent to the problem of determining the frequency.

When the size of the edge operator is fixed, the problem is that if the size of the eyes is photographed in the image to be processed, the error with respect to the actual face size is reduced. You can also. From this, the above problem is equivalent to the problem of how to convert the size of the eye image to scale.

An object of the present invention is to automatically determine the size of the edge operator, the cut-off frequency of the low-pass filter, and the scale conversion parameter of the image from other easily detectable amounts, thereby enabling human detection. Detecting the outer corner of the eye with the same or better accuracy.

[0014]

Of the present invention SUMMARY OF THE INVENTION, the eye area detection device, in a device for detecting an image from the eye area structure obtained by photographing the vicinity of the eyes of a person's face, constant times the reciprocal of the eyes of the spacing of the face
By, mosquitoes of the low-pass filter in the spatial frequency
Means for determining a cutoff frequency, characterized in that it comprises means for generating an image of the low frequency components by applying the low-pass filter to the image, and means for processing the low frequency component image.

An eye corner detecting device according to the present invention is a device for detecting the structure of the outer corner of the eye from an image obtained by photographing the vicinity of the eyes of a person's face. The image processing apparatus further comprises means for generating an edge image by applying the edge operator to an image, and means for processing the edge image.

Further outside corner detection apparatus of the present invention, there is provided an apparatus for detecting an image from the eye area structure obtained by photographing the vicinity of the eyes of a human face, based on the distance between the eyes of the face, between the eyes after conversion
The gap is a constant multiple of the edge operator size given in advance.
Means for generating a scaled image by scaling in so that means for generating an edge image by applying the size of the edge operator to the scaled image, in that it comprises means for processing said edge image Features.

[0017]

According to the principle of the present invention, the size of the edge operator or the cutoff frequency of the low-pass filter is determined by using the approximate position of the eye which is easy to detect.

Since the color difference between the skin and the white eye position is not so large, it is very difficult to determine the eye structure such as the outer corner of the eye. For this reason, high-precision detection is impossible unless the features of the eye structure are used properly. By the way, since the difference between the eyes (sometimes the eyes) and the skin color in the face is remarkable, it is easy to determine the approximate position of the eyes. Therefore, by examining the relationship between the rough distance between the eyes and the detection accuracy of the eye structure based on the size of the edge operator or the cutoff frequency of the low-pass filter, it was found that the correlation between the rough distance between the eyes and the accuracy of the outer corner of the eye was large. . Conventionally, such a fact has not been known, and it has been necessary to determine the optimal parameters by fumbling.

It is experimentally clarified by processing eye images of various persons that high-precision extraction becomes possible by normalizing the edge operator size or the cutoff frequency of the low-pass filter at the interval between the eyes. It became. A document that is shown as an example of the conventional method [Japanese Patent Application No.
153128], the detection accuracy is the best when the ratio between the pupil distance and the size of the edge operator is around 500: 3, and the detection accuracy is at least as high as that when a human designates the outer corner of the eye by hand. Was obtained.
For the first time, this fact makes it possible to automatically detect the outer corner of the eye with high accuracy comparable to the detection accuracy by hand.

[0020]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of an eye corner detecting apparatus according to the present invention will be described with reference to FIG.

The V-ditch Edge operator determining means 4 for determining the size of a V-ditch Edge operator used to detect a locally dark point from the inter-pupil distance signal 13 causes the V-ditch Edge operator size signal 23 to be determined. Is output. The V-ditch edge operator determination means is V-ditch Ed
The ge operator size is determined by a constant multiple of the eye width. The first V-ditch Edge detecting means 1 receives an input image signal 11 which is an image obtained by cutting out the vicinity of the eyes and the V-ditch Edge operator size signal 23, and detects a locally dark point in a certain direction. Detected and output as a first V-ditch Edge signal 21. Similarly, the second V-ditch Edge detecting means 2
The input image signal 11 which is an image obtained by cutting out the vicinity of the eyes and the V-ditch Edge operator size signal 23 are input, and a dark point is locally detected in a direction different from that of the V-ditch Edge detecting means 1. Second V-ditch Edge signal 22
Output as The OR means 3 generates the upper eyelid region signal 12, the first V-ditch Edge signal 21, and the second V-ditch Edge signal.
A logical sum is calculated by using the signal 22 as an input, and an eye structure point group signal 24 including an inner corner point, an outer corner point, and an upper eyelid line is output. Also, based on the face direction signal 14 for determining the direction of the outer corner of the eye indicating the upper and lower relationship of the face, the outer corner point selecting means 5 receives the above-mentioned eye structure point group signal 24 as input, and The outside point is selected as the outer corner point, and is output as the outer corner point signal 15.

Next, a second embodiment of the eye corner detecting apparatus of the present invention will be described with reference to FIG.

The low-pass filter cut-off frequency signal 26 is determined by the low-pass filter cut-off frequency determining means 9 for determining a cut-off frequency for extracting a frequency component effective for examining the structure of the outer corner of the eye from the pupil distance signal 13. Is output. The low-pass filter cut-off frequency signal 26 by the low-pass filter cutoff frequency determination means 9, for constant times the reciprocal of the pupil distance
It is determined by that. The low-pass filter means 8 performs a low-pass filtering process on the input image signal 11 based on the low-pass filter cut-off frequency signal 26, and outputs a low-frequency signal 25. First V-ditch Edge detection means 6
Receives the low frequency signal 25 as an input, detects a locally dark spot in an area fixed in a certain direction,
Output as V-ditch Edge signal 21. Second V-ditch
Similarly, the edge detecting means 7 receives the low frequency signal 25 as an input, and detects a locally dark point in a region fixed in a direction different from that of the first V-ditch edge detecting means 6. Output as the second V-ditch Edge signal 22. The OR means 3 calculates the upper eyelid region signal 12 and the first V-
A logical sum is calculated by using the ditch Edge signal 21 and the second V-ditch Edge signal 22 as inputs, and an eye structure point group signal 24 including an inner corner point, an outer corner point, and an upper eyelid line is output. Based on the face direction signal 14, the eye corner point selection means 5 receives the eye structure point group signal 24 as input, selects a point that is the outermost of the face from the eye structure point group 24, sets the point as the eye corner point, By outputting the signal as the signal 15, the present invention can be realized.

A third embodiment of the eye corner detecting device of the present invention is as follows.
This will be described with reference to FIG.

The edge operator size given in advance,
From the pupil distance signal 13, determine the size of an image for extracting frequency components effective for examining the structure of the outer corner of the eye,
The scale conversion parameter signal 27 is output by the scale conversion parameter determination means 29. The scale conversion means 10 performs a scale conversion process on the input image signal 11 based on the scale conversion parameter signal 27, and outputs a scale converted image signal 28. By the scale conversion, the partial image of the eye is converted such that the distance between the pupils is a constant multiple of the above-mentioned size . First V-ditc
h The edge detecting means 6 outputs the scale-converted image signal 28
As input, V-ditch Edge operator of the above size
, A dark point is locally detected in an area fixed in a certain direction , and is output as a first V-ditch Edge detection signal 21. The second V-Ditch Edge detecting means 7 Similarly, as an input the scaled image signal 28, the
The size of the first V-ditch Edge operator
-ditch Edge detection means 6 detects a locally dark point in an area fixed in a direction different from that of the second V-ditch E
Output as dge signal 22. The OR means 3 receives the upper eyelid region signal 12, the first V-ditch Edge signal 21, and the second V-ditch Edge signal 22 as inputs and calculates a logical OR, and calculates the inner and outer corner points, the outer corner points, and the upper eyelid line. Eye structure point cloud signal 24 including
Is output. The eye corner point selecting means 5 receives the face direction signal 14 and the eye structure point group signal 24 as input, selects a point which is the outermost of the face from the eye structure point group 24, sets the point as the eye corner point, and sets the eye corner point signal. by outputting as 15, it can be <br/> the implementation of the present invention.

As described above, the eye structure detecting means has been described based on the method based on the document [Japanese Patent Application No. 4-153128]. Can be realized.

[0027]

According to the present invention, the size of the edge operator, the cutoff frequency of the low-pass filter, and the scale conversion parameter for determining the detailed structure of the outer corner of the eye, which were difficult in the past, are automatically determined based on the distance information between the eyes. , The outer corner of the eye can be detected with an accuracy equal to or higher than that detected by a human.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a first embodiment of an eye corner detection device according to the present invention.

FIG. 2 is a partial image of the eyes of a person to be processed.

[Figure 3] Roof Edge / V-Ditch Edge / Step (+) / Step
It is explanatory drawing of (-) Edge and the area | region pinched by two types of edges.

FIG. 4 is a configuration diagram of an eye structure detection device according to a conventional method.

FIG. 5 is a configuration diagram of a second embodiment of an eye corner detection device according to the present invention.

FIG. 6 is a configuration diagram of a third embodiment of an eye corner detection device according to the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 First V-Ditch Edge detecting means 2 Second V-Ditch Edge detecting means 3 Logical OR means 4 V-Ditch Edge operator determining means 5 Eye corner point selecting means 6 First V-Ditch Edge detecting means 7 Second V-Ditch Edge detection means 8 Low-pass filter means 9 Low-pass filter cut-off frequency determination means 10 Scale conversion means 11 Input image signal 11 12 Upper eyelid region signal 13 Interpupillary distance signal 14 Face direction signal 15 Eye corner signal 21 First V -Ditch Edge signal 22 Second V-Ditch Edge signal 23 V-Ditch Edge operator size signal 24 Eye structure point group signal 25 Low frequency signal 26 Low pass filter cutoff frequency signal 27 Scale conversion parameter signal 28 Scale converted image signal 29 Scale conversion parameter determination means 30 Step (-) to V-Ditch area detection means 31 Upper eyelid candidate area detection means 32 Upper eyelid area detection means 3 Step (-) to V-Ditch area signal 34 Upper eyelid candidate area signal 50 Input image 51 Eyes 52 Upper eyelid 53 Lower eyelid 54 Eye corner 55 Tears bag 56 Eye corner 100 Input signal 101 Primary differential signal 102 Secondary differential signal 103 Maximum value Roof Edge 104 Inflection point Step (−) Edge 105 Minimal value V-Ditch Edge 106 Roof to Step (−) region 107 Step (−) to V-Ditch region

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-5-108804 (JP, A) Extraction of feature points of eyes from face images Shizuo Sakamoto, Yoko Miyao, Joji Tajima Transactions of the Institute of Electronics, Information and Communication Engineers D -▲ II ▼ VOL. J76-D- ▲ II ▼ NO. 8 PP. 1796-1804 August 1993 Development of Security System Using Face Images Motoki Doi, Kunihiro Chihara ASP Symposium Proceedings Vol. 1993 PAGE. 37-40 1993 (known on September 17, 1993) Feature analysis for human face matching Shizuo Sakamoto, Yoko Miyao, Joji Tajima Proceedings of the 1990 IEICE Autumn National Convention Vol. 1990, NO. AUTUMN PT 6 PAGE. 6.411 1990 Development of Frontal Face Image Matching System for Access Control Motoki Doi, Osamu Oshiro, Kunihiro Chihara Proceedings of the Conference on System Control Information Society Vol. 38th Page. 531-532 1994 (known after filing the application) (58) Fields investigated (Int.Cl. ⁶ , DB name) G06T 1/00 G06T 5/20 G06T 7/00 H04N 7/15

Claims (3)

(57) [Claims] 1. A device for detecting the eye area of the structure around the eye from the captured image of the face of a person, by a constant multiple of the reciprocal of the eyes of the spacing of the face, the cut-off frequency of the low-pass filter in the spatial frequency eye area detection device for a means for determining, means for generating an image of the low frequency components by applying the low-pass filter to the image, characterized in that it comprises a means for processing the low frequency component image. 2. An apparatus for detecting a structure of an outer corner of an eye from an image of the vicinity of an eye of a person's face, comprising: means for determining an edge operator based on a distance between both eyes of the face; and applying the edge operator to the image. And a means for processing the edge image. 3. An apparatus for detecting a structure of an outer corner of an eye from an image of the vicinity of an eye of a person's face, wherein the distance between the converted eyes is determined in advance based on the distance between the eyes of the face.
Means for generating a scale-converted image by performing scale conversion so as to be a constant multiple of the size of the given edge operator; means for generating an edge image by applying an edge operator of the size to the scale-converted image And a means for processing the edge image.