JP2000235648A - Extracting device for eye and blink detecting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correctly extract the positions of eyes by eliminating effects such as glasses and eyebrows from a facial image about an extracting device for eyes. SOLUTION: The extracting device for eyes which extracts a candidate position for eyes from an image and extracts a pattern image being similar to the eyes from a prescribed area of the image including the candidate position has an image processing part 8 which binarizes the image in the prescribed area including the extracted eye candidate position and extracts the pattern image and a recognizing part 9 which performs comparison processing among the plural extracted pattern images about prescribed evaluation items and recognizes the pattern image whose evaluation points are the highest as the eyes.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an improvement of an eye extraction device for extracting eyes from a face image and a blink detection device for detecting blinks by tracking eyes in a moving image.

[0002] In recent years, a drowsiness alarm device that monitors the eyes of a vehicle driver and detects the driver's state, such as drowsiness, by detecting the presence or absence of blinking and the like and generates an alarm has been developed. It is necessary to accurately extract and track eyes and detect blinks while eliminating the effects of individual differences such as eyebrow shape and eye size.

[0003]

2. Description of the Related Art As a method of tracking an eye in real time, a method of finding an eye by performing pattern matching between a face image and a standard eye type (template) is generally performed. However, there is a problem that a part similar to the eyes (eyebrows, frames of eyeglasses, etc.) is erroneously recognized as an eye, and the eyes cannot be tracked due to strict setting of the criterion. For blink detection, a method of tracking changes in the distance between eyes and eyebrows has been proposed, but since there are individual differences in eye size, eyelash length, etc., accurate recognition with a fixed threshold value is performed. It was difficult to do.

SUMMARY OF THE INVENTION In view of the above-mentioned problems, an object of the present invention is to provide an eye extraction device that eliminates individual differences and that simply and quickly extracts an eye, and a highly accurate blink detection device.

[0005]

In order to solve the above-mentioned problems, an eye extracting device and a blink detecting device according to the present invention are configured as follows. (First Invention) In the first invention, as shown in the principle diagram of FIG. 1, the candidate positions of an eye are extracted from an image, and similar to the eyes from a predetermined area of the image including the candidate position. An eye extraction device that extracts a pattern image, determines the eye position by comparing the pattern image based on a predetermined evaluation item, and binarizes an image of a predetermined region including the extracted eye candidate position. An image processing unit 8 for extracting a pattern image, and a recognition unit 9 for performing a comparison process between the extracted plurality of pattern images for a predetermined evaluation item and recognizing a pattern image having the highest evaluation point as an eye. Have.

As described above, even if a candidate eye position is extracted by pattern matching or the like, the eye is recognized by comparing it with a pattern image around the candidate position without immediately determining the candidate eye position. This prevents the eyes from being erroneously recognized as other parts such as eyebrows. (Second Invention) A second invention extracts a candidate position of an eye from an image, extracts a pattern image similar to the eye from a predetermined region of the image including the candidate position, and An eye extraction device that determines an eye position by comparing based on an evaluation item, an image processing unit that binarizes an image of a predetermined region including the extracted eye candidate position and extracts a pattern image, A recognizing unit that performs fuzzy evaluation processing on the extracted pattern images for the evaluation items and recognizes the pattern image with the highest evaluation point as an eye.

[0007] Since the criteria for the evaluation items are relatively ambiguous, the eyes can be more accurately recognized by fuzzy evaluation using a plurality of evaluation items as described above. (Third invention) A third invention is a blink detection device which detects and blinks by monitoring and tracking eyes in an extracted moving image. A plurality of evaluation items representing blinking, such as a change in the interval between other parts, are tracked, and fuzzy evaluation processing is performed for each item. When the total evaluation point reaches a predetermined value, it is determined that the eye blinks. It has a determination unit.

As described above, blinking can be detected more accurately by fuzzy inference based on a plurality of evaluation items.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 2 is a configuration diagram of one embodiment, and FIG.
4 is a diagram showing an example of grayscale image processing, FIG. 4 is a flowchart of eye extraction, FIG. 5 is an explanatory diagram of the length of a black block, FIG. 6 is a diagram showing a gap for each frame, FIG. 8
FIG. 4 is a flowchart of blink detection. Note that the same reference numerals represent the same object throughout the drawings.

FIG. 2 shows a configuration example of an apparatus having both a function of imaging a human face and extracting eyes from the obtained face image and a function of detecting blinks while tracking the extracted eyes. This is applied to the following first to third embodiments.

In FIG. 2, reference numeral 1 denotes an image pickup unit which is constituted by an image sensor or the like and is directed to, for example, the face of a driving person. Then, the obtained image is stored in the memory 2 for each frame. 3 is a standard template stored in memory,
For example, it is composed of an eye image obtained by averaging a plurality of human eyes. Reference numeral 4 denotes an eye tracking memory, which is used as a work for tracking processing, and stores eye position information as a tracking result.
Reference numeral 5 denotes a blink tracking memory which stores, for example, measured eye thickness information, intervals between the eyes and other parts, and the like in frame units.

Reference numeral 6 denotes an eye extraction unit which is composed of a pattern matching unit 7, an image processing unit 8, a recognition unit 9 and the like. As will be described later, an eye candidate position is extracted by pattern matching using a standard template 3. Then, an image of a predetermined area in the vicinity including the eye candidate position is cut out, an evaluation process is performed on the evaluation item, and the eye position is determined. Reference numeral 10 denotes an eye tracking unit that constantly searches the vicinity of the determined eye position by pattern matching to track the eye position. Reference numeral 11 denotes a blink detection unit, which includes a tracking unit 12, a determination unit 13, and the like. As will be described later, a change in the distance between the upper and lower eyelids, a change in the thickness of the eyes, and the like.
A plurality of evaluation items characteristically representing blinking are tracked temporally, and each evaluation item is evaluated fuzzy to detect blinking.

(First Embodiment) The first embodiment is an embodiment of the above-described first invention. First, eye candidate positions are obtained by pattern matching between an image and a standard template. A plurality of vertical images are cut out from a predetermined area including a candidate position and binarized, and two sets of pattern images are extracted from the bottom of a plurality of eye-like pattern images to obtain one evaluation item. 5 shows an example in which one of the two sets of pattern images is determined to be an eye by comparing the uniformity of the thickness.

First, the pattern matching unit 7 performs a pattern matching process while moving the standard template 3 in the image stored in the memory 2 and sets a position where the maximum pattern matching rate is obtained as an eye candidate position. The search result of FIG. 3 shows a pattern in which the standard template 3 in the face image matches the eyes. Next, the image processing unit 8 acquires grayscale data in a plurality of vertical lines having a predetermined length with the eye candidate position found by pattern matching as the center. At this time, the more vertical lines, the more detailed data can be obtained. However, if the number of vertical lines is too large, the processing time will not be within one frame. The grayscale data is binarized and separated into skin (white) and other (black) parts. In addition to the eyes, the eyebrows, eyeglass frames, bangs,
Mole etc. are included. So separate the eyes from the rest.

Next, an orthogonal projection of the binarized data is obtained. Then, the length of the black portion that continues in the horizontal direction at the center position of the black portion (hereinafter, black block) of each orthographic projection is obtained. If the length of the black portion is equal to or less than the threshold value, it is judged as bangs or moles and excluded. The remaining black parts become eyes, eyebrows, or eyeglass frames.

Next, the recognizing unit 9 makes a determination on the two lowermost black portions. As a result, a person wearing spectacles makes a comparison between the lower spectacle frame and the eye, and a person who does not wear spectacles makes a comparison between the eye and the eyebrow. In the embodiment, utilizing the fact that the thickness of eyebrows and eyeglasses is uniform compared to the eyes, the length in the vertical direction is obtained from the grayscale data of each line, and the pattern image having the larger difference is determined as the eye. I do. (Second embodiment) In the first embodiment, one evaluation item,
For example, the eyes are determined based on the uniformity of the thickness. However, the determination is slightly inaccurate for a person with fine eyes. In the second embodiment, a plurality of evaluation items are set with high accuracy so as not to be affected by such variations due to individual differences, and an evaluation point (score) of a black part is added for each item, and a score is calculated. Make a fuzzy comprehensive judgment so that you can see the higher one.

The processing prior to the determination of the eyes, that is, the processing until narrowing down to the lower two black portions of the pattern image obtained as eye candidates is performed in the processing flow shown in FIG.
As shown in (1) to (8), this is the same as in the first embodiment.
Thereafter, the recognizing unit 9 performs a step for the two black portions.
The lowermost black part is evaluated in (9), the second lowermost part is evaluated in step (10), and both scores added in step (11) are compared. The eye is determined.

Hereinafter, one example of the evaluation items and the additional points will be described. (1) Uniformity of thickness of black part Generally, eyebrows and eyeglass frames are uniform in thickness, so the higher the uniformity, the lower the score. (2) The maximum value of the thickness of the black part The width of the eyes varies slightly depending on the individual, but it is within a certain range, so points were added within that range, and deductions were made at other points. (3) Horizontal length of the black part Eyebrows and eyeglass frames are often longer than the eyes. (4) Distance between two black parts In general, “distance between lower frame of eyeglasses and eyes” is longer than “distance between eyes and eyebrows”. Are added to the black part, and when the value is equal to or less than the threshold, the point is added to the lowest black part.

As described above, by preparing a plurality of evaluation items and comparing them by adding scores, it is possible to perform eye judgment based on ambiguous evaluation items accurately and without influence of individual differences. Becomes

Although the case where the position of the eye is first extracted has been described above, the present invention can be applied to the case where the eye is tracked. (Third Embodiment) A third embodiment is an embodiment for detecting blinks according to the third aspect of the present invention. In the method of detecting a blink, the state of the eye obtained by tracking by the eye tracking unit 10 is tracked, for example, on a frame basis, and when a state change corresponding to the blink is detected, it is determined to be a blink.

Since this state change is influenced by individual differences such as the size of eyes, noise, changes in posture, etc., scores are added to a plurality of evaluation items. judge.

FIG. 5 illustrates the inaccuracy of blink detection due to a change in eye thickness. As shown in (1)-(a) and (b) of FIG. 5, the black block (orthogonal projection length) of the eye portion is generally long when the eye is open and short when the eye is open. However, if the eyelashes are long and the eyes are narrow, (2)-(a) and (b) in FIG.
As shown in (2), the length of the black block does not change much between when the eye is opened and when the eye is closed. In other words, the length of the black block alone
In some cases, blinking cannot be detected. Therefore,
A fuzzy method is used in which a plurality of evaluation items are provided and a score is added for each item.

In the present example, a change in the maximum value (gap) of the distance between the upper and lower eyelids (the steepness of the gap decrease and the time to return to the original state) and a change in the contour of the eyes are employed as a plurality of evaluation items. If the total score is above the threshold,
It is determined that the person blinked. FIG. 8 shows the processing flow. The evaluation items are not limited to these. (1) Extract and binarize an image of a predetermined area including the eyes. In addition, since the blink detection processing is generally performed simultaneously with the eye tracking processing, the binarization processing is omitted here, and the data binarized by the eye tracking unit 10 is extracted and used. (2) Obtain the maximum distance (gap) between the upper and lower eyelids, and (3) arrange them in chronological order and store them in memory. From the acquired gap, the steepness at the time of gap reduction (closed eyes) is measured, and a score according to the steepness is added. At the time of blink,
Since the gap sharply decreases, the higher the steepness, the higher the score. (4) Measure the time (the number of frames) until the gap returns to the original value, and add a score according to the number of frames.

Since blinking ends in several frames to several tens of frames, points are added within that range, and deductions are made in other cases. (5) Obtain eye contour data.

As shown in FIG. 7, since the distance between the line connecting both ends of the eye and the outline of the eyelid is large when the eye is open and small when the eye is closed, the smaller the average value of the outline data (the number of pixels in the vertical direction), the higher the score. did. Thereby, the score when blinking increases. (6) When the score added above exceeds the set threshold value, it is determined to be a blink.

As described above, blink detection can be performed accurately and with individual differences eliminated by fuzzy evaluation using a plurality of evaluation items.

[0027]

As described above, according to the present invention, eyes are extracted by fuzzy inference based on a plurality of evaluation items, and the tracking of eyes on an image is performed with higher accuracy than when only pattern matching is performed. It can be realized. Further, by performing blink detection by fuzzy inference, blink detection with high accuracy is possible, and it is possible to detect a swing of the driver, a drowsiness, and the like.

[Brief description of the drawings]

FIG. 1 is a principle diagram of the present invention.

FIG. 2 is a configuration diagram of an embodiment.

FIG. 3 is a diagram showing an example of grayscale image processing;

FIG. 4 is an eye extraction flowchart.

FIG. 5 is an explanatory diagram of the length of a black block.

FIG. 6 is a diagram showing a gap for each frame.

FIG. 7 is a diagram showing an outline of an eye;

FIG. 8 is a flowchart of a blink detection process.

[Explanation of symbols]

1 is an imaging unit 2 is a memory 3 is a standard template 4 is an eye tracking memory 5 is a blink tracking memory 6 is an eye extraction unit 7 is a pattern matching unit 8 is an image processing unit 9 is a recognition unit 10 is an eye tracking unit 11 is a blink Detection unit 12 is tracking unit 13 is judgment unit

Claims (3)

[Claims] 1. An eye candidate position is extracted from an image, a pattern image similar to the eye is extracted from a predetermined region of the image including the candidate position, and the pattern image is compared based on a predetermined evaluation item. An eye extraction device for determining an eye position, comprising: an image processing unit that binarizes an image of a predetermined area including the extracted eye candidate position to extract a pattern image; and a predetermined evaluation item indicating a feature of the eye. And a recognizing unit that performs comparison processing between the plurality of extracted pattern images and recognizes a pattern image having the highest evaluation point as an eye. 2. An eye candidate position is extracted from an image, a pattern image similar to the eye is extracted from a predetermined region of the image including the candidate position, and the pattern image is compared based on a predetermined evaluation item. An eye extraction device for determining an eye position, comprising: an image processing unit that binarizes an image of a predetermined region including the extracted eye candidate position to extract a plurality of pattern images; An eye extraction device, comprising: a recognition unit that gives fuzzy evaluation points to a plurality of extracted pattern images for evaluation items, and recognizes a pattern image with the highest evaluation point as an eye. 3. A blink detection apparatus for detecting blinks by monitoring eyes in a moving image, wherein a plurality of evaluation items characteristically representing blinks are given fuzzy evaluation points, and a total evaluation point is determined. A blink detection device, comprising: a determination unit that determines a blink of an eye when the value reaches a value.