KR20020019629A - A method of the feature extraction of eyes - Google Patents

Abstract

PURPOSE: A method for extracting eye features is provided to detect precisely and rapidly the feature areas of the eyes by detecting the slope of an image area and applying the image of the eyes to a template model. CONSTITUTION: A gray level image is inputted through a CCD(Charge Coupled device)(10). It is judged that eye information exists in the inputted image(20). In case that image information doesn't exist, face candidate areas are selected through a face detection algorithm(30). Then, as image preprocessing for the eye image is carried out(40), eye features are extracted(50). If eye feature extraction is completed, extracted eye feature information is transmitted.

Description

A method of the feature extraction of eyes}

The present invention relates to an eye feature extraction method for extracting eye features using area gradient, and more particularly, to face recognition, face authentication, face animation, facial expression extraction, user interface, driver alarm, and the like. The present invention relates to a method of extracting eye features to be applied.

In general, in the fields of face recognition, face authentication, facial animation, facial expression extraction, user interface, and driver alarm, eye features are extracted from facial images to improve accuracy.

In particular, in the process of extracting eye features, the segmentation method is mainly used. The segmentation of eye features in the face image must satisfy the following conditions.

1. It should be possible to extract the eye features in real time.

2. Extract the correct eye area.

3. Extract feature areas to minimize noise and light effects.

4. It should be possible to extract the features of the eyes from the tilted face image.

5. Eye features should be able to be extracted from face images turned left and right.

6. Be able to extract eye blink information.

7. Be able to automatically detect eye features.

As a result, the above seven conditions must be satisfied in order to achieve an improvement in efficiency, accuracy, and performance in dividing the eye feature region using the tilt detection method of the region.

In the prior art, a method of extracting eye features from a face image is largely classified into a feature-based matching method and a template-based matching method.

The feature-based matching method separates the feature region of the eye by accurately separating the feature of the feature region using a method of edge, valley, and segmentation. However, the feature-based matching method as described above has the advantage that it can be extracted in real time, but there is a problem that does not satisfy the conditions 2, 3, 4, 5, 6, 7 of the seven conditions.

In addition, the template-based matching method is a method of separating a feature region using a predefined template model, and separates an exact feature region by an energy equation of the defined template model. However, although the convergence speed is determined according to the parameter used for the applied template model, most template models have a problem of slow convergence speed. In addition, the condition 7 is not satisfied due to the initial area, and in the case of conditions 3 and 4, it is difficult to detect the feature area of the eye by separating the eyebrow area into the eye area in applying the template model. There is a problem.

In order to solve the above problems, a combination of two methods, that is, a feature-based matching method and a template-based matching method, has also been proposed. In this method, however, there is a problem that conditions 3 and 4 cannot be solved.

Also, recently, a method of separating a feature region of an eye by applying a wavelet method has been proposed to solve an illumination or noise problem. However, a problem that cannot satisfy condition 1 also occurs in such a method.

The present invention provides a method for accurately and quickly detecting the feature region of an eye by extracting an eye feature, detecting an inclination of an image region through a least square method, and applying the eye image to a template model. .

In the present invention for achieving the above object,

A first step of inputting a gray level image through the CCD;

A second step of determining whether information on eyes is recognized in the input image;

A third step of using a face search algorithm to find an image of the eye when the information on the eye is not detected;

A fourth step of preprocessing the image for the eye;

And a fifth step of transmitting the extracted feature information of the eye.

1 is a flowchart illustrating a process of extracting eye features according to an embodiment of the present invention.

2 is a diagram illustrating a search region for detecting the position of an eye in a face candidate region of the present invention.

3 is a diagram for explaining a least squares method applied in the present invention.

4 is a view for explaining a region separation process in the present invention;

5 is a diagram for describing a binarization pixel as a reference in the region separation process of the present invention.

6 shows a template model of both eyes used in the present invention.

7 shows an elliptic template model used in the present invention.

8 is a view for explaining the validity of the elliptic template model in the present invention

9 is a view for explaining a process of calculating the moving distance of a search area in the present invention.

The above objects and various advantages of the present invention will become more apparent from the preferred embodiments of the present invention described below with reference to the accompanying drawings by those skilled in the art.

Hereinafter, exemplary embodiments of an eye feature extraction method according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a process of extracting eye features according to an embodiment of the present invention, FIG. 2 is a view showing a search area for detecting the position of the eye in the face candidate area of the present invention, and FIG. 4 is a view for explaining a least square method applied to the present invention, FIG. 4 is a view for explaining a region separation process in the present invention, and FIG. 5 is a view for explaining a binarization pixel as a reference in the region separation process of the present invention. 6 is a view showing a template model of both eyes used in the present invention, FIG. 7 is a view showing an elliptic template model used in the present invention, Figure 8 is a view for explaining the validity of the elliptic template model in the present invention 9 is a view for explaining a process of calculating the moving distance of a search area in the present invention.

First, in FIG. 1, a gray level image that can identify the front or both eyes is input through a charge coupled device (CCD), and after the image capture step 10, detects eye characteristics in real time. do. In order to detect the feature of the eye, the image is input through the step 10, and the eye information step 20 determines whether there is information on the eye in the image information. If there is no information, the process proceeds to the step 40 of image preprocessing, and the step 30 of selecting a candidate region of the face through face detection algorism if there is no information, and proceeds to the step 40 of preprocessing the image. In step 40 of pre-processing the image, the feature of eye extraction is completed when the process of sessionization is completed by proceeding with electric noise reduction and error prevention of the region connected with region segmentation. In step 50 of the present invention, when the step 50 is completed, the eye feature extraction method undergoes a process of transmitting region information having eye features.

In step 10, the face search algorithm step 30 is applied to the first image of the gray image input through the CCD. The face search algorithm step 30 uses a hierarchical knowledge-based method using a mosaic method and an edge detection method for detecting the contours or boundary lines of an image in the entire image area. The candidate image of the face is obtained by using a plurality of region selection methods through neural network output after processing the input image by a mosaic method.

For the face candidate regions mentioned above, binarization (binarization) images are initially obtained for each face candidate, and eyes are searched. For the remaining frames, the method moves to the search area using the information of the previous frame.

In the method of searching the eye in the candidate area of the face, an area in which the area of the eye does not deviate is determined as the search area of the eye as shown in FIG. The method of performing image binarization on the determined search region of the eye actively mixes segmentation and edge detection so that the eye features are best represented as shown in a and b of FIG. 3.

At this time, since the influence of the hair or the influence of the lighting is an obstacle to the performance of image binarization, a plurality of threshold values are extracted, but in the present invention, an optimal threshold value method is used to actively cope with the threshold values based on the experimental results. .

As shown in a of FIG. 3, the image of the eye is ideally separated and separated, but in many cases, such as b of FIG. 3. In other words, the lines connected horizontally of the eyes tend to stick. Moreover, if the eyes are closed, the problem may be more serious. For the same reason as above, an eye region may be divided and extracted into an intact eye region.

In the present invention, in order to prevent the horizontal direction from being cut, the entire separation process is performed by looking into the connected area using a method of adjusting the connection strength between pixels. Here, the method of controlling the connection strength can be divided into the connection strength investigation between the horizontal pixels and the connection strength investigation between the vertical pixels. As a result, a level image representing the degree of clarity of the image is generated.

The method of investigating the horizontal connection strength is a constant level of pixels, depending on how many pixels are continuously from left to right as shown in FIG. Was determined. In the same way, a constant level of pixels according to the continuous distribution of pixels from right to left Determine.

Depending on the strength of the connection, the connection level is determined so that the connection can be made even when 1-3 pixels are separated. The above contents are further embodied as follows.

if ( ) then

if (Set level )

else

if (Set level )

And 0 α <1, 1 <β

here Is the nth pixel present from the left.

In this way , Obtain the level image, Obtain the level image.

The connection strength of the vertical region is also obtained by the same method. The difference from the method of obtaining the connection strength of the transverse region is that the change width of α and β is smaller than the connection strength of the horizontal region. The sum of the connection strength of the horizontal area and the connection strength of the vertical area may be briefly illustrated with reference to FIG. 5 to connect to an image generated by the connection strength.

It is necessary to prevent the error of the area which may still remain in the information of the connected area obtained by the above process, and to determine whether the connected area is valid as the eye area. Regions are merged and separated into adjacent regions to achieve a distribution or one-sided normal distribution. The image candidates of the eye that do not meet the above conditions are controlled by eye feature extraction by removing them from the eye candidate area.

The eyes have a long horizontal and short vertical distribution. The slope distribution of the areas of both eyes, that is, the angle formed by the two eyes, forms a symmetrical distribution as shown in FIG. 6. In other words, by determining the slope of the eye region, one method of determining the candidate pair of eyes among the candidates of the connected regions is determined. The example of FIG. 6 corresponds to an inclined face in which the left eye is lowered and the right eye is upward. For convenience of explanation, an example of a person whose both edges of the eye are drastically sag is shown. In fact, in the eyes of ordinary people, the angle formed by the straight line passing through the center of the two eyes is smaller than this.

That is, the slope between the straight lines L1 and L2 ( ) And the slope of the straight lines L1 and L3 ( ) Shows the same result as Equation 1 below.

Therefore, tilt detection in the connected area must be made in order to extract an accurate eye image.

Meanwhile, referring to the binarized image of the eye in FIG. 3, each pixel may be defined as a distribution of pixels corresponding to a vertical axis of a horizontal axis. That is, a mathematical function relationship Y = f (X) expressing a relationship between two variables by predicting or estimating a value of one independent variable and a dependent variable corresponding thereto may be called. Therefore, the distribution of the area pixels of the eye satisfies the following condition indicating the characteristics of the regression line.

(a) E (Y | x) =

(b)

(c) Var ( ) = Var ( ) = ... Var ( )

(d) , , ...

In the above equation, (a) represents a linear relationship, (b) represents normality, (c) represents equal dispersion, and (d) represents independence. When a straight line passing through the area is obtained by the least square method, which is one of the inference methods of the regression line, it is possible to infer the straight line (b) having an error as shown in Equation 2 below.

(a) E (Y | x) = α + βxi + ei

(b)

In Equation 2, α, β is inferred by the following equation (3).

,

Of course, the slope to be.

In order to detect the position of the eye, it is grouped into candidate pairs called eyes, and the template energy ( )

First, the energy according to the degree of normal distribution of the region of FIGS. ) Next, the energy using the method according to the distribution of the slope of the eye ( ) And energy by the heuristic rule ) That is, if the candidate pair of eyes is similar in area or width and length and the area of the eye, energy is added according to the degree of symmetry of the left and right pixels on the gray level and the presence of the area of the eyebrow on the eye.

if ( <Threshold1) continue

if ( <Threshold2) continue

if ( <Threshold3) continue

= + +

, n = number of candidates

In this way, the candidate with the highest energy value is selected as the eye position.

The necessity of extracting more accurate features of the eye is because it must be able to express eye blinking information including any state of the eye, that is, the state of being closed, floating, or only opening one side. The region extracted by the method described so far can be used as eye position information, but it cannot be said to have accurate region information. Therefore, in order to extract accurate eye features from the eye area information, it is necessary to extend the processing method of the face image in various directions. There is a need for a treatment method that can be extracted. To do this, use the following method.

The eye can be expressed in the form of a rectangle, an ellipse, or a specific function. Among the above methods, the present invention selects a method of extracting eye features in an elliptic form. When defining the characteristics of an eye as an elliptic template, it is important to determine how the energy function is defined and to determine which data to take and which interpolation method to use to interpolate the data. In the present invention, the energy function of the eye is defined by the method by Principal components analysis (PCA). The defined template model applied to the eye is an ellipse as shown in FIG. Here x1 represents the horizontal axis, x2 represents the vertical axis, c1 represents the longest length from the origin to the ellipse in the elliptic template model applied in the present invention and c2 represents the shortest length from the origin to the ellipse in the elliptic template model applied in the present invention. An example of applying the elliptic template model of FIG. 7 to an actual binarized fill cell can be seen in FIG. 8. 8 is a view for explaining the validity of the elliptic template model in the present invention by using the statistics of the data of the pixel of Table 1 in FIG. Information can be extracted.

Statistic X1 X2 N (Pixel) N N Mean Standard deviation Variance Correlation r

Table 1 shows data statistics of pixels.

The tracking method used in the above method is to find the most similar area in the next frame by using the feature of the area found in the previous frame. So we need to determine the search area.

On the other hand, in the movement of the search area, since the movement processing time is proportional to the area of the search area, the search area should be limited to the minimum area that can include the feature in the next frame. Movement of the search area is possible by assigning weights to the face law to the law of motion of the object and the image center. That is, as the distance from the center increases, the ratio of the moving distance of the face is gradually reduced as shown in FIG. 9.

FIG. 9 is a view for explaining a process of calculating a moving distance of a search area in the present invention. In FIG. 9, an arc portion of a circle refers to a cross section of a hemisphere. When the image is texture mapped to the surface of the sphere, point A to point B are the distances on the actual image. The distance from point A to point C is the distance used to move the actual area as described above.

According to this law of motion, the processing time of the frame (FRAME) ) Is compared with the previous frame, and the distance ), The speed at which the binocular moves in all frames ( ) Is applied to Equation 5, this frame acceleration ( ), Vector ( ) Is determined according to the movement speed of the obtained eye.

As described above, the present invention is an invention of eye feature extraction, which is widely used for face recognition, face authentication, face animation, facial expression extraction, user interface, and especially driver alarm, inclination detection and ellipse of the area by the least square method. The use of a template model has the effect of improving efficiency, accuracy and performance.

Claims (10)

In the eye feature extraction method for extracting eye features using area gradient, A first step of inputting a gray level image through the CCD; A second step of determining whether information on eyes is recognized in the input image; A third step of using a face search algorithm to find an image of the eye when the information on the eye is not detected; A fourth step of preprocessing the image for the eye; An eye feature extraction method comprising a fifth step of transmitting the extracted eye feature information. The method of claim 1, wherein the third step is A third method of mosaicing the input image of the gray level using the mosaic method and the edge detection method using a hierarchical knowledge-based method through a mosaic method and an edge detection method for finding the contour line or region boundary of an image Stage 1; 3. The eye feature extraction method of claim 2, wherein the candidate region of the face is obtained by using a plurality of region selection methods by neural network output after the mosaic processing. The method of claim 1, wherein the fourth step is Obtaining a binarization image for the candidate region of the face; Step 4-2 determining the search region of the eye from the candidate region of the face which has become the binarization image; 4-3, performing image binarization on the determined search region of the eye; Adjusting connection strength to prevent separation of the eye area when the image is binarized of the search area of the eye; 4-5 step of generating a level image as a result value by adjusting the connection strength; Steps 4-6 to prevent an error in the region image connected through the level image and determine whether the connected region is valid as an eye region; Steps 4 to 7 obtaining a template energy by binding candidate pairs of possible eyes for position detection of eyes when there are a number of valid images in the eye region; Steps 4-8 for determining candidate eye pairs having the largest template energy values as eye positions; 4 to 9, wherein the eye feature is extracted using a method of expressing the eye shape to obtain an accurate region image from the candidate pair of eyes determined as the eye position. The method of claim 3, wherein the image binarization is performed by using a segmentation method and an edge detection method. The eye feature extraction method of claim 3, wherein the connection strength control method comprises a connection strength investigation method between horizontal pixels and a vertical strength connection strength investigation method. The eye feature extraction method of claim 3, wherein the slope of the region is determined by using a least square method to determine a candidate pair of eyes among candidates of the connected regions. The method according to claim 3, wherein the energy according to the degree of the normal distribution and the energy based on the heuristic rule and the energy according to the heuristic rule are added when obtaining the template energy for the possible candidate pairs for detecting the position of the eye. The eye feature extraction method in which the candidate pair of eyes with the highest energy is determined by the eye position. The method of claim 3, wherein the eye shape expression method uses an elliptic template model. In the eye feature extraction method for extracting eye features using area gradient, A first step of inputting a gray level image through the CCD; A second step of determining whether information on eyes is recognized in the input image; A third step of pre-processing the image of the eye when the information on the eye is identified; The eye feature extraction method comprising the fourth step of transmitting the extracted eye feature information. A first step of comparing a processing time of the current image frame with the current image frame and a previous image frame to obtain a moving distance of the midpoint of the two eyes and a midpoint moving speed of the two eyes generated when moving from the previous image frame to the current image frame; A method of extracting eye features comprising a second step of calculating a frame acceleration and a velocity vector to determine a next region in which two eye centers are to be located according to a movement speed of the eye.