KR20030073879A - Realtime face detection and moving tracing method - Google Patents

Abstract

PURPOSE:A method for detecting a face and tracking motion in real time is provided to detect and track a face reliable and strong against a change in the surrounding environment. CONSTITUTION:An image captured through a camera is inputted(S1). It is judged whether the inputted image is an image inputted for the first time or a transformed image of the previously inputted image(S2). In the case of the image inputted for the first time, face candidate areas are selected by a color deciding method using a color model of a face and by oval fitting of the face using a direction image(S3-S4). A final candidate area is selected(S5). Parameters related to position, size, shape, and color of the face are extracted(S7). Motion of the face is tracked from images inputted afterward by the color deciding method using the color model of the face(S6).

Description

REALTIME FACE DETECTION AND MOVING TRACING METHOD}

The present invention relates to a real-time face detection and motion tracking method, and more particularly to a real-time face detection and motion tracking method for detecting a face in real time and tracking its movement in a limited computing power and embedded system environment, such as a robot. It is about.

Recently, application fields of vision processing for detecting, tracking, and recognizing faces from images inputted by cameras (CMOS, CCD, ...) are gradually expanding to home robots, game machines, mobile phones, and PDAs. There is an increasing need for a method of effectively detecting, tracking and recognizing faces in portable or mobile devices.

That is, such portable or mobile devices require real-time processing while being limited in size and computing power, and thus require a method of reducing the reliability of the operation while consuming less computing power.

For example, assuming that the robot and the user are talking, the robot faces the face according to the direction of the movement according to the movement of the talker (especially, as the face of the talker moves, since the user is staring at the other's face in the chat state). You need to move so that you can stare accurately.

The image processing method for face detection and tracking as described above is very important as a previous step in the field of face identification, face expression recognition, and the like. Image processing for face detection, tracking, and recognition In order for the method to be applied in a real environment, several conditions must be met.

First, real-time processing must be possible, second, there must be no special lighting conditions, third, the user need not wear or paint special equipment, and fourth, low cost.

In particular, real-time processing power and cost are the most important conditions for application to portable devices such as home robots, game machines, mobile phones, and PDAs.

In the conventional technology, a method of detecting and tracking a face includes a method of tracking a face by continuously obtaining a differential image of a continuous image, a method of marking a mark on a main part of the face, and tracking the light. There is a method using optical flow, a method of extracting facial contours, a method of obtaining a facial region using a color model, and the like.

However, it is difficult to apply the background complexity or dynamic background when the difference image is used, and it is difficult to apply the mark on the main part of the face in the real environment. There is a problem that is not suitable for real time processing because it requires time.

In addition, the method of segmenting the face region using the color model is sensitive to the external environment because it depends only on the color, and all of the above methods are applied without discriminating the detection and the tracking operation of the face, thereby detecting and tracking the face according to the situation. There is a problem that different characteristics and effects may appear in operation.

Therefore, the present invention has been created to solve the above-mentioned conventional problems, real-time face detection and tracking to detect the face in real time in a limited computing power and embedded system environment, such as a robot The purpose is to provide a motion tracking method.

The present invention for achieving the above object, the first step of receiving an image captured by the camera; Determining whether the input image is a first input image or a modified image of a previously input image; A third step of detecting a face region from the image and detecting and storing a related parameter of the face region when the input image is the first input image; When the input image is a modified image of a previously input image, a fourth step of tracking a face movement by referring to parameter information of a face region already detected and stored is performed.

1 is a flow chart showing a real time face detection and motion tracking method according to the present invention.

2 is an exemplary view for explaining a method of selecting a face candidate region by a color determining method according to the present invention.

3 is an exemplary view for explaining a method of selecting a face candidate region by elliptic fitting according to the present invention.

4 is an exemplary view of a face region finally detected by the degree of overlap of two regions in the face candidate region detected by FIGS. 2 and 3;

5 is an exemplary view for explaining a face tracking method by the color probabilistic method according to the present invention.

The present invention detects a face by simultaneously using the color information of the face and the shape of the face in order to be applicable to portable devices such as home robots, game machines, mobile phones, PDAs, etc. By tracking, the reliability of the detection operation and the tracking operation is increased, and it is robust to external environment changes, and the calculation amount is reduced by approximating to an integer when coding the algorithm of each operation, thereby real-time operation is possible.

That is, the present invention distinguishes the face detection process from the face detection process and applies the method having the most effective characteristics in each process. In particular, the face tracking process is robust to environmental changes and has a low probability of color calculation. By using the conventional method, the reliability and efficiency of face detection and tracking can be improved.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. Many specific details, such as specific process flows, are provided in the following description to provide a more general understanding of the invention, and it is obvious to those skilled in the art that the present invention may be practiced without these specific details. something to do.

In addition, detailed description of well-known functions and structures which are determined to unnecessarily obscure the subject matter of the present invention will be omitted.

1 is a flow chart illustrating a real-time face detection and motion tracking method according to the present invention, receiving a captured image through a camera (CCD, CMOS, ...) (S1), whether or not a face is detected from the previous step Determine (S2).

In other words, if the image input through the camera is not the first image, the face position and various parameters (size, shape, color, etc.) have already been detected and stored from the previous image. In order to track the movement of the first input image, the face position and the parameters are selected to be detected.

As a result of the determination, if the image is input for the first time, the face candidate region is selected from a color-deterministic method using the color model of the face (S3), and the face candidate region by elliptic fitting of the face using a direction image. To select (S4).

Next, considering the face candidate areas detected by the above two methods simultaneously, the candidate areas having the highest scores of a predetermined level or more are finally selected (S5), and the position, size, shape, and color of the face from the selected final face areas. Extract the parameters related to the (S7).

The extracted face region parameter is used as a reference material for tracking the movement of a face in an input image afterwards. When the face region is detected as described above, the input image is a color probabilistic method using a color model of the face. By tracking the face (S6), the process of extracting and updating the face position and detection-related parameters every time is repeated (S7).

Hereinafter, the specific operation and action of the present invention made as described above will be described.

First, a color model of a face serving as a reference for the color-deterministic method and the stochastic method used in the face detection and tracking will be described.

The RGB image input from the camera is sensitive to changes in brightness, so use the normalized r, g values (r = R / (R + G + B), g = G / (R + G + B)) The average and covariance of the normalized r, g are calculated from the estimated face images to obtain the attributes of the color model of the face.

Thereafter, when the colors are all normalized values, the mean (m) and the covariance (K) for r and g are expressed by Equation 1 below.

,

The distance value d for the color c for the color model of the face is expressed by the following equation.

, d = [cm] ^ TK ^ -1 [cm]

Next, a process of selecting a face candidate by a color determination method using the color model will be described in detail.

First, a portion of the face color region and the non-face color region with respect to a given image may be obtained using the mean value m and the covariance K of the face color model.

That is, the color (c (x, y)) for each pixel (x, y) of the input image and the distance value (d (x, y)) for the color model of the face fall within a predetermined range (k). If it comes, it is determined by the color of the face.

In Equation 3, an area that is a face has a value of 1 (that is, a value of 1 when the distance value d (x, y) is within a predetermined range k), and an area other than a face has a value of 0 ( That is, when the distance value d (x, y) is out of a predetermined range k, the distance value d (x, y) has a value of zero.

In order to select a face candidate region through the above process, as shown in FIG. 2, after labeling each region, a face rectangle is selected by obtaining a minimum rectangle including the region.

Next, an elliptic fitting of a face using a direction image and a process of selecting a face candidate are described.

First, the RGB image input from the camera is converted into a black and white image as shown in FIG.

The brightness information of the converted black and white image can be viewed as height information about a plane, and for each pixel, a plane best suited for the surface and a vector perpendicular to the plane, and a tangent vector perpendicular to and on the plane of the obtained vertical vector. Can be obtained, and the result is shown in the form of the directional image of FIG.

Modeling the face as an ellipse enables the face to be represented by its long axis and short axis length, and given the inclination (m), long axis (b), and short axis value (a) of the tangential vector, the center position of the face (x_c, y_c) It can be obtained from the following equation (4).

,

That is, the elliptic center (x_c, y_c) is obtained for one pixel of a given direction image, and the process of accumulating the tangent vector by multiplying the magnitude of the tangent vector by weight is performed on the entire image, and the most concentrated and constant of the accumulated values. The place above the value is selected as a face candidate suitable for the corresponding ellipse as shown in FIG.

Accordingly, the result is expressed as a face candidate region for the elliptic template of FIG.

Next, in order to detect the most reliable area as the final face from the face candidate areas selected by the above methods, the face candidates by selecting the area of the face candidate by the color-deterministic method, the directional image and the elliptic fitting of the face The degree of overlapping of the selected regions of S is scored, and the candidate region having the highest score over a predetermined level is determined as the final face detection region as shown in FIG.

When the face candidate area by the color deterministic method is C and the face candidate area by the elliptic fitting is E, the degree of overlap between the two areas can be expressed by the following equation (5).

or,

Next, when the final face area is detected in the candidate area detected by the two methods as described above, the face is tracked by the color probabilistic method for the image to be input later.

Face tracking by the color probabilistic method is a method of tracking a face by selecting n child pixels from n parent pixels with respect to an input image as shown in FIG. 5.

To select a child sample, the probability P1 of the color model attribute of the parent pixels as the factor and the probability P2 of the color model and the color distance of the given face are calculated. Probability value P is calculated by P = P1 * P2), and child pixels are selected probabilistically in proportion to the probability value P.

Next, the position of the face is calculated by calculating the average position of the selected pixel, and when the sum of the probabilities of the child pixels has a predetermined value or more, it is determined that the face tracking is effective.

Equation 6 below is a formula for obtaining a probability value, and FIG. 5 shows an example of face tracking by a color probabilistic method.

here,

In Equation 6, d1 and d2 are color distances obtained by Equation 2, d1 is a distance between the pixel color and the color model of the face, and d2 is a color model of the pixel color and the sign pixel. Color distance.

Next, when detected or newly tracked as described above, the position and various parameter information of the face are updated to a new value and used as information for tracking the movement of the face from the next image.

As described above, the real-time face detection and motion tracking method of the present invention has an effect of detecting a face in real time in a limited computing power and embedded system environment such as a robot to track its movement.

In addition, the present invention detects the face by using the color information and shape information of the face at the same time, by tracking the detected face by the color probabilistic method, to detect and track the face area that is robust to the change of the surrounding environment in real time It has the effect of making it possible.

In addition, since the present invention can operate with a small amount of computing power, the present invention can be applied to a portable device such as a robot, a game machine, a mobile phone, a PDA, and the like, thereby creating a new added value.

Claims (8)

A first step of receiving an image captured by the camera; Determining whether the input image is a first input image or a modified image of a previously input image; A third step of detecting a face region from the image and detecting and storing a related parameter of the face region when the input image is the first input image; If the input image is a modified image of a previously input image, a real-time face detection and motion tracking method comprising a fourth step of tracking a face movement by referring to parameter information of a face region already detected and stored . The method of claim 1, wherein the detecting of the face region from the first input image comprises: A first step of selecting a face candidate region of the image from a color determination method using a face color model; A second process of selecting a face candidate region of the image by elliptic fitting of a face using a direction image; And a third process of selecting the final face region to the extent that the two candidate regions overlap by the two methods of the color-deterministic method and the elliptic fitting. The method of claim 1, wherein the face motion tracking from the input image is tracked by a color probabilistic method using a color model of a face. The real-time face detection and motion tracking method of claim 1, wherein the parameter information detected from the face region includes information such as the position, size, shape, and color of the face. The color model of claim 2, wherein the color model of the face is obtained by calculating an average (m) and a covariance (K) of values (r, g) of normalizing an RGB image from a plurality of face images estimated as faces, and obtaining attributes. The distance value (d) for the color (c) for the color model is obtained by the following equation. (Mathematical formula) , d = [cm] ^ TK ^ -1 [cm] The method of claim 2, wherein the color determination method using the face color model includes a color c (x, y) for each pixel (x, y) of an input image and a distance value (r) for the color model of the face. real-time face detection, characterized in that if the d (x, y)) falls within a predetermined range (k), the face color is determined, and after labeling each area, the face candidate area is selected by obtaining a minimum rectangle including the area. Motion tracking method. The method of claim 2, wherein the elliptic fitting method of the face using the directional image is performed by modeling the face as an ellipse, and applying a gradient (m), a long axis value (b), and a short axis value (a) of the tangent vector of the modeled ellipse. The central position (x_c, y_c) of the face with respect to one pixel of the given direction image is obtained by the following equation, and the process of multiplying the magnitude of the tangent vector by the weight by weight is accumulated for the entire image, Real-time face detection and motion tracking method characterized in that the most concentrated of the values and a predetermined value or more as a face candidate suitable for the ellipse. (Mathematical formula) , The method according to claim 2, wherein the method for detecting the most reliable region as the final face from the face candidate region selected by the color determining method and the elliptic fitting comprises: a selection region of the face candidate by the color determining method; Real-time face detection and motion tracking, characterized by scoring the overlapping degree of the selection region of the face candidate by the direction image and the elliptic fitting of the face to determine the candidate region having the highest score over a predetermined level as the final face detection region. Way.