KR20000050501A - Method of face territory extraction using the templates matching with scope condition - Google Patents

Abstract

PURPOSE: A method for extracting face area using effective range condition template matching is provided to use an initial face area extraction module and an effective range condition face area extraction module, thereby reducing matching times in the section using the template matching and rapidly and accurately extracting the face area of mobile images with rapid processing. CONSTITUTION: A method for extracting face area using effective range condition template matching comprises: an initial face area extraction step extracting face areas appeared in one frame using any algorithm; and an effective range condition face area extraction step performing the template matching within limited effective range on the basis of the face area extracted in the initial face area extraction step.

Description

FIELD OF FACE TERRITORY EXTRACTION USING THE TEMPLATES MATCHING WITH SCOPE CONDITION}

The present invention provides an effective face condition face region that performs template matching within a limited effective range based on an initial face region extraction module for extracting face regions appearing in one frame by applying an arbitrary algorithm to a video. The present invention relates to a face region extraction method using an effective range condition template matching to extract a face region from a video quickly and accurately using an extraction module.

In particular, the face region extraction method can be used to track the face once found and to extract the face region of a new person even when a new person appears in the process of extracting the face region. It is about.

In general, the problem of extracting the facial region from the image has been very important as the face recognition problem is greatly raised.

The method of extracting an object region from a video can be divided into a region extraction method using only still image information and a region extraction method using temporal information between frames in a single frame.

[One]. "Pedestrian Detection Using Wavelet Templates" CVPR97, JuneJun 17-19, Michael Oren, MIT

This is a general face region extraction algorithm that pre-configures one or more human face templates to scan and match the entire image area starting from the minimum size of the template to the maximum size.

However, the region extraction method [1] as described above has a disadvantage in that the number of template matching is very large and the processing time is very large even though the template size is small.

[2]. "Automatic Extraction of Face from Color Reversal Film Using Statistical Multistep Filtering Technique" ACCV'95 Second Asian Conf.On Computer Vision. December 5-8

-Extraction of skin color pixels: After converting RGB color system to HSV color domain, skin-color pixels are extracted.

Segmentation of skin-color region: Sementation of the extracted skin-color region using edge detection, hole filling, and gap filling operations.

-Classification of region: Identify the face area using the pattern prepared for evaluation in advance.

However, the extraction area method [2] is not only different from the video because the color information is different depending on the race and the person. Not only the accuracy is reduced by using the information but also the color space is converted from RGB to HVS, which requires a lot of process time.

[3]. Facelt (Visionics Corporation)

It is a toolkit that has the function of automatically extracting the human face area when a person moves on the camera screen and checking who is already registered.

However, the area extraction method [3] is a method of finding a human face in a still image, and using an overall motion of an object using a difference image and a face identification method using a neural network.

The motion caught in the difference image has a problem that it is difficult to extract the face region again in the region where the movement exists while the movement of the body other than the human face is included.

In order to solve the above problems, the present invention applies an algorithm (eg, any one of the conventional region extraction methods [1] [2] [3]) to extract facial regions appearing in one frame. By using the face region extraction module and the effective range condition face region extraction module that performs template matching within the limited effective range based on the face region extracted once, the face of the video with fast processing while reducing the number of matching in the section using template matching The present invention provides a face region extraction method using effective range condition template matching to extract a region quickly and accurately.

On the other hand, after template matching is performed based on the face region extracted in the previous step, the template area having the highest score is designated as the face area, or the matching order is determined based on the template matching type applied in the previous step. By searching for each frame as it is, or by using the latest movement speed for each face instead of performing template matching every captured frame, the face once found can be tracked over time, and the face area can be tracked. The present invention provides a method for extracting a face region using an effective range condition template matching to extract a face region of a new person even when a new person appears in the extraction process.

1 is a flowchart showing an embodiment of a face region extraction method using the present invention validity condition template matching;

FIG. 2 is a flowchart illustrating another example of performing template matching by adaptively accumulating at a moving speed of an object through a limited effective range and an interval between frames for extracting a face region according to an exemplary embodiment of the present invention. FIG.

3 is a diagram for determining an effective range value in one embodiment of the present invention.

4 is a state diagram showing a simple movement matching type in one embodiment of the present invention.

5 is a state diagram showing the type of scale shift matching in one embodiment of the present invention.

Figure 6 is a table showing the similarity distance with each matching type in an embodiment of the present invention.

Hereinafter, an embodiment of a face region extraction method using the present invention effective range condition template matching will be described with reference to the accompanying drawings.

1 is a flowchart showing an embodiment of a method for extracting a face region using an effective range condition template matching according to an embodiment of the present invention.

A method of extracting an object region from a video by using only still image information or using temporal information between frames in a frame,

Extracting face regions appearing in a frame currently input by applying an arbitrary algorithm (eg, any one of the conventional region extraction methods [1] [2] [3]);

Performing template matching of an effective range condition based on the extracted face regions when one or more face regions are extracted by the above;

If even one of the face regions extracted in the initial face region extraction is not successful in the template matching result of the effective range condition, the initial face region extraction is performed again to solve the problem that occurs when the face of the person extracted so far disappears from the screen. Performing;

Performing initial face region extraction again after a frame valid time (cycle) of the face regions extracted by the template matching result of the effective range condition extraction; Proceed to

When the role of the initial face region extraction needs to change the face regions extracted by the coverage condition template matching due to the appearance or exit of a new person, the face regions already extracted through the coverage condition template matching in the previous frame are excluded. The initial face region extraction was performed on only the remaining regions.

FIG. 2 is a flowchart illustrating another example of performing template matching by adaptively accumulating at a moving speed of an object through a limited effective range and an interval between frames for extracting a face region in an embodiment of the present invention. As the progression step,

Time (t) of the currently input frame and time (t ') of the frame from which the face area was extracted in the previous step (t') + {Time interval between frames (Δt) × n (n is the first time changed in the previous step and mapped to the template Comparing and determining a movement / scale in a frame past Δt × n at frame intervals;

As a result of the comparison, if t <t '+ (Δt × n), the time interval Δt between frames is increased to the time t of the currently input frame, and if not t <t' + (Δt × n), the effective range condition Determining whether or not the template matching is successful;

If the validity condition template matching is successful, the time t 'of the frame from which the face region was extracted in the previous step is set to the time t of the currently input frame, and the time t of the currently input frame. Increasing (t + Δt) by adding a time interval [Delta] t between frames;

Comparing n≥1 at the time {t '+ (Δt × n) of the nth captured frame to perform template matching if the validity condition template matching is not successful as a result of the determination;

If the comparison result of the nth captured frame n to perform template matching and the captured frame between the frames is not n≥1, the failed range condition template matching result is detected, and the nth captured frame n to perform template matching is detected. If the captured frame between the frame and the frame is n≥1, decreasing the nth captured frame n to perform template matching (n = n-1);

If the nth captured frame (n) to be subjected to template matching is reduced by performing a template matching by reducing the time interval (Δt) between frames at the time t of the current frame, determining whether the template is successful; To proceed.

As described above, an operation of the face region extraction method using effective range condition template matching as an embodiment of the present invention will be described with reference to FIGS. 1 to 6.

First, when a frame is input, it is determined whether the current one frame is input for the first time.

In this case, if the current one frame is the first input, a random algorithm is applied to extract face regions appearing in the currently input frame, and then template matching of the effective range condition is performed based on one or more face regions.

In addition, if any one of the face regions extracted in the initial face region extraction is not successful in the template matching result of the effective range condition, the initial face region is again solved to solve the problem that occurs when the face of the person extracted so far disappears from the screen. Perform the extraction.

If successful, increase COUNT (COUNT = COUNT + 1) and go to the next frame input step.

In addition, when the frame valid time (period) of the face regions extracted in the initial face region extraction as a result of template matching with the effective range condition is passed (COUNT <period), the initial face region extraction is performed again.

In this case, the initial face region extraction process may be performed on the entire image region, but the face region extracted by matching the effective range condition template in the previous frame is excluded, and the initial face region extraction is performed only on the remaining regions. It was made.

This is because the role of initial face region extraction changes the face regions extracted by coverage condition template matching due to the appearance or exit of a new person, so that the face is added when a face other than the person extracted so far appears. It is intended to be extracted together.

Meanwhile, the initial face region extraction uses only the current frame without using previous face region extraction information, and an algorithm applied to the initial face region extraction process uses a template matching method, difference images, and skin color information.

That is, the template matching method scans the entire area and extracts the face area while converting a predefined template within a predetermined size range.

This has the advantage of extracting all the face areas of people who appear with only one frame, but has the disadvantage of requiring very much processing time.

In addition, the method using the difference image and the skin color information extracts the difference image in two consecutive frames that differ in time, and determines a region having a color belonging to the skin color range among the moving regions as a face region candidate.

This allows very fast processing, but has the disadvantage of causing extraction failure when two frames are required and there is no face movement.

Accordingly, the present invention applies the above algorithm to extract only the face region appearing in the current input frame, and to perform a very fast processing while performing template matching of the effective range condition based on the extracted face regions. will be.

In other words, the scope condition template matching of the present invention defines the scope in advance and performs template matching only within the scope.

The effective range is defined as the maximum range in which a face can be moved within a time interval of a frame, which is defined for each face region appearing in an image.

In addition, the determination of the effective range value refers to a range in which the maximum effective range is extended by an all possible movable distance d in the original face region.

That is, even if the original face area is moved by d / 2 to match the template, it is defined as the maximum movable distance (d) which does not affect the identification of the face area.

When the movable distance d is determined, the time interval Δt between frames is

Becomes

In this case, v is the movable speed and the maximum speed is determined by checking the movement speed of the face area by using the prior data. Therefore, the maximum movable distance of the face area that can occur between two frames is always guaranteed to be smaller than d. .

Further, the movable distance d is not related to the template size, and the movable distance d may be set differently according to the template size to which the time interval Δt between frames is applied to the current frame.

Therefore, as shown in FIGS. 4 and 5 (the thin solid line is the original image, the thick solid line is the image moved by template matching), the matching type for template matching within the actual effective range is simple shift matching and scale shift. Divide by matching.

That is, in the simple mobile matching, nine types are used in the eight-way matching, and the four types are used in the four directions, and in the case of four directions, b, d, f, h, and i are five types. Scale shift matching is divided into 10 types, each of which is divided into an enlarged scale and a reduced scale.

In this case, 19 types are used for 8-way matching as a matching type for finding one face area, and 15 types are used for 4-way matching.

The effective range template matching method may be divided into total matching, motion prediction matching, and motion-speed predictive matching based on the simple moving matching and the scale moving matching.

The overall matching can increase the accuracy by matching all of the nineteen matching types described above in order and then setting the matching area to obtain the highest score as the face area.

The motion prediction matching predicts the current face region based on the movement of the face region in the previous step.

It predicts the most likely matching types and matches them in order, and when the matching score is higher than a certain threshold, the matching area is regarded as the face area and no further matching is performed after that.

That is, the number of matching is small, which is advantageous for high speed, and the prediction of the matching type can be determined in the order in which the similarity distance is close to the matching type matched in the previous step.

At this time, since each matching type represents the movement of the face, the matching type matched in the previous step means the movement of the face region in the previous step.

Assuming that the movements of the faces are continuous, the matching type matched in the previous step is more likely to match the face in the present step.

In other words, the similarity distance with each matching type is shown in FIG. 5 obtained through experiments. Since the same matching type has a distance of 0, the first matching type in prediction is the matching type performed in the previous step. It can be used effectively when the movement that occurred in the previous stage is likely to continue in the next stage.

In addition, the motion-velocity prediction matching shows that some face regions of the faces appearing in the frame during the time interval Δt between two defined frames may move by the maximum movable distance d, while some face regions are slower. Moving bar,

If a face area has a moving speed of d / n during the time interval (Δt) between two frames, this area will be extracted as a moved face area due to movement in the frame captured after Δt × (n / 2). Previously, it is extracted as an existing face area.

Therefore, the effective range template matching is performed on a frame captured after Δt × n time without having to extract a face region for every captured frame, and the frame captured after Δt × n time becomes the nth captured frame afterwards. will be.

This will be described in more detail as follows.

As shown in FIG. 2, the interval between the frames for extracting the limited effective range and the face area is determined by the number of frames captured between the most recently matched frame and the most recently matched frame.

Template matching is performed by adaptively accumulating the moving speed of the object through the limited effective range and the interval between frames for extracting the face region.

First, the time t of the currently input frame is compared with the time t '+ {time interval Δt × n} of the frame from which the face region is extracted in the previous step.

In this case, if the comparison result is t <t '+ (Δt × n), the time interval Δt between frames is increased to the time t of the currently input frame, and if t <t' + (Δt × n), After matching the validity condition template, it is determined whether it is successful.

Subsequently, if the effective range condition template matching is successful, the time t 'of the frame from which the face region is extracted in the previous step is set to the time t of the currently input frame, and the time t of the currently input frame is set. Increase the time interval [Delta] t between frames.

In addition, if the scope condition template matching is not successful, (n≥1) is compared when there is an nth captured frame n to perform template matching.

As a result of the comparison, if the nth captured frame n to be subjected to template matching and the captured frame between the frames is not (n≥1), the failed range condition template matching result is detected.

On the other hand, if the n-th captured frame n for template matching and the captured frame between the frames is (n≥1), the n-th captured frame n for template matching is reduced (n = n−). 1), the template matching is performed by reducing the time interval Δt between frames in the time t of the current frame.

Here, n is changed for the first time in the previous step, and the movement / scale appears in the frame past Δt × n at the frame interval mapped to the template.

That is, when the motion of the face is not continuous, the face may appear beyond the effective radius in the captured frame after Δt × n time.

In other words, as shown in FIGS. 4 and 5, all 19 matching types do not score above the threshold, and in this case, when extraction of the face region from the excessive Δt × (n-1) frame to the Δt frame is successful, Performing effective radius template matching until

This allows the template matching to be performed only a few predefined types of matching, so that the extraction of the face region and the very fast processing are possible.

As described above, the present invention uses a combination of the periodic initial face region extraction and the effective range condition template matching to extract the face region from the entire image area, thereby reducing the number of matching in the section using the template matching and the video with fast processing. Fast and accurate extraction of the face area, while performing template matching based on the face area extracted in the previous step, select the template area with the highest score as the face area, or select the template matching type applied in the previous step. Based on the matching order, the search is performed in that order, or by selecting the frames to be template matched using the latest movement speed for each face without performing template matching every captured frame. So that And, although new figures emerge from the process of extracting the face area, which will be extracted along with the face area of the new figures.

Claims (13)

A method of extracting an object region from a video by using only still image information or using temporal information between frames in a frame, An initial face region extraction step of extracting face regions appearing in one frame by applying an arbitrary algorithm; An effective range condition face region extraction step of performing template matching within a limited effective range based on the face region extracted once by the above steps; A facial region extraction method using effective range condition template matching, characterized by proceeding to. 2. The method of claim 1, wherein the initial face region extraction uses only the current frame without using previous face region extraction information. The method of claim 1, further comprising: performing template matching of an effective range condition using the extracted face regions when one or more face regions are extracted by the initial face region extraction step; Performing initial face region extraction if any one of the face regions extracted in the initial face region extraction step is not successful in the template matching result of the effective range condition; Performing initial face region extraction again after an effective time of the captured frame in initial face region extraction; Facial region extraction method using the effective range condition template matching, characterized in that further comprising. The method of claim 1, wherein the limited effective range is determined as a range in which the face area may appear in the next frame using the maximum range after obtaining the maximum range in which the face can move during the time interval of the frame. A facial region extraction method using effective range condition template matching. The method according to claim 1 or 4, wherein the interval between the limited effective range and the frame for extracting the face region is determined by the number of captured frames between the most recently matched frame and the most recently matched frame of the previous step. Facial region extraction method using effective range condition template matching. The method according to claim 1 or 3, wherein in the initial face region extraction, if the face regions extracted by the coverage condition template matching are changed due to the appearance or exit of a new person, the coverage criteria template matching is already performed in the previous frame. A face region extraction method using effective range condition template matching, wherein the initial face region extraction is performed on only the remaining regions except for the extracted face regions. In a method for performing template matching by adaptively accumulating at the moving speed of an object through a limited effective range and a frame interval for extracting a face region in a video, Comparing the time of the currently input frame with the time interval xn between the time + frame of the frame from which the face region was extracted in the previous step and performing template matching of the effective range condition; Setting the time of the frame from which the face region was extracted in the previous step to the time of the currently input frame and increasing the time interval between the frames at the currently input frame time if the effective range condition template matching is successful; Comparing (n ≧ 1) in the nth captured frame to be subjected to template matching if validity condition template matching is not successful; Performing coverage template matching again by detecting a failed coverage condition template matching result according to the comparison result or by reducing a time interval between frames at the time of the current frame and the nth captured frame to perform template matching; A facial region extraction method using effective range condition template matching, characterized by proceeding to. 8. The method of claim 7, wherein the template matching within the effective range condition comprises a simple moving matching divided into eight directions or four directions and a scale moving matching divided into an enlarged scale and a reduced scale, respectively, in consideration of the moving direction and the size control. Facial region extraction method using range matching template matching, characterized in that the use. 10. The method of claim 7 or 8, wherein the template matching in the effective range condition comprises simple movement matching and scale shift matching for each face region based on the face region extracted in the previous step every frame captured at a predetermined time interval. Do everything, A face region extraction method using effective range condition template matching, wherein the template region having the highest score is determined as the face region. The method of claim 7 or 8, wherein the template matching in the effective range condition determines a matching order based on the matching type applied in the previous step to predict the current face area using the face motion of the previous step. A method for extracting a face region using range matching template matching, characterized in that determining a template region by searching in the predetermined order. The template matching according to claim 7 or 8, wherein the template matching in the effective range condition does not perform template matching on all face regions in every frame captured at a predetermined time interval, and template matching is performed using the latest movement speed for each face. A method for extracting a face region using range matching template matching, comprising selecting a frame to determine a template region. The method of claim 7, wherein when the face disappears from the screen or the face is missed in the template matching process, the face region is re-extracted using the template matching score, When a new person appears, the face region extraction method using the effective range condition template matching is applied again to add the face region of the person and extract the same together. 10. The method of claim 9, wherein in determining the template region, which is the highest matching score, as the face region, if the matching score is greater than or equal to the threshold value, the face region extraction using the effective range condition template matching is stopped. Way.