KR20030050617A - The system and method of face skin area extraction using combined information with color and edge - Google Patents

Abstract

PURPOSE: A system for extracting a face color area using edge and color information, and a method thereof are provided to improve the accuracy of face authentication by preventing a face area from merging with a background area. CONSTITUTION: A color area dividing unit(10) detects edge information from an input color image and divides areas from an entire image by the edge information. An isolated area forming unit(11) makes a background area and a face area binary for forming each isolated area. A skin color area selecting unit(12) extracts the isolated area adjacent to a center position of an image in the isolated areas for selecting an area satisfying conditions of mean gray level, size information, and color distribution as a representative face candidate area. An image merging unit(13) extracts the areas satisfying the conditions of the color and position information from the adjacent isolated areas based on the mean color distribution and position information of the selected representative face candidate area for merging images, so that the image merging unit extracts a face color area.

Description

The system and method of face skin area extraction using combined information with color and edge}

The present invention relates to face extraction corresponding to preprocessing in online face recognition, and more particularly, by combining edge information and color information from a continuous input image, and verifying, merging, and re-binarization of candidate regions without color information of a model. By extracting the face area by adapting the color of the input image, the edge / color information can be improved to prevent the human face area from being merged with the background area with similar color. The present invention relates to a facial skin color area detection system used and a method thereof.

As the high-speed Internet is popularized, the spread of computers is rapidly progressing and the IT industry is gradually being activated. Accordingly, the demand for security and authentication of computer users is increasing, and authentication software using biometric information such as fingerprints, irises, signatures, faces, and voices is gradually commercialized and spread.

Among them, fingerprint authentication software is most commonly used and commercialized, but recognition results depend on devices for fingerprint authentication, and the reliability of fingerprint recognition results decreases according to a person's occupation. There is a limit to development. Therefore, rather than constructing an authentication system using only one biometric information, an attempt has been made to implement an authentication system using multiple biometric information.

In realizing a multi-biometric authentication system, a face recognition system has an advantage in that a user can perform authentication without rejection, and can also use a PC camera, which is a trend that is currently installed in a computer as an input camera.

If the classification of research in the face recognition field is classified step by step, it is a study to accurately extract the face region from the input image, and a study to determine whether the face information in the database is the same through the interpretation and analysis of the face image (certification) ), A study for determining whether the input face information is the same as the face information in the database, and a study for recognizing facial expressions by analyzing facial features.

As the first step to recognize the face as described above, the development of the face extraction method should be preceded.

Initially, the face extraction methods assume that only the face region is included in the input image in the lab environment or that there is only a single color tone background. However, when developing a face recognition system aiming at practical use, it is possible to obtain robust face extraction results in various environments such as background, lighting condition, face and camera distance, input face up, down, left and right angles, facial expressions, camera input characteristics, etc. Should be

Information used to extract a face using a camera includes motion information in a moving image, edge information, color information, and contrast information in a still image.

The face extraction method using edge information mainly extracts the edges from the face image, creates an isolated region, extracts a specific shape such as eyes from the isolated region through prior knowledge and various verifications, and based on the face components How to extract. This method works well for men who do not wear glasses, but it is very likely to fail to extract specific isolated regions in women with various hairstyles and glasses.

The face extraction method using color information mainly registers the skin color of a person and uses the registered skin color information to find the skin color from a newly input image. This method produces satisfactory extraction results when the input device (camera) is the same and the data is taken under the same lighting conditions, but good performance is expected when the camera's performance and background conditions (lighting, background color, etc.) are different. Hard.

The face extraction method using contrast information mainly extracts a face region using a pamphlet, or extracts a face region by applying a Gaber filter or a wavelet filter. This method takes longer than other methods, but it can extract the face area relatively accurately.

However, this method also fundamentally uses the contrast feature, which is highly likely to cause an error under varying contrast of the input image and various backgrounds. As such, a method of extracting a face using only one information has a problem in that it cannot adapt to all environmental changes and produce a good result.

An object of the present invention is to combine the edge information and color information from a continuous input image, and then to extract the face region by adapting to the color of the input image without color information of the model through verification, merging and re-binarization of candidate regions. The present invention provides a system for detecting a facial skin color region using edge / color information that can improve the accuracy of authentication by preventing a human face region from being merged with a background region having a similar color.

The facial skin color region extraction system of the present invention for achieving the above object is a detection device for detecting the skin color region of the face from the color image, detecting the edge information in the input color image, the area from the entire image by the edge information Color region dividing means for dividing the image; Isolated region generating means for binarizing a background region and a face region of the color regions divided by the edge information to generate respective isolated regions; Skin color region selection means for extracting an isolated region close to the center of the image from the isolated regions and selecting a region satisfying the condition of average contrast, size information and color distribution as the skin color isolated region; And extracting an area that satisfies the condition of color and location information of neighboring isolated areas based on the average color distribution and position information of the skin color isolated area selected by the skin color area selection means, and merging the face by merging the images. And image merge means for detecting a skin color region.

In addition, the method for extracting the facial skin color region according to the present invention includes a method for extracting a facial skin color region from an input image image under the control of the facial skin color region extraction system, the method comprising: detecting edge information from the input image image; Dividing a background area and a face area into isolated areas based on the detected edge information; Determining a representative face candidate region among the isolated regions of a predetermined size or more closest to the center of the image to find a face candidate region among the isolated regions displayed after edge information detection; And merging images by extracting regions satisfying the conditions of color and position information of neighboring isolated regions based on the average color distribution and position information of the representative face candidate regions. It is characterized in that the extraction.

According to the above configuration and method, the present invention extracts the color distribution of the face region from the input region from the isolated region using the edge to obtain the color distribution of the face region which can be adapted to the color change of each input image, and is also isolated. The accuracy of face authentication can be improved by detecting areas of facial skin color by separating areas having the same color but not existing in the center of the image by the merge conditions between the areas.

1 is a block circuit diagram of a facial skin color region extractor according to the present invention;

2 is a flowchart illustrating a method of extracting a facial skin color region according to the present invention;

3 is a mask for obtaining adaptive horizontal and vertical edges,

4 is a sampling area setting for extracting the color distribution of the skin color area;

5 is a diagram illustrating an experiment input image;

6 is a diagram showing adaptive horizontal and vertical edge detection results;

7 is a diagram illustrating a result of extracting an isolated region;

8 is a diagram showing a result of an isolated region determination.

* Description of the symbols for the main parts of the drawings *

10; color region divider 11; isolated region generator

12 skin color selection unit 13 image merge unit

14; color distribution extracting section

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. 1 is a block diagram of a facial skin color area detector for presenting the present invention. The present invention efficiently combines edge information and color information from a continuous input image, and then validates, merges, and rebinarizes a candidate area. Disclosed is a technique for detecting a face region by adapting the color of an input image without color information.

Referring to the drawings, the facial skin color region extraction system combining the edge information and the color information of the present invention includes a color region division unit 10, an isolated region generation unit 11, a skin color region determination unit 12, and an image merging unit 13 ) And a color distribution extractor 14.

The color region dividing unit 10 detects edge information from the input color image, and divides the edge information into a background region and a face region by the edge information. In the present invention, the edge is detected through an adaptive edge detector in the entire image. Divides the background area and the face area.

The isolated region generating unit 11 binarizes the background region and the face region divided by the edge information to generate respective isolated regions.

The skin color region selection unit 12 extracts an isolated region close to the center of the image from the isolated regions and selects a region satisfying the conditions of average contrast, size information, and color distribution as a representative face candidate region.

The image merging unit 13 satisfies the conditions of the color and position information of neighboring isolated regions based on the average color distribution and position information of the representative face candidate region selected by the skin color region selecting unit 12. Extracts the face skin color region by merging the images.

The color distribution extractor 14 extracts the face region from the entire image when an unclear image that cannot divide the background region and the face region is input by the color region divider 10. The color distribution extractor 14 samples the center position from the entire input image, extracts the color information of the skin color region from the sampled center position and binarizes it, and then extracts the largest region as the face region after extracting the isolated region. The skin color region of the face is detected from the full color image.

The present invention can extract the facial skin color region combining edge information and color information under the control of the above system, and the extraction method related to this will be described with reference to the flowchart of FIG. 2.

The present invention detects edge information for distinguishing a face region and a background region from an entire image under the control of the color region dividing unit 10 with respect to the input image.

This edge information is performed by an adaptive horizontal and vertical edge detection algorithm on the input image. The edge detection algorithm can execute the algorithm through an adaptive edge detector. That is, in order to satisfy the assumption that the face and background region of a person basically used in the present invention can be separated by an edge.

According to the present invention, if the camera is in a normal state and in focus, the boundary, that is, the edge, can be clearly distinguished between the face area and the background area in the input image, but the edge information may be broken according to the clothes color or lighting state of the person. To prevent this, use horizontal and vertical edge detectors that can extract more horizontal edges than commonly used edge detection techniques.

The reason for using the adaptive horizontal and vertical edge detectors that can extract a lot of horizontal edges is that when the face and background areas of a human are separated, the intensity of the horizontal edge is weaker than that of the vertical edge. Because it is not.

In general, when the edge strength value is measured and then the edge is to be determined, it is mainly used to determine by using a fixed threshold value. This method is used according to the light and dark of the input image, that is, the light change. The disadvantage is that the amount of edges cannot be adjusted.

In order to solve this disadvantage, the present invention uses an adaptive edge detector that determines a threshold value according to a local feature on a mask applied to an image.

temp = abs (h) + abs (v);

sum = sum / 64;

if (sum == 0.) sum = 1;

if (temp / sum> 1.0) then edge;

else non-edge;

Analyzing the edge detection algorithm, the vertical edge is obtained from V in FIG. 3, the horizontal edge is H, and the horizontal and vertical edge intensities are SUM. As shown in FIG. 3, the strength of the vertical edge is multiplied by 0.5 to lower the edge strength, and the strength of the horizontal edge is multiplied by 2.0 to increase the edge strength.

The reason for adjusting the strength of the vertical edge and the horizontal edge is that the horizontal edge is generally detected weaker than the vertical edge, so that the sharpness of the horizontal edge is compensated so that the overall vertical and horizontal edges can be kept uniform. It is to.

In addition, SUM is equal to the result of multiplying 16 over the entire mask, so dividing by 16 yields the average edge strength, but divided by 64 to extract more edges. That is, in the above edge determination condition, the smaller the sum value, the more edges are extracted, and the larger the temp value, the more edges are extracted.

Using the above-described adaptive horizontal and vertical edge detectors, edge detection results that are relatively insensitive to changes in the brightness of an image can be obtained. As a result of applying the adaptive horizontal edge detector to the input image of FIG. 5, the input image is binarized into an edge portion and a pixel portion as shown in FIG. 6.

When the binarization is performed as in step 21, an eight-way labeling algorithm is performed under the control of the isolated region generating unit 11 to generate an isolated region. (S22-S23)

The present invention obtains the average color distribution, position information and size information in the skin color region selecting section 12 for each isolated region generated by the above-described method. FIG. 7 illustrates a result of performing an 8-way labeling algorithm on the binarized image of FIG. 6.

In step 23, when various pieces of information defined in the present invention are obtained for each isolated region, the isolated region closest to the center point of the input image is extracted from the above-described isolated regions to extract the representative face candidate region.

This is based on the assumption that a typical user looks at the camera and tries to move his or her face to the center of the camera when attempting to authenticate the face.

This assumption is that when the purpose of the present invention is to manage computer use through facial authentication, a favorable user will expose his / her face to the camera as accurately as possible in order to obtain a high result of facial authentication. You will try to hide your face.

Therefore, the face authentication system should configure the system considering only the favorable users, and it is not necessary to consider the assumption that is not necessary to deal with the malicious user.

Under these assumptions, if the face is placed in the center of the input image, it is very likely that the isolated region close to the center of the input image among the previously isolated islands is a face region, so that the skin distribution of the isolated region closest to the center of the input image is input. It can be determined by the skin distribution of the injured face.

However, sometimes, due to some errors, the hair or background area is located in the center of the input image to determine the skin color area. To eliminate this error, the skin color area selection unit 12 targets the isolated areas closest to the center of the current image. In turn, the following conditions are checked:

In the present invention, the criterion for determining the representative face candidate region is possible when the following conditions are satisfied.

Condition 1) Is the average contrast of skin color islands darker than the average contrast of other islands?

Condition 2) Does the size of the skin color islands have an area above and below a certain size?

Condition 3) Does the mean R of skin color islands satisfy the condition of R> G, R> B for mean G, B?

The above condition 1) is a condition to check the case where the predetermined isolated area is the hair area. Generally, since Korean hair is relatively black, it is used to remove the result that the hair area is determined as the skin color area.

Condition 2) is a condition for eliminating small noise areas that are determined as skin color islands while the adaptive horizontal and vertical edge detector detects relatively many edges in the input image. Condition to remove the case.

In the present invention, the horizontal and vertical length of the input image size is determined by dividing the size divided by 4 as the minimum area (that is, if the input image has a resolution of 320 * 240 (320 + 240) / 4 = 140), the maximum area is determined. Determined the maximum area by multiplying the horizontal vertical length by 20.

Finally, condition 3) shows that the color value of R in each pixel is higher than that of G or B. Therefore, the average color distribution of skin color isolated area must satisfy the above condition. It can be determined.

As described above, if the region closest to the center of the image does not satisfy the three conditions defined above, the regions satisfying the above three conditions are extracted by sequentially examining the isolated regions close to the input image. The extracted region is determined as the representative face candidate region (S24).

8 shows an example in which the representative face candidate region is determined through the skin color isolated region determination condition.

When the representative face candidate region is determined according to the conditions as described above, the next step performs the merging step under the control of the image merging unit 13 (S25).

The image merging unit 13 merges the image by referring to the position of the isolated region (representative face candidate region) and the color distribution of the isolated region in terms of the merging.

That is, although only one skin color isolated region may exist among the binarized isolated regions in the input image, since the adaptive horizontal and vertical edge detectors extract edges strongly, the skin color region may be separated into several regions.

Therefore, when the skin color region is separated into several regions, a process of combining into one skin color region is necessary. Therefore, in the present invention, the following conditions are defined in order to merge the skin color regions, and the skin color regions are examined according to the defined conditions.

In the present invention, the skin color can be merged when the following conditions are satisfied for merging the skin color regions.

Condition 1) Investigate whether they have color values similar to the average R, G, and B values of the found skin color islands.

Condition 2) Investigation of the location adjacent to the found skin color islands

Condition 1) uses the assumption that the skin color distribution of the same person has uniform R, G and B values. In general, there may be a noise region similar to the human skin color region in the background region, so the candidate region should have a distribution very similar to the skin color isolation region found in order to merge the skin color isolation regions.

In the present invention, it was examined whether the conditions of R-Th <Rinput <R + Th, G Th <Ginput <G + Th, B Th <Binput <B + Th are satisfied. In this case, R, G, and B are average R, G, and B values of the isolated islands, and Rinput, Ginput, and Binput are average R, G, and B values of candidate islands to be merged. At this time, Th assigned a value of 10 as a threshold.

Merging condition 2) means that only one face area to be authenticated is found in the input image. That is, in the present invention, even if several faces exist in the input image, only one face positioned at the center of the image is regarded as the authentication target.

Therefore, examine under condition 2) to merge only those regions that are very close in position to the skin color islands already found. In this case, the very close area is determined as whether the minimum bounding rectangle (MBR) of the two areas to be merged overlaps even one pixel.

When merging is performed between areas satisfying the merge condition of the skin color islands, the skin color area of the face to be authenticated is finally detected (S26).

In general, the human skin color region can be efficiently found through the steps described in the present invention, but the condition that the face and the background image of the pre-assumed person in the present invention can be separated by an edge is as described above. If the camera is in a normal state of focus and a clear image is obtained, a satisfactory result can be obtained.

If the image does not satisfy this condition, that is, the image is blurred or the image is not clear enough to distinguish the isolated region, the method described above may not be used in the present invention.

Therefore, the present invention samples the central portion 9a of the input image 9 to a predetermined size under the control of the color distribution extractor 14 as shown in FIG. The face area can be found using the color distribution existing in the area (S27).

The color distribution extractor 14 equally and vertically and horizontally samples the entire image. The equally divided sampling area is a vertical width of an intermediate position where the vertical width is equal to three times the vertical width of the entire input image. The horizontal length is determined by a ratio of 0.75 when the vertical length is 1 in consideration of the fact that a human face is generally longer in the vertical direction than the horizontal direction.

In the present invention, the ratio of the sampling area is defined as described above, and the sampling area is equally divided according to the ratio. After that, the color candidate color distribution of the corresponding pixel is accumulated by investigating the following skin color candidate color distribution detection conditions in the sampling area. (S28)

In the present invention, the facial region is determined by detecting whether the skin color color distribution satisfies the following conditions for each of the sampling regions obtained by dividing the input image by the defined method.

Condition 1) Does R> G, R> B satisfy each pixel?

Condition 2) Does the average intensity (R + G + B) / 3 of each pixel have brightness above the threshold?

In condition 2), the threshold value accumulates the average intensity of the sampled area as a histogram, and then uses the intensity of 20% of the total number of pixels by accumulating the number of pixels from the dark part. This method uses the assumption that if there is a face in the sampled area, there are no more than 20% of the skin area of the eye, nose and other hair areas.

Therefore, when the average R, G, and B values of pixels satisfying the above conditions 1) and 2) are accumulated in the sampled region, the average value of the skin color region of the input image may be estimated. That is, the conditions R-Th <Rinput <R + Th, G Th <Ginput <G + Th, and B Th <Binput <B + Th are satisfied using the above average R, G, and B values obtained for the entire image. Check if it is.

In this case, R, G, and B are average R, G, and B values obtained, and Rinput, Ginput, and Binput become respective pixels of the entire input image. Th assigned a value of 10 as the threshold.

After the examination of each equalized sampling area for the above conditions, the input image is binarized into the skin color region and the non-skin color region. Finally, after performing the labeling algorithm for generating the isolated region, the previously calculated sampling average R, The final face area is detected by determining an isolated area having a mean R, G, B value most similar to G and B and having a predetermined size or more similar to the size of the sampling area as the skin color area (S29-S30).

The above method provides an extraction method for extracting a face region under the assumption that the face region of the person to be authenticated is located at the center of the image based on the present invention. In this case, if the conditions are defined through the above-described method, and the face region is extracted by the defined method, the same result may be obtained.

Therefore, in the present invention, a region is defined not only for the center position of the image but also for the region deviated from the center position, and when the face region is extracted by analyzing the input image by the defined method, a satisfactory result for face region extraction is obtained. It can be self-evident.

As described above, the present invention extracts a color distribution of a face region from an input image from an isolated region using an edge to obtain a color distribution of a face region that can be adapted to a color change of each input image. The face region may be easily extracted by separating regions having the same color but not existing in the center of the image by the merging condition.

In addition, it is possible to increase the success rate for face region extraction even in various environmental changes such as background, lighting condition, face and camera distance, and input characteristics of the camera.

Claims (14)

A detection apparatus for detecting a skin color region of a face from a color image, Color region dividing means for detecting edge information in the input color image and dividing regions from the entire image by the edge information; Isolated region generating means for binarizing a background region and a face region of the color regions divided by the edge information to generate respective isolated regions; Skin color region selection means for extracting an isolated region close to the center of the image from the isolated regions and selecting a region satisfying conditions of average contrast, size information, and color distribution as a candidate face region; And Based on the average color distribution and position information of the representative face candidate region selected by the skin color region selection means, a region is extracted by merging the images satisfying the conditions of the color and the position information with respect to neighboring isolated regions and merging the images. Facial skin color region extraction system using the edge / color information, characterized in that it comprises a; image merging means for extracting the skin color region. The method of claim 1, wherein when the image fails to extract a face region from an unclear input image, the center position of the input image is sampled, and color information of pixels satisfying skin color color detection conditions is sampled from the sampled region. And a color distribution extracting means for extracting a face skin color region by binarizing the extracted color information based on the extracted color information. The facial skin color using edge / color information according to claim 2, wherein the sampling area samples an area located at the center of the image from a plurality of areas in which the entire image is divided at least at a vertical length greater than the horizontal length. Domain extraction system. 3. The color distribution extracting means according to claim 2, wherein the color distribution extracting means is a skin color color distribution detecting condition, wherein the first condition is R> G, R> B per pixel, and the average intensity (R + G + B) / 3 of each pixel. A facial skin color region extraction system using edge / color information, characterized by extracting, as color information, a region satisfying a second condition having a brightness above the threshold. The facial skin color region extraction system using edge / color information according to claim 1, wherein the edge information comprises three masks of a horizontal edge, a vertical edge, and a SUM. The method of claim 1, wherein the skin color region selection means is a condition for selecting a representative face candidate region, wherein the first condition that the average contrast is brighter than the average contrast of the other isolated regions, and the size of the isolated region is within the reference area range. Using the edge / color information, the area that satisfies the third condition in which the value of the second condition and the average R is R> G and R> B with respect to the average G value and B value is selected as the representative face candidate area. Facial skin color extraction system. The method of claim 6, wherein the reference area is determined by a minimum area of a value obtained by dividing the horizontal and vertical lengths of the input image size by 4, and the maximum area is multiplied by 20 by the horizontal and vertical lengths. Facial skin color region extraction system using edge / color information, characterized in that the reference. The method of claim 1, wherein the image merging means is a condition of an isolated region for merging with neighboring islands with respect to the selected representative face candidate region, and is similar to an average R, G, B value of the selected representative face candidate region. A facial skin color region extraction system using edge / color information, characterized by merging a first condition having a color value and an isolated area satisfying a second condition which should be an area adjacent to the selected representative face candidate area. In the method for extracting the facial skin color region from the image image input under the control of the facial skin color region extraction system, Detecting edge information from an input video image; Dividing a background area and a face area into isolated areas based on the detected edge information; Determining a representative face candidate region among the isolated regions of a predetermined size or more closest to the center of the image to find a face candidate region among the isolated regions displayed after edge information detection; And Comprising a step of merging the image by extracting a region that satisfies the conditions of the color and position information for neighboring isolated regions based on the average color distribution and position information of the representative face candidate region; Extraction of facial skin color area using color / color information. 10. The method of claim 9, wherein when the image fails to extract a face region from an unclear input image, the center position of the input image is sampled, and color information of pixels satisfying the skin color distribution detection condition is sampled from the sampled region. And extracting a face skin color region by binarization based on the extracted color information after extraction. The method of claim 10, wherein the extracting the facial skin color region comprises: A first step of determining whether R> G, R> B per pixel; A second step of determining whether the average intensity (R + G + B) / 3 of each pixel has a brightness higher than a threshold; Facial skin color area extraction method using the edge / color information, characterized in that to extract the. The method of claim 9, wherein the determining of the representative face candidate region comprises: Determining whether the average contrast is brighter than the average contrast of the other isolated regions; Determining whether the size of the island area does not exceed the reference area; And A third step of determining whether the value of the average R is R> G, R> B with respect to the average G and B; and if the conditions of the first, second, and third steps are satisfied, the representative face candidate area is used. Facial skin color region extraction method using the edge / color information characterized in that the selection. The method of claim 12, wherein the reference area is determined by a minimum area of a value obtained by dividing the horizontal and vertical lengths of the input image size by 4, and the maximum area is multiplied by 20 by the horizontal and vertical lengths. Facial skin color region extraction method using edge / color information characterized in that the reference. The method of claim 9, wherein in the image merging step, A first step of determining whether the color values are similar to the average R, G, and B values of the selected representative face candidate region; Performing a routine of determining whether the region is adjacent to the selected representative face candidate region, and merging the representative face candidate region and the isolated region when the conditions of the first and second stages are satisfied. Extraction of facial skin color area using color / color information.