JP2007257087A - Skin color area detecting device and skin color area detecting method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a skin color area detecting method hardly having erroneous recognition in dynamic image data, and capable of quickly corresponding to unforseen situations. <P>SOLUTION: Existence of a face of a person is specified from an input image, and a skin color distribution model is created from pixel data of skin color in the specified face area. Skin color distribution models created from skin color data of a variety of people are prepared beforehand, and a skin color distribution model created from input image data and a "corrected skin color distribution model" added with weighting are created. In the corrected skin color distribution model, an appropriate skin color likelihood can be derived even when another person suddenly appears in the input image data, while deriving a large skin color likelihood with respect to the skin color in the input image data. Comparison with a threshold of the derived skin color likelihood is carried out to determine whether respective pixel data groups in the input image data are the skin color or not. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique suitable for application to a skin color region detection apparatus and method.

A technique for detecting a face area and a skin color area including a face from image data is required in various applications.
For example, when a robot interacts with a human, it is necessary to be able to recognize that the other party is a human. In order to recognize being a human, technology for recognizing utterances and technology for recognizing skin tone are required.
For example, to perform special effects such as replacing the background and the clothes you are wearing with completely different images for moving image data of a person, you can recognize skin color and hair. Technology is needed.

The inventor has filed an application for detecting a skin color area from arbitrary image data as Patent Document 1.
The general technical contents of the invention disclosed in Patent Document 1 are as follows.
(1) Collect skin color sample data from face photo data of as many people as possible.
(2) The collected flesh color image data is HSV converted to collect flesh color (H, S) data.
(3) Create a histogram of the collected skin color (H, S) data.
(4) A histogram is approximated by a combination of a plurality of Gaussian distribution models to obtain parameters of a mixed Gaussian distribution model.
(5) The skin color likelihood is calculated for each element of the input image data using the parameters of the mixed Gaussian distribution model.
(6) The calculated skin color likelihood is compared with a threshold value to determine whether the skin color is used.
The invention of Patent Document 1 creates a skin color mixed Gaussian distribution model from skin color image data of a wide variety of races and persons, and determines whether it is a skin color based on this, so-called “general skin color” Detection technology ".
JP 2004-246424 A

In the general-purpose skin color area detection device according to Patent Document 1, it is easy to pick up noise.
For example, if a robot is placed in a room and the conversation partner is a Japanese, using the general skin color detection technology, the wood grain of the desk may be misrecognized as black skin or on the desk There is a possibility that white paper may be mistaken for white skin.

  The present invention has been made in view of the above points, and provides a skin color region detection method that is less likely to be misrecognized especially in moving image data and can be quickly applied even when an unexpected situation occurs. Objective.

The present invention for solving the above problems is as follows.
A skin color pixel extraction unit that extracts pixel data that seems to be skin color from input image data;
A histogram generation unit that generates a histogram from information about the color of the skin color pixel data obtained by the skin color pixel extraction unit;
A histogram approximation unit that approximates the histogram obtained by the histogram generation unit to obtain a first statistical model;
A composite statistical model creation unit that performs weighted addition processing on the first statistical model obtained by the histogram approximation unit and a second statistical model different from the first statistical model;
Using the composite statistical model obtained by the composite statistical model creation unit, the skin color likelihood corresponding to the pixel data of the input image data is obtained, the obtained skin color likelihood is compared with a threshold value, and the pixel A binarization processing unit that determines whether or not the data is skin color, and obtains a skin color area detection result for the input image data based on the determination result;
The skin color area detection apparatus includes a threshold color update unit that uses the skin color area detection result obtained as a result of the binarization processing unit to update the threshold value.

It is rare that a large number of persons having various skin color variations are included in an arbitrary input image. That is, the technique of Patent Document 1 which is a conventional technique has high versatility, and therefore noise is inevitably included. Therefore, the presence of a human face is identified from the input image, and if there is skin color pixel data in the identified face area, it is assumed that skin color pixel data similar to the pixel data exists in the entire image data, and input A statistical model (hereinafter referred to as “skin color model”) for estimating whether the image data is a skin color or not is created.
However, this method alone may cause false detection when the face area cannot be detected correctly.
In view of this, a statistical model created from skin color data of a wide variety of people is prepared in advance using a part of the technical idea of Patent Document 1, and a modified model (hereinafter referred to as “the skin color model created from input image data” is weighted and added. Create a modified skin color model ").
The corrected skin color model can derive a large likelihood for the skin color in the input image data, and can derive a corresponding likelihood even if another person suddenly jumps into the input image data.
The likelihood of the skin color for the pixel data thus derived is compared with a threshold value, and it can be determined very quickly and accurately whether the individual pixel data group in the input image data is the skin color.

According to the present invention, it is possible to determine a skin color region from image data having a large amount of data with a relatively small amount of calculation extremely quickly and accurately.
The present invention is useful for still images, but is particularly useful for applications using moving images that require a large amount of data and require rapid processing.

  Hereinafter, a first embodiment of the present invention will be described with reference to FIGS.

  FIG. 1 is a functional block diagram of a skin color area detecting apparatus according to the first embodiment of the present invention.

The main part of the skin color area detection apparatus 1 is actually a function provided by a computer program.
The image information acquisition unit 2 acquires color image data by a predetermined method. Any means for acquiring color image data may be used. In the present embodiment, for example, a well-known CCD camera capable of shooting a moving image.
The image data may be a still image. However, it must be a color image. This is because the present invention is a technique for estimating a skin color region from input image data, and the present invention cannot be realized unless there is information about color.
However, in the present embodiment, rough processing of image data is performed in units of one still image. This is because the present embodiment depends on a face area recognition process described later.
In the following description, image data refers to a collection of data constituting one still image, and pixel data refers to the color, brightness, etc. of each dot whose position is expressed by a predetermined XY coordinate. Means the data to be represented. That is, the image data is a collection of pixel data.

In many cases, the input image data is information suitable for handling by predetermined application software or hardware such as RGB or YMCK. This is HSV converted.
The HSV converter 3 converts RGB color image data into three elements of hue, saturation, and value (hereinafter abbreviated as “H”, “S”, and “V”, respectively). “H component”, “S component”, and “V component” represent respective information components in the pixel data, and “H data”, “S data”, and “V data” are converted by the HSV conversion unit 3. Represents the data for each component.) This is because the information V component related to lightness is extracted from the color image data, and the color information H component and S component independent of the light and darkness of the input screen are handled. It is for use in.
HSV conversion is a well-known technique, and details are omitted because it is described in part in Patent Document 1 consisting of the same inventor as the present application.

The histogram generation unit 4 generates a histogram of the frequency of appearance of the H and S component pairs corresponding to the skin color, using a data group and a V data group composed of H and S data sets included in one still image data. To do. This process will be described in detail later with reference to FIG.
The histogram obtained by the histogram generation unit 4 represents the frequency of appearance of a set of H and S components corresponding to skin color, that is, likelihood (probability).

The histogram approximation unit 5 approximates the obtained histogram with a Gaussian mixture model.
A histogram obtained from input image data is an aggregate of a huge amount of discrete values. With this enormous amount of discrete values, it is not possible to determine whether or not the unknown data is flesh-colored. Therefore, the histogram is approximated by a Gaussian mixture model that can be expressed with a small amount of information. By this approximation processing, it is possible to calculate the likelihood for arbitrary image data with a small amount of calculation and good practical accuracy.
The Gaussian mixture model is the most representative statistical model, and is an aggregate of a plurality of Gaussian distributions. That is, it is an aggregate of a plurality of sets of mean and variance values constituting a Gaussian distribution. An enormous collection of discrete values is approximated by the plurality of Gaussian distributions. As the statistical model, other known models other than the Gaussian mixture model can be applied. That is, the embodiment is not limited by the type of statistical model.
The details of the approximation processing in the histogram approximation unit 5 are described in detail in Patent Document 1, and are also disclosed in the following papers, which are well known.
(1) S. McKenna, S. Gong, and Y. Raja.Modeling Facial Color and Identity with Gaussian Mixtures.Pattern Recognition, 31 (12): 1883-1892, 1998.
(2) HP Graf, E. Cosatto, D. Gibbon, M. Kocheisen, and E. Petajan.MultimodelSystem for Locating Heads and Faces.In Proceedings of the Second IEEE International Conference on automatic Face and Gesture Recognition, pages 88-93, 1996.

The histogram which is an aggregate of a huge amount of discrete values by the histogram approximation unit 5 is a data group (hereinafter referred to as “a group of multiple sets of average and variance data constituting a Gaussian distribution, which has an extremely small amount of data”. It is abbreviated as “GMM parameter group.” GMM is converted into “Gaussian Mixture Model”.
The GMM parameter group obtained by the histogram approximation unit 5 is a “GMM parameter group specific to the target person” depending on the skin color of the person existing in the currently input image data.
For convenience of explanation, this “GMM parameter group specific to the target person” is referred to as a “first GMM parameter group”.
The Gaussian mixture model expressed by the first GMM parameter group is a skin color model that expresses the likelihood of the skin color of the target person.

The created flesh color GMM 7 is data in the nonvolatile memory. For example, it is stored as a file in a known hard disk device.
The created skin color GMM7 is obtained as a result of extracting skin color information from image data of people of various races, genders, ages, etc. using the technique of Patent Document 1 and normalizing it with a Gaussian mixture model. Stored GMM parameter groups are stored.
The GMM parameter group stored in the created skin color GMM 7 is created based on image data of people of various races, genders, ages, etc., that is, “general-purpose GMM parameter group” that does not depend on the target person. It is.
For convenience of explanation, this “general-purpose GMM parameter group” is referred to as a “second GMM parameter group”.
The Gaussian mixture model expressed by the second GMM parameter group is a skin color model that expresses a general skin color likelihood.

The modified GMM creating unit 6 performs weighted addition processing on the first GMM parameter group and the second GMM parameter group to create a modified GMM. For example, the first GMM parameter group and the second GMM parameter group are added at a ratio of 3: 2. Note that by changing the ratio of addition, the modified GMM to be created can be made to strongly depend on the skin color of the person in the input image data, or conversely, can correspond to a wide range of races.
The Gaussian mixture model expressed by the modified GMM is a complex skin color model in which a general skin color likelihood is added to the skin color likelihood of the target person.

  The skin color likelihood calculation unit 8 calculates the skin color likelihood based on the corrected GMM for H and S data corresponding to individual pixel data in the input image data obtained by the HSV conversion unit 3.

The binarizing unit 9 compares the likelihood data obtained by the skin color likelihood calculating unit 8 with the threshold value 11 to determine whether the skin color is used, and outputs the determination result as a binary value of 0 or 1. .
The threshold value 11 is a single scalar value, and is stored in, for example, a well-known RAM.
The determination result obtained by the binarizing unit 9 is stored as skin color area data 10 in, for example, a RAM.
The skin color area data 10 is subjected to a predetermined integration process in order to “fill” details that are not recognized as skin color, such as eyes, eyebrows, and nostrils in the face area.

  The threshold update unit 12 updates the threshold 11 by a predetermined calculation process based on the rectangular face region data obtained from the rectangular face region creation unit 21 in the histogram generation unit 4 and the skin color region data 10.

  FIG. 2 is a functional block diagram detailing the internal configuration of the histogram generation unit 4.

  Of the H, S, and V data obtained from the HSV conversion unit 3, the V data is passed to the rectangular face region creation unit 21. The rectangular face area creation unit 21 searches for a bright area of a circle or an ellipse from one screen of V data representing brightness, and creates rectangular data surrounding the circle or ellipse. This is a process for specifying a rough face area on the basis of the property that the face area is an almost elliptical area having uniform brightness.

  Of the H, S, and V data obtained from the HSV conversion unit 3, the H and S data is transferred to the face area data extraction unit 22. Further, the rectangular face area data obtained from the rectangular face area creating unit 21 is also passed to the face area data extracting unit 22. The face area data extraction unit 22 identifies a set of H and S data in the rectangular face area.

The simple skin color determination unit 24 determines whether the H and S data selected by the face area data extraction unit 22 is included in the H and S range data group in the general-purpose skin color GMM 23.
The data obtained in this way is data with a correspondingly high possibility of skin color.

  A histogram creation unit 25 creates a histogram from the obtained H and S data of the skin color. In this way, a histogram is created based on the image data of the target person based on data that seems to be skin color data in the face area.

  FIG. 3 is a functional block diagram detailing the internal configuration of the threshold update unit 12.

For the skin color area data 10, the appearance rate of the skin color data inside and outside the face area is calculated based on the rectangular face area data obtained by the rectangular face area creating unit 21.
The face area skin color data appearance rate calculation unit 31 counts the number of skin color area data included in the rectangular face area data, and calculates the appearance rate from the total number of rectangular face area data. This is parameter a.
The face area outside skin color data appearance rate calculating unit 32 counts the number of skin color area data outside the rectangular face area data, and subtracts the number of the entire rectangular face area data from the number of the entire image data. calculate. This is parameter b.
The threshold update unit calculates a and b obtained by the skin color data appearance rate calculation unit 31 and the skin color data appearance rate calculation unit 32 outside the face region, and α, β, and S _rate stored in the constant 34. To perform the following update operation.

In the formula, max (x, y) is a function for selecting the maximum value from x and y.
The constant 34 is data in the nonvolatile memory similar to the created flesh color GMM7. For example, it is stored as a file in a known hard disk device. α and β are preset learning rate coefficients, and S _rate is a minimum value of the skin color data appearance rate in the face area.
The value of α is set so that the threshold value is reduced at a high speed when the skin color appearance rate a in the face area is smaller than the minimum value S _rate of the skin color data appearance rate. For example, α is 0.6.
The value of β is always set to increase the threshold value at a slow speed. For example, β is 0.4.
The value of S _rate is the minimum value when the face is facing sideways. When rectangular face area data is created by the rectangular face area creation unit 21 for image data showing a profile, the skin color appearance rate in the face area is about 0.56, so this value is set. To do.

As the skin color appearance rate a in the rectangular face region decreases, the threshold T decreases.
As the skin color appearance rate b outside the rectangular face area increases, the threshold T increases.
The detection accuracy of the skin color area is improved by the update calculation process of the threshold value T by the threshold value calculation unit 33. When the input image data is a still image, a and b converge to a predetermined value and the threshold T also converges to a predetermined value by repeated calculation processing of the threshold update unit 12.

  FIG. 4 shows a skin color area detection process according to this embodiment.

FIG. 4A shows input image data. It is color image data of the upper body of a woman.
FIG. 4B shows a result of detecting the face area F by the rectangular face area creating unit 21. A sample of the skin color of the target person is extracted within the range of the detected face area, and a skin color distribution model is created.
FIG. 4C shows the detected skin color area data 10. Large flesh-colored regions S1, S2, and S3 are detected by the integration process. In addition, the threshold update process reduces noise and improves detection accuracy.

The embodiment is not limited to the above, and the following applications are possible.
(1) In particular, in the skin color detection process for a moving image, a different threshold value may be created according to the coordinates based on the form of the detected skin color area data 10. In other words, since the human subject is continuously moving, the skin color detection device that is resistant to noise is set by setting the threshold value of the portion determined to be a skin color region low and setting the threshold values of other portions high. Can be obtained.
(2) Instead of preparing the constants 34 in advance, the optimum values of the respective constants can be obtained by a prior calculation process. For example, using the sample still image data, it is conceivable to perform threshold update processing several times while changing α, β, and S _rate, and to select an optimum value.

It is a block diagram which shows the example of the skin color area | region detection apparatus by the 1st Embodiment of this invention. It is a block diagram which shows the inside of a histogram production | generation part. It is a block diagram which shows the inside of a threshold value update part. It is a block diagram which shows the example of a skin color area | region detection process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Skin color area | region detection apparatus, 2 ... Image information acquisition part, 3 ... HSV conversion part, 4 ... Histogram generation part, 5 ... Histogram approximation part, 6 ... Correction GMM creation part, 7 ... Created skin color GMM, 8 ... Skin color likelihood Degree calculation unit, 9 ... binarization unit, 10 ... skin color area data, 11 ... threshold, 12 ... threshold update unit

Claims (3)

A skin color pixel extraction unit that extracts pixel data that seems to be skin color from input image data;
A histogram generation unit that generates a histogram from information about the color of the skin color pixel data obtained by the skin color pixel extraction unit;
A histogram approximation unit that approximates the histogram obtained by the histogram generation unit to obtain a first statistical model;
A composite statistical model creation unit that performs weighted addition processing on the first statistical model obtained by the histogram approximation unit and a second statistical model different from the first statistical model;
Using the composite statistical model obtained by the composite statistical model creation unit, the skin color likelihood corresponding to the pixel data of the input image data is obtained, the obtained skin color likelihood is compared with a threshold value, and the pixel A binarization processing unit that determines whether or not the data is skin color, and obtains a skin color area detection result for the input image data based on the determination result;
A skin color region detection apparatus comprising a threshold update unit that uses a skin color region detection result obtained as a result of the binarization processing unit to update the threshold. The skin color pixel extraction unit
A face area estimation unit that estimates the position of a person's face from information indicating the brightness of the input image data and obtains a face area estimation result;
A skin color determination unit that determines whether or not the input image data is pixel data of skin color from a pixel data group within the range of the face region estimation result;
The histogram approximation unit approximates the histogram by a Gaussian mixture model,
The second statistical model is a skin color statistical model that does not depend on input image data,
The threshold update unit uses the pixel data appearance rate of the skin color area within the range of the face area estimation result and the pixel data appearance rate of the skin color area outside the range of the face area estimation result for the threshold update calculation. The skin color area detecting apparatus according to claim 1, wherein the apparatus is a skin color area detecting apparatus. Skin color pixel extraction procedure for extracting pixel data that seems to be skin color from input image data;
A histogram generation procedure for generating a histogram from information relating to the color of the skin color pixel data obtained in the skin color pixel extraction procedure;
A histogram approximation procedure for approximating the histogram obtained in the histogram generation procedure;
A composite statistical model creation procedure for performing a weighted addition process on the first statistical model obtained by the histogram approximation procedure and a second statistical model different from the first statistical model;
Using the composite statistical model obtained in the composite statistical model creation procedure, the skin color likelihood corresponding to the pixel data of the input image data is obtained, the obtained skin color likelihood is compared with a threshold value, and the pixel A binarization processing procedure for determining whether the data is a skin color and obtaining a skin color area detection result for the input image data based on the determination result;
A skin color region detection method comprising: a threshold update procedure that uses a skin color region detection result obtained as a result of the binarization processing procedure to update the threshold.

Images