JPH06309433A - Picture identification system - Google Patents

Abstract

PURPOSE:To correctly identify human's skin on a picture by detecting tone candidates of a skin area by extracting the picture elements of a skin color and determining the skin area from its candidates. CONSTITUTION:A skin color picture element detector 3 successively reads out the data of the lumimance L, hue H and chroma C of each picture element from a frame memory 2, judges whether the data satisfies a prescribed skin color condition or not and deletes a picture element disabled to satisfy the condition from the memory 2. A skin color candidate detector 4 groups skin color picture elements mutually connected in the memory 2 as one area to find out a candidate of a skin area. A skin area detector 6 detects an area supposed to practically correspond to a human's skin out of skin area candidates by using a skin index expressing the likeness of skin and sends a control signal cmtll indicating the detected result to a skin image judging device 7. Consequently, the device 7 judges that input picture is a skin picture including the skin area and outputs '1' e.g. as a judged result.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image identification system suitable for use in, for example, an image processing apparatus for subjecting an image to predetermined processing.

[0002]

2. Description of the Related Art When an image contains a person, the person tends to focus on the skin area of the person in the image,
It is known that the image quality of the skin area has a great influence on the subjective image quality of the entire image.

Therefore, in the conventional video equipment, when the image quality of a reproduced image is adjusted, the skin area is emphasized and various parameters are set so that the image quality of the area is optimized. ing.

By the way, conventionally, the extraction of the skin region is performed based on only the pixel-level color information.

That is, in the conventional art, for example, in the RGB space shown in FIG.
The range (the shaded area in the figure) that is considered to be occupied by skin color is set in advance, and the pixels with the colors included in that range (the area indicated by a mark in the figure) are extracted from the image. Then, the area formed by the pixel is regarded as a skin area (a shaded portion in the drawing).

As a three-dimensional space for representing colors, in addition to the above-mentioned RGB space, for example, a YUV space used for television signals, L ^* a ^* b established by CIE (International Commission on Illumination). ^{* A} uniform space is used.

[0007]

As described above, conventionally, the skin color region is assumed to be the skin region. However, even if the color on the image is flesh,
Actually, it is not always the area of human skin, that is, there are flesh-colored objects other than human skin, and thus it is difficult to correctly identify the skin only from the color information.

Therefore, the image quality adjustment is performed on the image not including the skin region as in the case of the image including the skin region as described above, and there is a problem that the image quality is adversely affected.

The present invention has been made in view of such a situation, and makes it possible to correctly identify human skin in an image.

[0010]

An image identification method according to claim 1 is an image identification method for determining whether an image is a skin image including a skin area corresponding to a person's skin. A skin color pixel detector 3 as an extraction means for extracting pixels, and a skin area candidate detector 4 as a detection means for detecting a skin area candidate from each pixel of the skin color extracted by the skin color pixel detector 3, The skin region detector 6 or the skin region importance calculator 31 and the skin region detector 6 as the determining means for determining the skin region from each of the skin region candidates detected by the skin region candidate detector 4. Alternatively, based on the determination result of the skin area importance calculator 31, it is determined whether or not the image is a skin image, and the skin image determiner 7 or the skin image importance calculator as a determination unit that outputs the determination result. 32 and .

According to the image identification method of the second aspect, the skin area detector 6 is made to calculate the importance of each candidate of the skin area, and based on the importance, from among the candidates of the skin area, The skin area is determined, and the skin image determiner 7 determines whether the image is a skin image based on the determination result of the skin area detector 6, and determines whether the image is a skin image. It is characterized in that the binary judgment result of is output.

According to the image identification method of the third aspect, the skin area importance calculator 31 determines the skin area from each candidate of the skin area, and calculates the importance of the skin area. Based on the importance of the skin area calculated by the skin area importance calculator 31, the importance calculator 32 calculates the likelihood that the image is a skin image, and outputs it as a determination result. Characterize.

[0013]

According to the image identifying method of the first aspect, the pixels of the skin color are extracted, and the candidate of the skin region is detected from each of the pixels. Then, the skin region is determined from each candidate of the skin region, and it is determined whether the image is a skin image based on the determination result. Therefore, it is possible to accurately identify the skin.

In the image identification method according to the second aspect, the skin area detector 6 determines the skin area from each of the skin area candidates based on the importance of each of the skin area candidates. Let Then, the skin image determiner 7 is caused to determine whether or not the image is a skin image based on the determination result of the skin area detector 6, and a binary value indicating whether the image is a skin image or not. Output the judgment result. Therefore, when this method is applied to a still image, for example, the skin of a person in the still image can be accurately identified.

In the image identifying method according to the third aspect, the skin area importance calculator 31 determines the skin area and calculates the importance thereof. Then, the skin image importance calculator 32 is caused to calculate the certainty that the image is a skin image based on the importance of the skin area calculated by the skin area importance calculator 31, and as the determination result. Output. Therefore, when this method is applied to a moving image, it is possible to identify the skin of a person in a moving image that changes with time without impairing temporal continuity.

[0016]

1 is a block diagram showing the configuration of an embodiment of an image identifying apparatus to which the present invention is applied. Coordinate converter 1
Converts the digital data of R, G, and B, which are the three primary colors, into a luminance L, a hue H, and a chroma C representing the three attributes of the color. The frame memory 2 stores the brightness L, hue H, and chroma C for one screen (frame) from the coordinate converter. The flesh color pixel detector 3 adaptively sets the flesh color condition (hereinafter referred to as flesh color condition) on the screen from the luminance L, hue H, and chroma C for one screen from the coordinate converter 1. Further, the skin color pixel detector 3 sequentially reads the data of the luminance L, the hue H, and the chroma C of each pixel from the frame memory 2,
It is determined whether or not they satisfy the set skin color condition, and pixels that do not satisfy the skin color condition are deleted from the frame memory 2.

The skin region candidate detector 4 detects skin region candidates. That is, the skin region candidate detector 4 groups the skin color pixels connected to each other in the frame memory 2 as one region and assigns the same label to all the pixels included in the same region, thereby That is, candidates for the skin region (skin region) are obtained. The frame memory 5 stores the label assigned to the pixel by the skin area candidate detector 4.

The skin area detector 6 determines (detects) an area that is considered to actually correspond to the skin of a person from among the skin area candidates detected by the skin area candidate detector 4. At this time, a skin index (which will be described later) indicating the importance of each skin area candidate and skin-likeness is used, and these values are calculated from the data stored in the frame memories 2 and 5.
Furthermore, when the skin region detector 6 can determine (detect) the skin region, it sends a control signal cntl1 indicating that to the skin image judging device 7. The skin image determiner 7 receives the control signal cntl from the skin area detector 6.
When 1 is received, that is, when at least one skin area is determined (detected) from the image for one screen input to the device, the input image is a skin image including the skin area corresponding to the skin of the person. And outputs 1 out of 0 and 1 as the determination result. Further, when the skin area detector 6 does not output the control signal cntl1, the image determiner 7 determines that the input image is not a skin image,
For example, 0 is output from 0 and 1 as the determination result.

In the image identifying apparatus configured as described above, first, for example, RGB data of an image is input to the coordinate converter 1. In the coordinate converter 1, for example, the luminance / color difference space YUV of the NTSC television signal is used to calculate the RG
Coordinate conversion from B to luminance L, hue H, and chroma C is performed.

That is, the coordinate converter 1 converts RGB into YUV according to the equation Y = 0.299R + 0.587G + 0.114B U = BY V = RY, and further the equation L = Y H = tan ^-1. Convert YUV to LHC according to (V / U) C = (U ² + V ² ) ^1/2 .

In the coordinate converter 1, the NTSC
Not only the luminance / color difference space YUV of the television signal,
For example, L ^* established by CIE (International Commission on Illumination)
The coordinate conversion from RGB to LHC can be performed using an a ^* b ^* uniform space or the like.

That is, in this case, first, the RGB is CIE XYZ display system according to the formula X = 0.6067R + 0.1736G + 0.2001B Y = 0.988R + 0.5868G + 0.1144B Z = 0.0661G + 1.1150B (1) Converted to
Further formula ^{L * = 116 (Y / Y} 0) 1/3 -16 a * = 500 ((X / X 0) 1/3 - (Y / Y 0) 1/3) b * = 200 ((Y / Y ₀₎ ^1/3 - to convert the XYZ into L ^* a ^* b ^* according to ^{_{(Z / Z 0) 1/3)}} ··· (2). And
Convert L ^* a ^* b ^* to RGB according to the equation L = L ^* H = tan ⁻¹ (b ^* / a ^* ) C = (a ^{* 2} + b ^{* 2} ) ^1/2 .

However, X ₀ , Y ₀ , Z in the equation (2)
₀ is the value of X, Y, Z on the white reference plane, which originally changes depending on the white balance of the light source or the camera, but here, for example, when R, G, B take the maximum values, respectively. X, Y, Z of. That is, X ₀ ,
Y ₀ and Z ₀ are X, Y and Z when R = G = B = 1 in the equation (1).

The expression (2) is a defining expression (approximate expression) when the expressions X / X ₀ , Y / Y ₀ , Z / Z ₀ > 0.008856 are satisfied. , XYZ to L ^* a ^*
Needs to be converted to b ^* .

For details of this approximate expression and the above-mentioned color space, see: Ikeda, "Basics of Color Engineering", Asakura Shoten. G. Wyszecki and WSstiles, "Color Science: Conce.
pts and Methods, Quantitative Data and Formulae ",
Wiley ・ Since it is described in the Color Society of Japan, "Color Handbook", The University of Tokyo Press, etc., its explanation is omitted here.

As described above, the coordinate converter 1 performs RGB
After conversion from LHC to LHC, the luminance L, hue H, and chroma C are input to the frame memory 2, and the frame memory 2 stores the luminance L, hue H, and chroma C for one screen.

Further, the luminance L, the hue H and the chroma C are also input to the skin color pixel detector 3, where the skin color pixels are detected.

That is, as shown in FIG. 2, the flesh color pixel detector 3 is composed of a hue deciding unit 11, a luminance range detector 12, a chroma range detecting unit 13, and a flesh color deciding unit 14, and the hue deciding unit 11 first. Then, the LH of each pixel from the coordinate converter 1
Of C, it is determined whether the hue H satisfies the formula H _min <H <H _max (3).

Here, H _min and H _max are preset values and represent the minimum and maximum values that the hue of the skin color can take.

When it is determined that the hue H of the pixel satisfies the expression (3), the luminance L and the chroma C of the pixel are
It is output to the luminance range detector 12 and the chroma range detector 13. In the luminance range detector 12 or the chroma range detector 13, among the pixels for one screen, the luminance L of the pixel supplied from the hue determination circuit 11 and satisfying the expression (3), that is, the pixel having the hue of the skin color. Alternatively, the average value L _ave or C _{ave of the} chroma C is calculated, respectively. And this brightness L
Maximum value L _max, or C from the mean value C _ave of the average value L _ave or chroma C, the brightness or chroma, which may regarded as skin color
and _max, the minimum value L _{mi n} or C _min is calculated respectively according to the following equation.

L _min = L _ave × p L _max = L _ave × q C _min = C _ave × r C _max = C _ave × s where p, q, r, and s are average values L _ave or C _ave On the other hand, it is a constant that determines what value is the minimum value and the maximum value, and is preset to an appropriate value in the range of the equations p, r ≦ 1.0 q, s ≧ 1.0.

L _max and L _min calculated by the luminance range detector 12 and C _max calculated by the chroma range detector 13.
And C _min are supplied to the skin color determiner 14. In the skin color determiner 14, first, from the frame memory 2 (FIG. 1), 1
L, H, C data of each pixel for the screen are sequentially read out,
It is determined whether it satisfies the equations (3) to (5), respectively. L _min <L <L _max ... (4) C _min <C <C _max ... (5)

If any of the expressions (3) to (5) is not satisfied, the skin color determiner 14 removes the pixel from the frame memory 2.

In general, in the color space of the luminance L, the hue H, and the chroma C, the chroma C cannot take a large value when the luminance L is large due to the problem of the color reproduction range. Maximum value v that can be expressed by the word length of
_max (for example, v _max = 255 when the word length is 8 bits)
In reality, there is no such a color, and therefore there is no problem even if a pixel having all LHC as v _max is used as a pixel other than the skin color.

[0035] Therefore, in this embodiment, the pixels that do not satisfy any one of expressions (3) to (5), i.e., all non-skin color (considered) a pixel LHC, the maximum value v _{ma x} By replacing and re-storing in the frame memory 2, the LHC of the pixel is regarded as removed from the frame memory 2.

The skin color appearing in the image (not the so-called skin color, but the color of the human skin portion) changes depending on individual differences and lighting conditions, and therefore is limited by the above equations (3) to (5). If is too strict, there are times when it is not possible to extract a region that is actually skin.

However, before performing the processing described later,
At this stage, failing to extract a region that is actually skin eventually makes it impossible to detect the region that is actually skin. Therefore, in formulas (3) to (5),
For example, the limitation of the hue H according to Expression (3) is set loosely so as to include almost all skin hues (because of the thresholds L _min and L _max in Expressions (4) and (5)). , C _min and C _max are expressed by the equation (3) as described above.
Since it is calculated from L and C of the pixels that have not been subjected to the restriction of (3), loosening the restriction of Expression (3) is equivalent to loosening all the restrictions of Expressions (3) to (5). ).

Further, as a result, the average value L _ave or C _{ave of the} luminance or chroma considered to be the pixel of the skin becomes extremely large or small, and as a result, the equation (4),
It is possible that L _min , C _min , L _max , and C _max in (5) become improper values (become too large or too small).

Therefore, the upper limits of L _min and C _min are set to L _max ,
A lower limit is set for C _max to prevent L _min and C _min from becoming too small, and L _max and C _max from becoming too large.

Originally, the tint is expressed in chroma, but in reality, even if the pixels have the same chroma, it seems that the tint is higher as the luminance is smaller. Therefore, the only limitation to the tint is the chroma. Instead, it is considered more appropriate to use the chroma / luminance ratio C / L. In particular, for pixels with low luminance, it is considered that limiting the tint by the ratio of the chroma and the luminance is more effective than using only the chroma.

The value of C / L does not increase even if the chroma C is high if the luminance L is high, but empirically, even if the luminance L is high, it can be accepted as a skin color. The upper limit of chroma C does not become so large.
Therefore, it is considered appropriate to set an upper limit on the chroma C for a pixel having a high luminance L.

Therefore, in the flesh color pixel detector 3, the flesh color is limited by adding the limitation of the hue of the pixel shown by the equation (6) to the limitation of L, H, C according to the above equations (3) to (5). It is preferable to detect the pixels of. C / L <c × L _max (6) where c is a predetermined constant.

As described above, after the skin color pixel detector 3 removes the LHC of the pixel which is not the skin color from the frame memory 2, the pixels left in the frame memory 2, that is, the skin color pixels (hereinafter, skin color pixels). Segmentation (detection of candidates for the skin region) is performed by the skin region candidate detector 4 (FIG. 1).

That is, the skin region candidate detector 4 refers to the skin color pixels stored in the frame memory 2 and performs labeling by examining the pixels in the vicinity of the skin color pixels to construct skin region candidates. Whether or not the pixel is a skin color pixel is determined by whether or not all the values of LHC are v _max (at least one of L, H, and C is v
If it is not _max , the pixel is determined as a skin color pixel).

Specifically, first, when a pixel x ₀ stored in the frame memory 2 is a skin color pixel, for example, four pixels adjacent to the pixel x ₀ as shown in FIG. It is determined whether each of the middle and shaded portions) is a skin color pixel. Then, for example, when the upper right pixel x _n of the pixel x ₀ is a skin color pixel, the value of the pixel x _{n in} the frame memory 5 is referred to, and it is determined whether a label has already been assigned to that pixel. It

As a result of the judgment, when the label t _n is already assigned to the pixel x _n , the same label t _{n as} that label is assigned to the pixel x ₀ . This is done by writing the label t _n to the pixel x ₀ on the frame memory 5.

If two or more pixels out of the four adjacent pixels of the skin color pixel x ₀ are skin color pixels, and labels are already assigned to them, and they are different labels, Skin area candidate detection circuit 4 (FIG. 1)
Assigns any one of the labels to the pixel x ₀ , stores it in the frame memory 5, and stores the fact that all the labels are adjacent to each other in the label table of its built-in memory (not shown). Has been done.

That is, for example, as shown in FIG. 3B, three of the four adjacent pixels of the skin color pixel x ₀ are skin color pixels, and the label has already been assigned, and 3
When two of the two skin color pixels have the label t _m and the remaining one label has the label t _n , the pixel x ₀ in the skin region candidate detection circuit 4 has the labels t _m and t _n . of out, for example, a label t _m is assigned, while being registered in the position (address) corresponding to the pixel x ₀ of the frame memory 5, for example, in the label table as shown in FIG. 3 (c), the label t _m and the label It is remembered that t _n are adjacent.

However, if no label has been assigned to any of the adjacent pixels, the skin area candidate detector 4
Creates a new label, which is assigned to pixel x ₀ .

In the label table of FIG. 3C,
In the column to the left, the labels generated so far are
The labels are described in the order in which they occur, and in the right column, the label adjacent to the label described in the left column is described. Further, in the right column, the label generated before the label in the left column is described.

Therefore, as shown in the label table of FIG. 3C, when the label t _m is generated before the label t _n, the pixel to which the two labels t _m and t _n are assigned is When they are adjacent to each other, t _m is described in the right column corresponding to the label t _n .

[0052] However, the label t _m label t _k in the corresponding column to the (is adjacent the label t _m of the pixel is labeled t _k generated earlier than the pixel) already written and the label when the pixels t _m is assigned, the pixels labeled t _n generated is assigned to a later it is adjacent, as shown in FIG. 3 (d), the label t _k
Is written in the column corresponding to the label t _n .

Thereafter, in the skin area candidate detector 4,
Relabeling is performed on the label data stored in the frame memory 5. This is because, as shown in FIG. 4A, in the above-described one-time labeling, there may exist pixels (regions of pixels) to which different labels are assigned despite being adjacent. Because. The re-labeling is performed based on the label table created in the first labeling and storing the label adjacency relationship.

That is, in the skin area candidate detector 4, the label table (FIG. 3 (c) or FIG. 3 (d)) is referred to, and among the labels stored in the frame memory 5, the adjacent label is described in the label table. By converting the existing label into an adjacent label, adjacent skin color pixels are allotted with the same label and stored again in the frame memory 5.

As a result, for example, the region labeled for the first time as shown in FIG. 4 (a) is re-labeled as shown in FIG. 4 (b).

A set of pixels to which the same label is assigned by relabeling is regarded as one skin area candidate,
The set of labels (t0, t1, t2, ...)
Is output from the skin area candidate detector 4 to the skin area detector 6.

In the above case, if the pixel is a flesh color pixel, labeling is always performed. However, even if the pixel is a flesh color pixel, if that pixel has a small portion in contact with another flesh color pixel. , It is considered to be the edge part of the skin-colored area of a certain object (including human). If such a pixel is left as a skin-colored pixel, the skin-colored area including the pixel is There is a risk that the area will be connected to the flesh-colored area and the area originally supposed to be two areas will become one area.

Therefore, when the pixel is a skin color pixel, for example, eight surrounding pixels (upper, lower,
Of the left, right, upper left, lower left, upper right, and lower right pixels), the pixel can be labeled only when a predetermined number or more of pixels are skin color pixels.

The skin area detector 6 includes a histogram generator 21 and a histogram analyzer 2 as shown in FIG.
2, a skin area determiner 23, a region analyzer 24,
The skin region is determined from the skin region candidates detected by the skin region candidate detector 4.

That is, first, in the area analyzer 24, the frame memory 5 is referred to, and the size, position, and hole size of each candidate of the skin area constituted by the same label are obtained, and the threshold value processing is performed. As a result, the too small area, the area located at the corner on the screen, and the area having the large hole are deleted from the skin area candidates (hereinafter referred to as the skin area candidates).

Here, the size of the area is the number of pixels forming the area, and the position of the area is represented by the barycentric coordinates of the area. In addition, a hole in a region is defined as a set of non-skin color pixels (non-skin color pixels) surrounded by skin color pixels in a certain skin region candidate, and the number of pixels is defined as the size of the hole.

The size of the area and its position on the screen are considered to represent the degree of importance of the area in the image.
This is because a large area or an area at the center of the screen is generally conspicuous, and it is considered that the importance is high also from the content of the image. On the other hand, the size of the hole is considered to be one amount that expresses the skin-likeness of the area. This is because the skin region is usually filled with skin-colored pixels and is therefore considered not to have a very large hole. Of course, for example, a face, etc., also includes non-skin-colored parts such as eyes and lips as holes,
Its size is not so large compared to the size of the whole face. On the other hand, a region corresponding to a non-skin may have a large hole. From the above,
The threshold value processing based on the size of the area, the barycentric coordinates, and the size of the hole can be considered as processing of cutting off the skin area candidate according to the importance of the area and the skin-likeness.

Specifically, in the area analyzer 24, threshold processing is performed as follows. First, regarding the size of the area, the number of skin color pixels in the area is compared with a predetermined threshold value Th _size set in advance. If the number of skin color pixels is smaller than the threshold Th _size , the area is It is excluded from the skin area candidates.

Here, the threshold Th _size is set by determining how large a region is detected as skin with respect to the number of pixels of the entire screen.

However, in the case of a dark image as a whole, there is a tendency that the area determined as the skin area candidate by the skin area candidate detector 4 becomes small due to the influence of the shadow or the like. Therefore, when the brightness of the entire screen is low, the threshold Th _size can be set to a small value, for example, as a function of the average brightness of the screen.

For the barycentric coordinates, for example, the importance ps of the position of the area is calculated according to the equation (7), and the importance ps is a predetermined threshold Th _pos (however, T
When h _pos is smaller than a preset appropriate value of 0.0 or more and 1.0 or less), the area is excluded from the skin area candidates.

Ps = exp (-((d _x / σ _x ) ² + (d _y / σ _y ) ² )) (7) where d _x and d _y are the center of gravity of the screen and the center of gravity of the region, respectively. And the horizontal direction component and the vertical direction component of the distance (number of pixels).
Further, σ _x and σ _y are fixed parameters of appropriate values which are parameters for adjusting the degree of influence on the importance ps in the horizontal direction and the vertical direction, respectively.

The importance ps has a larger value as the center of gravity of the area is closer to the center of the screen, and conversely has a smaller value as the center of the screen is closer to the periphery of the screen.

The parameters σ _x and σ _y may be fixed to appropriate values as described above, and may be adaptively changed according to the shape of the skin region candidate.

That is, the shape of the skin area candidate is, for example, as shown in FIG.
As shown in (a), in the case of the portrait orientation, the horizontal shift has a great influence on the importance of the skin region candidate on the screen, and the vertical shift has little effect on it. it is conceivable that. Therefore, in this case, the parameter σ _{y in the} vertical direction is set to a large value so that the importance ps does not change significantly due to the vertical shift of the skin region candidate.

For example, as shown in FIG. 6B, when the shape of the skin area candidate is horizontally long, the horizontal parameter σ _x is set to a large value for the same reason as above. .

Further, assuming that the unit of d _x , d _y , σ _x , σ _{y in} the equation (7) is the number of pixels, it is necessary to consider the aspect ratio of the pixels of the screen on which the input image is displayed. There is. This is because if the aspect ratio of the pixels of the screen on which the input image is displayed is not 1: 1, the skin area candidate is displaced from the position actually displayed on the screen and the position calculated from the number of pixels,
This is because the importance ps obtained by the equation (7) does not correctly reflect the importance of the skin area candidate on the screen.

Next, regarding the size of the hole, a value obtained by dividing the size of the _hole (the number of pixels) by the size of the skin area candidate (the number of pixels) is a predetermined threshold value Th _hole (however, Th _hole is 0.
If it is larger than 0 and equal to or less than 1.0 and is larger than a predetermined value, the area is excluded from the skin area candidates.

Here, a method for obtaining the size of the hole in the area will be described with reference to FIG. First, as shown in FIG. 7A, the skin area candidate is horizontally scanned from the left and right sides, and the leftmost pixel P forming the area in each line is scanned.
L _v the most right pixel PR _v to detect. However, the subscript v represents the line number.

Then, the number of non-skin color pixels existing between PL _v and PR _v is counted for each line, and the total number of pixels obtained for all lines is Nh.

Further, as shown in FIG. 7B, scanning is performed in the vertical direction, and the total number Nv of the non-skin color pixels obtained for all the lines is obtained in the same manner as described above. The larger one of the obtained Nh and Nv is set as the size of the hole.

In this method, sizes other than those that can be strictly called holes are also detected. For example, in the case of the area shown in FIG. 7C, the hatched portion is not a hole in the strict sense, but the size is calculated as a hole in this method. However, it is considered that there is no problem in sensuously considering such an area as a hole.

The scanning in both the horizontal and vertical directions is performed by the above method for the following reason. That is, for example, in FIG.
When only the horizontal scanning is performed on the area as shown in (c), what is detected as a hole is a shaded portion in FIG. 7D, but this is a sensational hole. Does not correctly reflect the size of.

Therefore, in this case, the one obtained by scanning in the vertical direction (FIG. 7 (c)), which correctly reflects the size of the sensory hole, is used. On the contrary, since it is possible that horizontal scanning is effective,
As described above, the larger one of the two results is used here.

A set of labels (t ₀ ′, t ₁ ′, t ₂ ′, ...) Of skin region candidates remaining by the threshold processing of the region size, position, and hole size as described above. Is output from the area analyzer 24 to the histogram generator 21 (FIG. 5).

In the histogram generator 21, the position (address) of the pixel forming the skin region candidate corresponding to each label from the region analyzer 24 is detected from the frame memory 5 (FIG. 1) and the position (address) is detected. Based on this, the pixel brightness L and chroma C are read from the frame memory 2.
Then, from these data, chroma / luminance (C / L)
Is calculated and its histogram is generated for each skin area candidate.

The C / L histogram of each skin area candidate generated by the histogram generator 21 is input to the histogram analyzer 22, where the histogram features are:
The position of the peak, the width of the peak, and the number of peaks are detected.

Further, in the histogram analyzer 22, based on the peak position, the peak width, or the number of peaks of the C / L histogram of the skin region candidate, the three indices of the so-called skin-likeness of the skin region candidate, That is, the peak position skin index posIdx, the peak width skin index widIdx,
Alternatively, the peak number skin index numIdx is calculated as follows.

(A) Peak position The peaks of the histogram of the skin index posIdx C / L are considered to be present at a plurality of positions as shown in FIG. 8 (a). And let it be the position of the peak, peakPos.

That is, the positions p ₁ , p ₂ ,. ． ． , P _N , N peaks are obtained, and the histogram frequencies at the respective positions are f ₁ , f ₂ ,. ． ． , F _N , the position of the peak is calculated by the following equation. peakPos = Σ (p _i × f _i ) / Σf _i where Σ means the summation of i = 1 to N.

Then, this peak position peakPos
However, for example, according to the conversion graph shown in FIG.
It is converted to the peak position skin index posIdx.

That is, the fact that the peak position of the C / L histogram on the skin does not take a large value empirically is used, and the peak position skin index posIdx is the peak position p.
When eakPos is less than or equal to the predetermined threshold Th _peakpos, it is set to 1.0 as the maximum value, and the threshold Th
If _peakpos greater than the peak position peakPo
A small value is set according to the difference between s and the threshold Th _peakpos .

(B) Peak width skin index widIdx As shown in FIG. 9A, first, the maximum frequency f _max in the histogram is detected, and the value of α% f α (= f _max)
× α / 100) is calculated. Then, the histogram is scanned from both left and right, the positions at which the frequency first becomes fα are p _L and p _R , respectively, and the difference between these values (p _R −p _L ) is the peak spread (peak Width) peak
Wid.

The width of this peak, peakWid
However, for example, according to the conversion graph shown in FIG.
Converted to peak width skin index widIdx.

That is, since the peak spread does not become so large in the skin region, the peak width skin index widIdx.
Indicates that the value of the peak width peakWid is a predetermined threshold Th.
if: _{peak wid} is set to 1.0 as a maximum value, when the threshold Th _Peakwid larger, depending on the difference between the width peakWid and the threshold Th _Peakwid peak,
Set to a small value.

(C) Number of peaks Skin index numIdx The number of peaks is considered to be one of the features of the histogram, but it is difficult to find it correctly. This is because which of the maximum values existing in a large number in the histogram is detected as a peak depends on the detection method, and if the parameters such as the threshold value are different, the number easily changes.

Therefore, in the present embodiment, the value obtained as follows instead of the number of detected peaks is taken as the number of peaks peakNum in the histogram.

That is, for example, when a histogram as shown in FIG. 10A is given, the number of peaks peak is calculated from the areas a ₁ and a _{2 of the two} areas A ₁ and A ₂ according to the following equation.
Num is calculated. peakNum = a ₁ / (a ₁ + a ₂ ) ... (8)

In FIG. 10 (a), the area A ₁ is a portion (shaded portion) sandwiched between the polygonal line obtained by connecting the adjacent peaks detected on the histogram with a straight line and the envelope of the histogram. The area A ₂ is a portion sandwiched between the envelope of the histogram and the horizontal axis.

Therefore, the number peakNum of peaks obtained from the equation (8) increases as the number of peaks in the histogram increases, but also depends on the positional relationship of each peak. The values are small when the respective peaks are located close to each other, but the values increase as the peaks are separated from each other. If there is one peak, the area A ₁ does not exist, and the number of peaks pe
akNum is also zero. From these things, equation (8)
It is considered that the number of peaks PeakNum defined in 1 above represents the multimodality of the peaks, has a correlation with the number of peaks, and also includes ambiguity in the number of peaks.

Then, the number peakNum of this peak
Is converted into the peak number skin index numIdx according to the conversion graph shown in FIG. 10B, for example.

That is, utilizing the fact that the number of peaks is one in the skin region, the peak number skin index numIdx is set to 1.0 when the number of peaks peakNum is 0, and the number of peaks peakNum is It is set to a smaller value as it gets larger. Therefore, the peak number skin index num
Idx can be considered, so to speak, as 1.0 minus multimodality.

The peak position skin index posIdx, the peak width skin index widIdx, and the peak number skin index numIdx calculated as described above are sent from the histogram analyzer 22 to the skin area determiner 23 (FIG. 5). The skin area determiner 23 determines the peak position skin index po of each skin area candidate.
From sIdx, peak width skin index widIdx, and peak number skin index numIdx, a comprehensive skin index skinIdx of the skin region candidate is calculated according to the following equation. skinIdx = A × posIdx + B × widIdx + C × numIdx (9)

However, A, B, and C are weighting factors, which are obtained by the least squares method so that the value of the equation (10), that is, the squared error σ ^2, is minimized using the actual image as the learning data. σ ² = Σ (1.0-skinIdx) ² + Σ (0.0-skinIdx) ² (10) Note that Σ in the first term on the right side of Expression (10) in the actual image as learning data , Means the summation regarding the skin index skinIdx calculated from the actual skin area,
The second term Σ is a skin index s calculated from a region other than the actual skin region in the real image as the learning data.
It means the summation regarding kinIdx. Therefore, the skin index skinIdx indicates that the closer the value is to 1.0, the more skin-like.

Skin index skinId according to equation (9)
After calculating x, the skin area determiner 23 further thresholds the skin index skinIdx of each skin area candidate, that is, the skin index s.
It is determined whether or not kinIdx is larger than a predetermined threshold and the skin area candidate can be regarded as a skin area. When the skin area determiner 23 determines that at least one of the skin area candidates can be regarded as a skin area,
That is, when at least one of the skin region candidates is determined as the skin region, the control signal cntl1 is output to the skin image determiner 7 (FIG. 1).

When the skin image determiner 7 receives the control signal cntl1 from the skin area detector 6 (skin area determiner 23), the skin image determiner 7 determines that the image input to the apparatus is a skin image including skin, and makes a determination. The resultant 0 and 1 of 1 are output to a processing circuit (not shown).

In the processing circuit (not shown), when the determination result that the input image is the skin image is received from the skin image determiner 7, the image quality of the skin area is optimized for the input image, for example. , The image quality of the input image is adjusted.

As described above, four parameters having different parameters are used.
FIG. 1 shows a result of an experiment (simulation) for determining whether or not the types of still images P1, P2, P3, P4 are skin images.
Shown in 1.

In the figure, the correct answer rate is calculated by the following equation. Correct answer rate = number of correct answers / total number of images However, the number of correct answers is the number of times that a determination result that the skin image includes the skin is obtained and that the determination result that the image other than the skin image does not include the skin is included. It is the sum of the numbers obtained.

From the figure, it can be seen that it is almost completely identified whether the image is a skin image or not.

It should be noted that one of the 25 P2 images was erroneously determined, but this is a face image of a black woman. Since the thresholds and other coefficients are set as shown, it can be said that it is unavoidable that this image cannot be identified as a skin image.

By the way, when the apparatus shown in FIG. 1 is applied to a still image, an effective result is obtained as described with reference to FIG. Moreover, since it is determined whether or not the moving image is a skin image, there is a problem with respect to temporal continuity of the moving image.

That is, when the size of the skin area gradually increases, such as a scene in which a person zooms in, a skin image is not initially determined because the skin area is small, but At the moment when the skin area reaches a certain size, a situation occurs where the skin image is suddenly determined. In this way, the judgment result suddenly changes (reverses) in the middle of the process, even though the image changes continuously.
In a sense, it can be said that the temporal continuity is not correctly reflected in the determination result for the moving image.

Therefore, FIG. 13 is a block diagram showing the configuration of the second embodiment of the image identifying apparatus to which the present invention is applied.
This device is based on a skin area importance calculator 31 for calculating the importance of the skin area candidates detected by the skin area candidate detector 4 or an importance output from the skin area importance calculator 31. A skin image importance degree calculator 32 that calculates the degree of importance as a determination result of whether an image is a skin image is provided instead of the skin area detector 6 or the skin image determiner 7, respectively. It has the same configuration as the image identification device,
The determination result as to whether or not the image is a skin image is output as importance Sidx that takes continuous values.

In the image discriminating apparatus having such a configuration, the RG for each frame or field of a moving image is
When the B data is input, the same processing as in FIG. 1 is performed in the coordinate converter 1, the frame memory 2, the skin color pixel detector 3, the skin area candidate detector 4, and the frame memory 5, and the skin area is processed. The candidates t ₀ , t ₁ , ... Are output from the skin region candidate detector 4 to the skin region importance calculator 31.

The skin area importance degree calculator 31 is, for example, as shown in FIG.
Each of the skin area candidates t detected by the skin area candidate detector 4 is composed of a histogram generator 41, a histogram analyzer 42, a skin area determiner 43, an importance calculator 44, and an area analyzer 45 as shown in FIG. Importance of imp [t]
To calculate.

That is, in the skin area importance calculator 31, first, in the area analyzer 45, the skin area detector 6 shown in FIG.
Processing similar to the processing in the area analyzer 24 of is performed. In other words, the area analyzer 45 obtains the size, position, and hole size of each skin area candidate formed by the same label, and performs threshold processing.

However, in the area analyzer 45, the threshold processing is
Only done by the size of the hole. That is, first, the size of the hole (the number of pixels) is divided by the size of the skin region candidate (the number of pixels), and a region whose value is larger than a predetermined threshold Th _hole is excluded from the skin region candidates. Then, as a result, a set of labels (t ₀ ′, t ₁ ′, t ₂ ′, ...
..) is the area analyzer 45 to the histogram generator 4
It is output to 1.

Further, in the area analyzer 45, the size of the area and the coordinates of the center of gravity are described with the label t _m of the skin area candidate remaining as a result of the threshold processing as an index.
For example, a table as shown in FIG. 15 (hereinafter referred to as a parameter table) is created. This parameter table is output to the importance calculator 44 and stored therein.

In the histogram generator 41 or the histogram analyzer 42, the histogram generator 21 or the histogram analyzer 22 of the skin area detector 6 shown in FIG. 5 is used.
The same process is performed for each of the skin area candidates, the peak position skin index posIdx, and the peak width skin index widId.
x and the peak number skin index numIdx are output to the skin area determining unit 43.

The skin area determiner 43 operates in the same manner as in the skin area determiner 23 of FIG.
Peak position skin index posId of each skin region candidate from 2
Based on x, the peak width skin index widIdx, and the peak number skin index numIdx, the comprehensive skin index skinIdx of each skin region candidate is calculated according to Expression (9), and threshold processing is performed. Then, the remaining skin area candidates, that is, the skin area labels (t ₀ ″, t ₁ ″, ...) Are output from the skin area determiner 43 to the importance calculator 44.

In the importance calculator 44, first, the area analyzer 4
5, the size and barycentric coordinates of each area corresponding to the label output from the skin area determiner 43 are read from the parameter table (FIG. 15) sent from FIG. 5, and the importance level imp [t] of each area t is read. It is calculated according to the following formula. imp [t] = area [t] × ps [t] (11) where area [t] indicates the size (number of pixels) of the area of the label t. Further, ps [t] is a value calculated from the barycentric coordinates of the skin area of the label t according to the above equation (7).

The degree of importance imp [t] increases as the area [t] or ps [t] increases, that is, the size of the area increases, and the position of the area on the screen (the position of the center of gravity) is in the center. Takes a value.

The importance level imp [t] for each skin area is
It is output from the skin area importance calculator 31 (importance calculator 44) to the skin image importance calculator 32 (FIG. 13).

In the skin image importance calculator 32, the importance imp for each skin area from the skin area importance calculator 31 is calculated.
[T] are sequentially added, and the importance of the input image as a skin image (hereinafter referred to as skin image importance) Sidx is calculated according to the following equation.

Sidx = Σimp [t] / nL However, when Sidx> 1.0, Sidx = 1.0 (12) Here, Σ is the summation regarding all the labels of the region remaining as the skin region. Means Further, nL is simply a constant for normalization, and therefore, the size of the skin image importance Sidx is essentially determined by the numerator on the right side of Expression (12), that is, the importance of each skin region. Sum (Σimp
[T]).

The skin image importance Sidx is the importance imp
For images containing many skin areas with large [t],
The value becomes zero for an image in which the value becomes large and the skin region is not included at all. The skin image importance level Sidx is output to a processing circuit (not shown) as a result of identifying whether or not the input image is a skin image.

In the processing circuit (not shown), the skin image importance degree Sidx as a determination result as to whether the input image is a skin image is received from the skin image importance degree calculator 32. The image quality is adjusted according to the skin image importance level Sidx.

That is, for example, the chroma C _{old of the} input image is converted into a _new chroma C _new according to the following equation, corresponding to the skin image importance Sidx.

For the skin color area, C _new = (1.0-Sidx) × (k−1) +1) × C _{old For} other areas, C _new = k × C _old Then, the flesh-colored area means, for example, the entire area composed of the flesh-colored pixels detected by the flesh-colored pixel detector 3. Further, k is a predetermined constant of 1 or more. ... (13)

Here, for many objects appearing in a natural image, it tends to be preferred that the chroma of the color is higher than the actual one. This tendency is particularly remarkable for so-called three primary colors such as blue in the sky and green in plants. However, this tendency does not apply to the color of human skin, and it is likely that the chroma is relatively low as compared with chromas of other colors.

According to the above equation, chroma other than skin color is always k
(> 1.0) times, while the skin chroma is set to a value corresponding to the skin image importance Sidx of the image, that is, the skin chroma is not increased as the skin image importance Sidx is larger. Therefore, it is possible to provide an image that gives a good impression.

Further, when this apparatus is applied to a moving image which zooms in on a person, for example, the skin image importance level Sid of continuously changing images as shown in FIG.
It is possible to obtain x, that is, it is possible to obtain a determination result that reflects the temporal continuity of the moving image, and using this skin image importance Sidx, the moving image of the moving image is calculated according to Expression (13). By converting the chroma, as the skin area of a person gradually increases, the chroma of that area can be continuously decreased (relative to the background chroma). It is possible to prevent the image processing from suddenly changing the impression of.

In this embodiment, RGB data of an image is input, but for example YUV data or L ^* a ^* is input ^.
b ^* Data can be input. However, in this case, it is necessary to change the configuration of the coordinate converter 1 so that YUV data and L ^* a ^* b ^* data can be converted into LHC data.

Further, in the present embodiment, the C / L histogram is used to determine the skin area from the skin area candidates, but it is conceivable to use the luminance, hue, or chroma histogram. To be

However, when about 90 images were used and luminance, hue, and chroma histograms were created for the skin area and the other areas of each image, the histograms for the skin area and the other areas were obtained. There is no noticeable difference in the luminance, hue, and chroma histograms, so the luminance, hue, and chroma histograms show that
Experimental results have been obtained that it is difficult to distinguish the skin region from the other regions.

On the other hand, an example of a C / L histogram created in the same manner is shown in FIG. FIG. 17A shows the skin area, and FIGS. 17B and 17C show the area other than the skin area.

Assuming that the spectral reflectance is constant in a certain skin region, the C / L histogram has (C1) one peak, and (C2) the spread (width) of the peak is not so large. The position of C3) peak can be considered to have the property that it does not depend on the brightness and varies to some extent depending on individual differences, but there is a range that can be taken as skin.

As a result of the experiment, the C / L histogram of the skin region almost satisfies the above properties (C1) to (C3), while the C / L histogram of the region other than the skin region is
At least one of the above properties (C1) to (C3)
Since it does not satisfy the requirement, it is effective to use the C / L histogram in distinguishing the skin region from the other regions.

Further, in the present embodiment, the area composed of the pixels detected as the skin color pixels by the skin color pixel detector 3 is set as the skin area candidate, but the algorithm for obtaining the skin area candidate is not limited to this. Not a thing. That is, for example, the skin area candidate can be obtained based on a more sophisticated algorithm. In this case, it is possible to improve the identification accuracy of whether or not the image is a skin image, but on the other hand, the amount of calculation and the cost of the device increase. It is necessary to decide the algorithm for obtaining.

Further, in this embodiment, the skin image importance Sidx is calculated by using the sum Σimp [t] of the importance of each skin region as shown in the equation (12).
It is not limited to this. That is, for example, as shown in the following equation, the maximum value max _t (imp
The skin image importance level Sidx can be calculated using only [t]).

Sidx = max _t (imp [t]) / nL However, when Sidx> 1.0, Sidx = 1.0 (14)

However, in a moving image in which a plurality of skin areas are connected or one skin area is separated into two, such as a scene where two people pass each other, the maximum is Of importance max _t (imp [t])
There is a case where the area that becomes is suddenly changed, and thereby, the value of the skin image importance Sidx calculated according to the equation (14) also suddenly changes.

Therefore, when the expression (14) is used, the skin image importance Sidx may become discontinuous in time, so to speak, the expression (1
It is preferable to use 2).

In this embodiment, the skin image importance Sid is
Although x is used as it is in a processing circuit (not shown), the skin image importance level Sidx may be used after being subjected to processing such as smoothing in the time axis direction.

Further, in this embodiment, the peak position skin index posIdx, the peak width skin index widIdx, and the peak number skin index numIdx obtained based on the C / L histogram of the skin region candidates are used only for the threshold processing. However, the feature amount indicating the degree of skin-likeness may be considered when calculating the skin image importance Sidx.

[0142]

According to the image identification method of the first aspect, the pixels of the skin color are extracted, and the candidate of the skin region is detected from each of the pixels. Then, the skin region is determined from each candidate of the skin region, and it is determined whether the image is a skin image based on the determination result. Therefore, it is possible to accurately identify the skin.

According to the image identification method of claim 2,
The determining means, based on the importance of each candidate of the skin area, from among the candidates of the skin area, to determine the skin area, the determining means, based on the determination result of the determining means, the image, It is determined whether the image is a skin image, and a binary determination result indicating whether the image is a skin image is output. Therefore, this method
For example, when applied to a still image, the skin of a person in the still image can be accurately identified.

According to the image identification system of claim 3,
The determining means determines the skin area and calculates the degree of importance thereof. Then, the determination unit is caused to calculate the certainty that the image is a skin image based on the importance of the skin region calculated by the determination unit, and outputs the determination result. Therefore, when this method is applied to moving images,
The skin of a person in a moving image that changes with time can be identified without impairing temporal continuity.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of an image identification device to which the present invention has been applied.

FIG. 2 is a block diagram showing a more detailed configuration of a skin color pixel detector 3 of the embodiment of FIG.

FIG. 3 is a diagram illustrating labeling processing of a skin area candidate detector 4 in the embodiment of FIG.

FIG. 4 is a diagram illustrating re-labeling processing of a skin area candidate detector 4 in the embodiment of FIG.

5 is a block diagram showing a more detailed configuration of a skin area detector 6 of the embodiment of FIG.

FIG. 6 is a diagram illustrating the relationship between the shape of a region and the importance of the region on the screen.

FIG. 7 is a diagram illustrating holes in a region.

FIG. 8 is a diagram illustrating a relationship between a peak position and a peak position skin index.

FIG. 9 is a diagram illustrating a relationship between a peak width and a peak width skin index.

FIG. 10 is a diagram illustrating a relationship between a peak number and a peak number skin index.

FIG. 11 is a diagram showing experimental results according to the example of FIG. 1.

12 is a diagram showing threshold values and coefficients used in obtaining the experimental results of FIG.

FIG. 13 is a block diagram showing a configuration of a second embodiment of an image identification device to which the present invention has been applied.

14 is a block diagram showing a more detailed configuration of the skin area importance calculator 31 of the embodiment of FIG.

FIG. 15 is a diagram showing a parameter table.

16 is a diagram showing a skin image importance Sidx of a moving image, which is output from a skin image importance calculator 32. FIG.

FIG. 17 is a diagram showing a histogram of chroma / brightness of an image.

FIG. 18 is a diagram illustrating conventional extraction of a skin area based on a color space.

[Explanation of symbols]

 1 Coordinate converter 2 Frame memory 3 Skin color pixel detector 4 Skin area candidate detector 5 Frame memory 6 Skin area detector 7 Skin image determiner

Claims (3)

[Claims] 1. An image identification method for determining whether or not an image is a skin image including a skin area corresponding to a person's skin, and extracting means for extracting skin color pixels; From each pixel of the skin color, a detection unit that detects a candidate of the skin region, a determination unit that determines the skin region from among the candidates of the skin region detected by the detection unit, and the determination unit Based on the determination result of 1., the image identification method, comprising: determining whether the image is the skin image and outputting the determination result. 2. The determining means calculates the importance of each candidate of the skin area, and determines the skin area from each candidate of the skin area based on the importance. The determination means, based on the determination result of the determination means,
The image according to claim 1, wherein it is determined whether the image is the skin image, and a binary determination result indicating whether the image is the skin image is output. Identification method. 3. The determining means determines the skin area from each candidate of the skin area and calculates the importance thereof, and the determining means determines the skin calculated by the determining means. 2. The image identification method according to claim 1, wherein the likelihood that the image is the skin image is calculated based on the importance of the area, and the image is output as the determination result.