JP2009205469A - Extraction system and method of object area - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an extraction system and an extraction method of an object area, robust in white balance, allowing real time processing. <P>SOLUTION: This extraction system 1 of the object area includes: a color image acquisition means 2 acquiring a color image of an object; a first object area extraction means 31 extracting a first object area from the color image by use of a chromaticity component histogram and a hue histogram; a second object area extraction means 32 extracting a second object area by use of a chroma histogram; a third object area extraction means 33 extracting a third object area by use of a luminance histogram; and an object area composition means 34 obtaining an overlap area between the first to third object areas as the object area. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an object region extraction system and method. Specifically, the present invention relates to an extraction system and an extraction method that are robust to changes in white balance when extracting an object region such as a skin region from a color image.

In recent years, visual communication using a personal computer (PC) can be easily introduced even in general households, and has begun to spread. For example, a method has been proposed in which the area of continuous skin color regions in color image data is obtained and the face region is obtained for each skin color region in consideration of features such as the total number of skin color pixels and the width in the vertical and horizontal directions. In addition, as an elemental technique of image processing technology for extracting only a person from an image and replacing the background with an image prepared in advance, the inventors have developed a perceptual chromaticity a ^* histogram and hue h _{ab in} an L ^* a ^* b ^* color space. We proposed a method for extracting human skin areas that are robust to the effects of lighting using histograms. Considering the ambiguity of image information and lip contour, the fuzzy inference process is used to determine the lip area and skin area of a person without using a template (reference image information stored in the database). A method to extract was proposed. (See Patent Document 1 and Non-Patent Documents 1 to 4)

JP 2000-187721 A (paragraphs 0016 to 0060, FIGS. 1 to 10) Takashi Sai, Satoshi Nishida, Yoichi Kageyama, Kenta Otsuki, “Study on face area extraction using color information”, National Convention on Information Processing, 5P-1 (2007). Yoichi Shirasawa and Kenta Otsuki, “Study on Lip Tracking Focusing on Color Difference”, IEICE General Conference, D-12-89 (2007). Yoichi Shirasawa, Kei Nishida, Kenji Nishi, “Examination of Lip Shape Extraction by Fuzzy Reasoning Using Color Information”, IEEJ Transactions C, Vol. 125-C, no. 9, pp. 1430-1437. Yoichi Shirasawa, Atsushi Nishida, Fumi Kurisu, “Estimation of Lip Position Robust to Zoom and Face Direction Changes”, Journal of the Institute of Image Information and Television Engineers, Vol. 59, no. 12, pp. 1616-1821.

  However, the fluctuation of white balance is large in a digital camera or the like that is a familiar camera device in a general home PC environment. Conventionally, although a skin extraction method using a fixed threshold has been reported, it has been difficult to cope with white balance. Conventionally, processing using a template has been mainstream, and real-time processing has been difficult.

  An object of the present invention is to provide an object region extraction system and an extraction method that are robust to white balance and can be processed in real time.

  In order to solve the above problems, an object region extraction system 1 according to a first aspect of the present invention includes, as shown in FIG. 1, for example, a color image acquisition unit 2 that acquires a color image of an object, First object region extraction means 31 for extracting a first object region using a chromaticity component histogram and a hue histogram, and a second object region for extracting a second object region from a color image using a saturation histogram An extraction unit 32; a third object region extraction unit 33 that extracts a third object region from a color image using a luminance histogram; and a first object region, a second object region, and a third object region. An object area synthesis means for obtaining an overlapping area as an object area. .

Here, a histogram in any color space is used as the chromaticity component histogram, the hue histogram, the saturation histogram, and the luminance histogram. As the color space, for example, in addition to the HSV color space (hue hue, saturation saturation value, and lightness value are used as indices), the HLS color space (including the luminance luminance in the index), a uniform color space (Uniform Color Space: UCS) , A color space that focuses on differences in human perception, and is a color system specified by the CIE (International Commission on Illumination). L ^* a ^* b ^* color space, L ^* u ^* v ^* color space Included). The hue histogram, saturation histogram, and luminance histogram are feature quantities of the HSV color space, the HLS color space, the L ^* a ^* b ^* color space, and the L ^* u ^* v ^* color space. However, although it is a feature amount of the L ^* a ^* b ^* color space and the L ^* u ^* v ^* color space, the chromaticity component histogram is not a feature amount of the HSV color space and the HLS color space. Therefore, four histograms of hue histogram, perceptual chromaticity histogram, saturation histogram, and luminance histogram in one color space such as L ^* a ^* b ^* color space, L ^* u ^* v ^* color space may be used. Since there is no chromaticity component histogram in the color space or HSV color space, the saturation histogram and luminance histogram of the HLS color space or HSV color space are used, and the L ^* a ^* b ^* color space or the L ^* u ^* v ^* color space. A plurality of color spaces may be used together, such as using a hue histogram and a chromaticity component histogram.

The L ^* a ^* b ^* color space is the luminance L ^{* in the} z direction, the perceptual chromaticity a ^* relating to reddish-greenishness in the x-axis, and the perceptual chromaticity b ^* relating to yellowish-blueness in the y-axis This is a color system in which the circumferential direction is a hue (metric hue angle) h _ab . An achromatic color is arranged on the z-axis. Also, since the L ^* u ^* v ^* color space is a color system that uses perceptual chromaticity u ^* and v ^* and the uniformity of the wavelength interval is taken into account, if the specific wavelength direction is adopted in the x direction, the wavelength It is possible to express the intensity of the color with respect to. The perceived chromaticity can be said to be a chromaticity component having equal color difference, and the perceived chromaticity may be used as the chromaticity component. The perceptual chromaticity histogram is a histogram indicating the intensity of the perceptual chromaticity (hue) relating to a specific hue. When the object is skin, it is preferable to use an a ^* histogram indicating the perceived intensity of redness. is there. The perceptual chromaticity depends on the hue and the saturation, but is represented by an x coordinate value having two perceptual chromaticities (for example, a ^* and b ^* ) as xy axes. In addition, a histogram indicating the perceived intensity of blue, a histogram indicating the perceived intensity of yellow, a histogram indicating the perceived intensity of green, and the like can be selected as appropriate according to the object. In addition, the luminance is a lightness weighted by RGB (it is said to be close to the light seen from the human eye), and a lightness histogram may be used instead of the luminance histogram.

  With this configuration, an object area extraction system that is robust to white balance because an overlapping area with the first to third object areas is obtained as an object area, and that can perform real-time processing without using a template. Can provide.

  In the object region extraction system according to the second aspect of the present invention, in the first aspect, the object is a human skin. With this configuration, the skin region is frequently used for color images including captured images and is strongly influenced by white balance. Therefore, it is practical to apply the first aspect that is robust to white balance.

The object region extraction system according to the third aspect of the present invention, in the first or second aspect, uses the chromaticity component histogram as an a ^* histogram in the L ^* a ^* b ^* color space as a hue histogram. Use a h _ab histogram in the L ^* a ^* b ^* color space. When configured in this manner, the L ^* a ^* b ^* color space is robust to the influence of illumination when extracting both the human skin area and the lip area, and thus is suitable for extracting the human skin area.

  The object region extraction system according to the fourth aspect of the present invention uses fuzzy reasoning for extraction of the first object region in any of the first to third aspects. With this configuration, it is possible to automatically cope with the ambiguity of the color associated with the change in the shooting environment, which is efficient for extracting the first object region.

  The object region extraction system according to the fifth aspect of the present invention is a memory for storing a chromaticity component histogram, a hue histogram, a saturation histogram, and a luminance histogram for an object in any one of the first to fourth aspects. Means 4 are provided. With this configuration, the color of the measured pixel can be compared with the histogram stored in the storage unit 4, and it is efficient for extracting the first to third object areas.

  In the object region extraction system according to the sixth aspect of the present invention, in the fifth aspect, the storage unit 4 stores the saturation histogram and the luminance histogram in association with the color temperature. With this configuration, it is possible to efficiently cope with a change in white balance.

  In order to solve the above problem, an object region extraction method according to a seventh aspect of the present invention includes a color image acquisition step (S010) for acquiring a color image of an object, as shown in FIG. A first object region extraction step (S030) for extracting a first object region from the color image using a chromaticity component histogram and a hue histogram, and a second object for extracting a second object region from the color image using a saturation histogram. Object region extraction step (S040), a third object region extraction step (S050) for extracting a third object region from a color image using a luminance histogram, a first object region and a second object region, An object for which an overlapping area with the third object area is obtained as an object area And a preparative area combining step (S060).

  Here, the process order can be changed as appropriate. For example, the first to third object region extraction steps may be performed in any order and may be performed in parallel. If comprised in this way, the extraction system and extraction method of the object area | region which are robust to white balance and can be processed in real time can be provided.

  A program according to the eighth aspect of the present invention is a program for causing a computer to execute the object region extraction method according to the seventh aspect.

  ADVANTAGE OF THE INVENTION According to this invention, the extraction system and extraction method of the object area | region which are robust to white balance and can be processed in real time can be provided.

  Embodiments of the present invention will be described below with reference to the drawings.

[First Embodiment]
In the first embodiment, the object is the skin of a person, particularly the skin of the face, the chromaticity component histogram is the perceptual chromaticity a ^* histogram and the hue histogram is L ^* a ^{* in the} L ^* a ^* b ^* color space ^. An example of a metric hue angle _hab histogram in the b ^* color space will be described.

[System configuration]
FIG. 1 shows a configuration example of an object area extraction system. In FIG. 1, reference numeral 1 denotes an object area extraction system, which can be configured by installing a program in a personal computer (PC), for example. In the present embodiment, the object is human skin, the perceptual chromaticity a ^* histogram in the L ^* a ^* b ^* color space is used as the chromaticity component histogram, and the metric in the L ^* a ^* b ^* color space is used as the hue histogram. An example using the hue angle _hab histogram will be described. Reference numeral 2 denotes color image acquisition means for acquiring a color image of an object. For example, a digital camera such as a CCD camera or other camera capable of obtaining a color image is photographed and obtained as digital information by the color image obtaining means 2. The color image acquisition means 2 may include such a camera, or may not include a camera and may acquire an image taken with another camera.

Reference numeral 3 denotes an object area extracting means, which has first to third object area extracting means 31 to 33, an object area synthesizing means 34, and a part area extracting means 35. The first object region extraction unit 31 extracts a first object region from the color image acquired by the color image acquisition unit 2 using a perceptual chromaticity histogram and a hue histogram. The second object area extraction means 32 extracts a second object area from the color image using a saturation histogram. The third object area extraction unit 33 extracts a third object area from the color image using a luminance histogram. The object area synthesizing unit 34 obtains an overlapping area of the first to third object areas as an object area as a final result, whereby the object area is determined. The part area extracting unit 35 extracts, for example, a lip, which is a part of a face, before extracting a skin area for a person's skin, which is an object, and facilitates extraction of a face area adjacent to the lip. As the lip extraction method, the same method as the first object region extraction method can be used. For this reason, the part area extraction means 35 may be included in the first object area extraction means. The part area extracting means 35 may not be provided, and it is not particularly necessary to obtain a skin area other than the face such as a limb. Note that the saturation histogram and the luminance histogram may be a histogram in the L ^* a ^* b ^* color space or a histogram in the HLS color space.

  A storage unit 4 stores the color image acquired by the color image acquisition unit, the images extracted by the first to third object area extraction units 31 to 33, and the image synthesized by the object area synthesis unit. By storing these histograms and images in the storage means 4, for example, the color of the measured pixel can be compared with the histogram stored in the storage means 4, and the extraction of the first to third object areas can be made efficient. . Input / output means 5 has input means such as a mouse and keyboard, and output means such as a display and a printer. Reference numeral 6 denotes a calculation means, which appropriately performs calculations necessary for object extraction in accordance with instructions from the control means 7. The control means 7 has a program for the object area extraction method, and controls the entire object area extraction system 1 and each means 2 to 6 to execute the object area extraction method.

[Processing flow]
FIG. 2 shows a processing flow example of the object region extraction method. First, a color image including a human skin as an object is acquired (S010). For example, a color image is taken by a camera that can obtain a color image, which is acquired by the color image acquisition means 2 as digital information. Next, the lip region is detected from the acquired color image by the component region extraction means 35 (S020). The lip area is extracted prior to the skin area because the lip area is easier to identify the features than the other areas and is adjacent to the face area. This is because it can be extracted. Focusing on the L ^* a ^* b ^* color space, which has a feature that shows the difference between facial skin and lips, and is robust to the effects of lighting, perceptual chromaticity a ^* histogram and hue (metric hue angle) _hab histogram Used to detect lips by fuzzy reasoning and acquire color information of the lip peripheral area. Then, in the first object region extraction unit 31, a skin area of a first object region extracted with the same perception chromaticity a ^* histogram and hue h _ab histogram (S030). That is, since the distribution range of the two histograms is divided between the lips and the skin, both can be extracted separately. Since the perceptual chromaticity a ^* histogram and hue h _ab histogram are not easily affected by the color temperature (white balance), a robust first object region can be extracted. Also, if there is ambiguity in the lip area or skin area, if fuzzy reasoning is used for extraction, it is possible to automatically cope with the ambiguity of the color due to changes in the shooting environment, so the object area can be extracted efficiently . Next, the skin area as the second object area is extracted by the second object area extracting means 32 using the saturation histogram (S040). In addition, the third object area extraction unit 33 extracts a skin area as the third object area using the luminance histogram (S050). Since the saturation histogram and the luminance histogram change depending on the color temperature, correction according to the white balance can be performed using the saturation histogram and the luminance histogram. The extraction of the first to third object areas may be performed in any order and may be performed in parallel. Next, the object area synthesizing unit 34 obtains the overlapping area of the first to third object areas as the final object area (S060). As a result, since the overlapping area with the first to third object areas is obtained as the object area of the final result, the object area extraction that is robust to white balance and does not need to use a template and can be processed in real time is extracted. Can provide a method. Hereinafter, a method for extracting an object area in the present embodiment will be described in detail.

[Face parts (lip) detection]
First, a characteristic lip portion is detected in the face portion. The part area extracting means 35 detects the lip portion. Focusing on the L ^* a ^* b ^* color space, which has a feature that shows the difference between facial skin and lips, and is robust to the influence of lighting, using perceptual chromaticity a ^* histogram and hue _hab histogram, and fuzzy inference The lip is detected, and color information about the lip peripheral area is acquired. Next, a skin region is extracted.

Shows the metric hue angle h _ab histogram as perceived chromaticity a ^* histogram and hue histogram as chromaticity component histogram in lip surrounding area and the area in Figure 3. FIG. 3 (a) shows the lip peripheral region, FIG. 3 (b) shows the a ^* histogram, and FIG. 3 (c) shows the _hab histogram. The perceptual chromaticity a ^* is an index indicating the intensity of redness. The vertical axis of each histogram indicates the frequency at which pixels appear. As shown in FIG. 3A, as the lip peripheral area (indicated by a square in the figure), an area that extends in the horizontal direction up to the left and right ends of the lip and extends in the vertical direction is set. As shown in FIG. 3B, the a ^* histogram is divided into a region indicating skin and a region indicating lips. The histogram representing the skin region is relatively low in redness and approximates a triangle having a vertex at μ ₁₁ and having L ₁ and k ₁ as hems. The histogram representing the lip region is relatively reddish and approximates a triangle having a vertex at μ ₂₁ and skirts of k ₁ and R ₁ . In this way, the skin area and the lip area can be well distinguished on the a ^* histogram. Further, from FIG. 3C, the _hab histogram is divided into a region indicating skin (Skin) and a region indicating lips (Lip). Histogram representing the lip region, hue relatively close to red, approximates a triangle having the hem has a vertex L ₂ and k ₂ in mu _12. The histogram representing the skin region approximates a triangle whose hue is relatively yellow, has a vertex at μ ₂₂ and has k ₂ and R ₂ as hems. In this way, the skin area and the lip area can be well distinguished on the _hab histogram. Thereby, the lip area and the skin area are separated and extracted from the color image acquired as an object by referring to the a ^* histogram and the _hab histogram stored in advance in the storage unit 4 for the skin information and the lip information. be able to. And it can be set as the clue which extracts the skin which shows the same histogram as a face area | region and the face area | region continuous in the periphery of a lip | lip area | region.

[Skin Information Acquisition]
If the perceptual chromaticity a * histogram and the hue _hab histogram are used, the lip and the skin can be separated and extracted, and this is used to extract the skin region. The perceptual chromaticity a * histogram and the metric hue angle _hab histogram acquired from the color image for the lip peripheral region set in FIG. 3A can be used. Skin information acquisition is performed by the first to third object region extraction means 31-33.

FIG. 4 shows a histogram that is independent of the color temperature of the skin region (not easily influenced by the color temperature). FIG. 4A shows an a ^* histogram, and FIG. 4B shows a _hab histogram. Each of these is a histogram obtained by separating the lip area from the histograms of FIGS. 3B and 3C and acquiring only the skin area again. The perceptual chromaticity a ^* histogram and the metric hue angle h _ab histogram have distributions corresponding to the skin and lips as described above, and this tendency is maintained even when the color temperature (white balance) changes. It is said to be a temperature independent histogram, and this can be used to robustly extract the skin region. An entire color image or an area including a target skin area from the color image is designated as a search area, and a ^* and _hab of each pixel of the color image are stored in the storage unit 4 for the designated search area. ^{* By} comparing the histogram with the _hab histogram, it can be determined whether each part of the color image is a skin region. The extraction of the skin area using the histogram independent of the color temperature is performed by the first object area extracting means 31, and the extracted skin area becomes the first object area.

FIG. 5 shows a histogram dependent on the color temperature of the skin region (easily affected by the color temperature). FIG. 5A shows a saturation histogram, and FIG. 5B shows a luminance histogram. The saturation histogram and the luminance histogram change depending on the color temperature. Therefore, by using the saturation histogram and luminance histogram corresponding to the color temperature at the time of shooting, it is determined whether each pixel of the color image is a skin region, and an appropriate skin region corresponding to the change in color temperature Can be extracted. The extraction of the skin area using the saturation histogram is performed by the second object area extraction means 32, and the extracted skin area becomes the second object area. The extraction of the skin area using the luminance histogram is performed by the third object area extracting means 33, and the extracted skin area becomes the third object area. The first object area can be corrected using the second object area and the third object area. Note that the saturation histogram and the luminance histogram may be a histogram in the L ^* a ^* b ^* color space or a histogram in the HLS color space.

[Skin area determination (uses an independent feature for color temperature)]
Next, the skin area is determined based on the acquired skin information. Pattern recognition, such as fuzzy inference, can be used for skin area determination. When fuzzy reasoning is used, it is possible to automatically cope with the ambiguity of the color according to the change of the shooting environment, so that the skin region can be extracted efficiently. First, paying attention to the fact that the intensity of the perceptual chromaticity a ^* and the hue h _ab are robust to lighting and have a distribution that corresponds well to the skin, the degree of skininess is calculated by pattern recognition such as fuzzy inference, and the skin region Ask for. For example, for each pixel in the color image, use the histogram endpoints (maximum and minimum) and class average levels (corresponding to two peaks each of lips and skin, but the distribution need not be bimodal) The degree of skininess is calculated, and it is determined whether or not it belongs to the skin region based on the threshold value set by the automatic threshold selection method (setting the threshold value to the degree of attribution indicating the degree of skininess) in the histogram. Skin area determination using a histogram independent of color temperature is performed by the first object area extraction means 31.

FIG. 6 schematically shows fuzzy inference. FIG. 6A schematically shows a fuzzy rule, and FIG. 6B shows a calculation formula for defuzzification. The color image stored in the storage unit 4 is subjected to preprocessing such as smoothing processing using a Gaussian filter, and an a ^* histogram and a _hab histogram are obtained from the preprocessed image. As shown in FIG. 6A, using a fuzzy rule, the a ^* histogram and the _hab histogram are expressed by an antecedent part membership function approximated by a triangular pattern μ (x), and a ^* , Using the antecedent part membership function, μ (x) is obtained from the measured values x0 and x1 of h _ab . Further, using the antecedent part membership function y1 ′ = {μ (x)} ^1/2 and y2 ′ = μ (x), the measured values of a ^* and _hab for each pixel x0, x1 to y1 ′ , Y2 ′. Then, as shown in FIG. 6B, using the obtained y1 ′ and y2 ′, defuzzification is performed by the weighted average method, and the degree of attribution y1 (of the skin class) is obtained by the following equation.

y1 = (w1 × y1 ′ + w2 × y2 ′) / (w1 + w2)

Here, w1 and w2 are weighting factors, and the results when w1 = 1.5 and w2 = 1.0 are good, so these values are used. The degree of attribution y1 is an evaluation value that represents skinness. If the threshold values are y2 and y3 and the condition of y2 <y1 <y3 is satisfied, the pixel is determined to belong to the skin region, and if the condition is not satisfied, it is determined that the pixel does not belong to the skin region. Thereby, the skin area and other areas can be binarized and displayed for the color image. In this example, the weighted average method is used, but a simplified fuzzy inference method or the like may be used instead.

  FIG. 7 shows the distribution of the skin area obtained by fuzzy reasoning. FIG. 7A shows an image represented by the skin class attribution level y1, and FIG. 7B shows a binarized image. The binarized white portion corresponds to the first object area. The lip area can also be obtained by fuzzy inference.

[Skin area determination (uses characteristics dependent on color temperature)]
Next, the color image has a portion that changes depending on the color temperature. That is, the color of the color image changes and the color of the skin part also changes depending on how the sunlight and illumination are applied. That is, since the saturation and luminance are greatly affected by the white balance setting, the saturation histogram and luminance histogram change depending on the color temperature.

  FIG. 8 shows changes in the saturation histogram and the luminance histogram depending on the color temperature (white balance). FIG. 8A shows a change in the color image due to the color temperature, FIG. 8B shows a change in the saturation histogram due to the color temperature, and FIG. 8C shows a change in the luminance histogram due to the color temperature. It can be seen from FIG. 8A that the color image becomes darker as the color temperature increases. FIG. 8B shows that the saturation histogram expands toward the low saturation side as the color temperature increases. Further, FIG. 8C shows that the luminance histogram moves to the low luminance side as the color temperature increases. Therefore, for each pixel in the color image, a skin region is extracted by specifying a range using the maximum and minimum values of the saturation histogram and the luminance histogram at each color temperature. The determination of the skin area using the saturation histogram is performed by the second object area extraction means 32, and the determination of the skin area using the luminance histogram is performed by the third object area extraction means 33. First, the color temperature is obtained from the saturation histogram and luminance histogram of the entire color image, and if it is within the range between the maximum value and the minimum value of the saturation histogram at that color temperature, it is obtained as the second object region, and at that color temperature If it is within the range between the maximum value and the minimum value of the luminance histogram, it is determined as the third object region. Note that the maximum value and the minimum value may be set by taking measurement error into consideration and excluding, for example, several% above and below.

  FIG. 9 shows a skin region extracted using the saturation histogram, and FIG. 10 shows a skin region extracted using the saturation histogram. Both images are binarized images, and the white portion corresponds to the skin region. Desirably, processing such as smoothing processing for excluding elements with a very low frequency in the histogram may be added. In the image extracted as the first object area (see FIG. 7) and the image extracted as the third object area (see FIG. 10), part or all of the background wall portion is extracted as the skin area. However, in the image extracted as the second object region (see FIG. 9), all the background wall portions are removed. Further, in the image extracted as the first object region (see FIG. 7) and the image extracted as the second object region (see FIG. 9), the hair part, the eyebrow part and the eye part are partly, Although extracted as a skin region, in the image extracted as the third object region (see FIG. 10), the hair portion, the eyebrows portion, and the eye portion are all removed.

[Skin area extraction (synthesis)]
FIG. 11 schematically shows a process of extracting the skin region by superimposing the first to third extracted images. That is, the overlapping part of the first object area shown in FIG. 7, the second object area shown in FIG. 9, and the third object area shown in FIG. 10 is determined as the final object area. That is, an AND of the three types of images obtained in the skin area determination is obtained and used as the skin area determination result. This superposition is performed by the object area synthesis means 34. FIG. 11A shows an image of the first to third object areas, FIG. 11B shows an image of the object area extracted by overlapping them, and FIG. 11C shows an object area extracted. An image subjected to coloring processing is shown. According to FIG. 11B, it can be seen that the wall, hair, eyebrows, and eye portions have been removed and corrected to the correct skin area. Further, the coloring process of FIG. 11C is a color image obtained by pre-processing the image acquired by the color image acquisition unit 2 in the range of the object area of FIG. 11B. Desirably, a shape smoothing process may be added.

[Experimental result]
FIG. 12 shows a comparison between an original image, a skin region extraction result example according to the conventional method, and a skin region extraction result example according to the present embodiment. FIG. 12A shows the change in the original image due to the color temperature, FIG. 12B shows the change in the skin region extraction result according to the conventional method due to the color temperature, and FIG. 12C shows the skin region extraction result according to the present embodiment. Shows changes. In the conventional method, skin is extracted when the color temperature is low to medium, but extraction of the skin region fails when the color temperature is high. On the other hand, in this embodiment, it can be seen that skin regions can be extracted at all color temperatures. In the present embodiment, the calculation time required for processing one image is about 30 ms. It has been confirmed that image data taken with a digital camera can be processed in real time. As described above, the object region extraction system or the extraction method in the present embodiment shows that skin region extraction corresponding to white balance change is possible in real time.

[Second Embodiment]
In the first embodiment, perception chromaticity ^{a *} Histogram chromaticity component histogram in ^{the L} * ^a * ^{b *} color space, the hue histogram ^L * ^a * ^{b *} metrics in a color space hue angle _{h ab} is a histogram Although an example has been described, in the second embodiment, an example using another color space will be described. In the first embodiment, an example using fuzzy inference has been described. In the second embodiment, an example in which fuzzy inference is not used will be described.

As a color space other than the L ^* a ^* b ^* color space, for example, L ^* u ^* v ^* that is a uniform color space may be used. Since the L ^* u ^* v ^* color space is a color system in which the uniformity of the wavelength interval is taken into account, if a specific wavelength direction is adopted in the x direction, the intensity of the color relative to the wavelength can be expressed. Further, a uniform color space may be used for the perceptual chromaticity histogram, an HSL color space or an HSV color space may be used for the hue histogram, saturation histogram, and luminance histogram, and a uniform color space may be used for the perceptual chromaticity histogram and hue histogram. HSL color space and HSV color space may be used for the saturation histogram and the luminance histogram. In addition, various combinations are possible as long as the feature amount exists. In this way, even when a color space expressed based on another uniform color space, uniform color space, other hue, saturation, perceptual chromaticity, brightness, or luminance is combined, it is perceived as in the first embodiment. The first object region can be extracted using the chromaticity histogram and the hue histogram, and the second and third object regions can be extracted using the saturation histogram and the luminance histogram. Further, instead of fuzzy inference, the first object region may be obtained as long as it is within the range between the maximum value and the minimum value of the histogram. A target object area can be obtained from the overlapping area of the first to third object areas. When the L ^* u ^* v color space, HSL color space, and HSV color space are used, the robustness is inferior to that when the L ^* a ^* b ^* color space is used, but the practical range is maintained. It is. Also in the present embodiment, a robust object area can be extracted by using the first object area, an object area corresponding to a white balance change can be extracted by using the second and third object areas, and real-time The object area can be extracted with.

[Third Embodiment]
In the first embodiment, an example in which the object is a person's skin has been described. In the third embodiment, another example will be described. For example, the white portion of the panda in the color image and the white portion of the eyeball region are strongly affected by the white balance. Therefore, it is preferable to apply the methods of the first and second embodiments to these objects. Expect results. Using a perceptual chromaticity histogram, a hue histogram, a saturation histogram, and a luminance histogram, an object region that is robust and corresponds to a change in white balance can be extracted in real time.

  The present invention can also be realized as a program for causing a computer to execute the object region extraction method described in the above embodiments. The program may be used by being stored in a built-in memory of the computer, may be stored and used in an external storage device, or may be downloaded from the Internet and used. Moreover, it is realizable also as a recording medium which recorded the said program.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and it is obvious that various modifications can be made to the embodiments.

  For example, when the first to third object regions are obtained using the histogram, the example using fuzzy inference or the example using the maximum value and the minimum value as threshold values has been described. For example, the skin is divided into several small parts. Then, the histogram of each small portion may be compared with the histogram stored in the storage means to determine whether or not it is a skin region. As for the threshold value, it is possible to appropriately determine the upper and lower percentages. In the first embodiment, an example in which a redness histogram is used as the perceptual chromaticity histogram has been described. However, various perceptual chromaticity histograms can be selected according to the object.

  The present invention can be used to extract an object region such as a skin region.

It is a figure which shows the structural example of an object area | region extraction system. It is a figure which shows the example of a processing flow of an object area | region extraction method. It is a diagram illustrating a h _ab histogram as a ^* histogram and hue histogram as perceived chromaticity histogram in lip surrounding area and the area. It is a figure which shows the histogram independent of the color temperature of a skin area | region. It is a figure which shows the histogram dependent on the color temperature of a skin area | region. It is a figure showing fuzzy reasoning roughly. It is a figure which shows distribution of the skin area | region calculated | required by the fuzzy reasoning. It is a figure which shows the change of a saturation histogram and a brightness | luminance histogram by color temperature. It is a figure which shows the skin area | region extracted using the saturation histogram. It is a figure which shows the skin area | region extracted using the saturation histogram. It is a figure which shows typically the process of extracting a skin area | region by superimposing the 1st thru | or 3rd extraction image. It is a figure which compares and shows an original image, the skin region extraction result example by the conventional method, and the skin region extraction result example in 1st Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Object area extraction system 2 Color image acquisition means 3 Object area extraction means 4 Storage means 5 Input / output means 6 Calculation means 7 Control means 31 First object area extraction means 32 Second object area extraction means 33 Third object area Extracting means 34 Object area synthesizing means 35 Parts area extracting means

Claims (8)

Color image acquisition means for acquiring a color image of the object;
First object area extraction means for extracting a first object area from the color image using a chromaticity component histogram and a hue histogram;
Second object region extraction means for extracting a second object region from the color image using a saturation histogram;
Third object area extraction means for extracting a third object area from the color image using a luminance histogram;
An object area synthesizing unit that obtains an overlapping area of the first object area, the second object area, and the third object area as an object area;
Object area extraction system. The object is human skin;
The object region extraction system according to claim 1. The ^{a *} histogram in ^{the L} * ^a * ^{b *} color space as the chromaticity component histogram, using _{h ab} histogram in ^{the L} * ^a * ^{b *} color space as the color histogram;
The object region extraction system according to claim 1 or 2. Using fuzzy inference to extract the first object region;
The object region extraction system according to any one of claims 1 to 3. Storage means for storing the chromaticity component histogram, the hue histogram, the saturation histogram, and the luminance histogram for the object;
The object region extraction system according to any one of claims 1 to 4. The storage means stores the saturation histogram and the luminance histogram in association with a color temperature;
The object region extraction system according to claim 5. A color image acquisition step of acquiring a color image of the object;
A first object region extraction step of extracting a first object region from the color image using a chromaticity component histogram and a hue histogram;
A second object region extraction step of extracting a second object region from the color image using a saturation histogram;
A third object region extracting step of extracting a third object region from the color image using a luminance histogram;
An object area synthesis step of obtaining an overlapping area of the first object area, the second object area, and the third object area as an object area;
Object area extraction method.   A program for causing a computer to execute the object region extraction method according to claim 7.