JP2009111947A - Image correction device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image correction device which automatically performs a process in which desired color reproduction or gradation is provided to image data, especially a face part. <P>SOLUTION: The image correction device comprises a color space conversion unit 1 which converts input video data into image data of many types of color systems for detecting a specific color region, a specific color region detection unit 3 which detects the specific color region based on an output of the color space conversion unit 1, a face part detection unit 5 which performs a detection by judging whether the color region detected by the specific color region detection unit 3 is a human face part, an image filter unit 6 which reduces noise of the inputted video data, an HSV color system conversion unit 7 which converts the output of the image filter unit 6 into HSV color system data, and an image correction processing unit 8 which adjusts hue, saturation, and value to the data that is converted into HSV color system data and performs image processing to the specific color region detected by the specific color region detection unit 3. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image correction apparatus, and more particularly to image processing that provides a better image.

  When image data such as a portrait image recorded with a digital camera or a video camera is confirmed on a TV monitor or printer, the color reproduction is often not as expected. Also, recent improvements in the resolution of digital cameras, etc. have made skin wrinkles, spots, and acne appear more clearly. However, the skin color is preferably a so-called memory color, and the facial wrinkles, stains, acne and the like are preferably blurred.

The heel component is extracted from the image, a processed image from which the heel component has been removed is generated, the heel amount detection means detects the amount of the heel component for each of the nine divided areas, and the region determination means The region having the largest amount is determined as the confirmation region, and the image extraction means extracts the pre-processing confirmation image and the processed confirmation image corresponding to the confirmation region from each of the image and the processed image, and the display control means extracts these images. It has been proposed to display a composite image on a monitor (see Patent Documents 1 and 2). However, to remove wrinkles and spots, an image of the same size as the original image is required, and a large memory is required to perform image processing from the result of this image. Further, since the means for detecting the face uses the conventional face detection, there is a possibility that the consistency does not match.
JP 2006-53859 A JP 2005-65688 A

  An object of the present invention is to provide an image correction apparatus that automatically performs processing for realizing preferable color reproduction and blurring on image data, particularly a face portion.

  The present invention includes means for converting image data including a person into a plurality of color systems, means for accumulating predetermined values for the data converted by the color conversion means, and accumulated data Skin color detection means for detecting a region corresponding to skin from the skin, means for converting the skin color data detected by the skin color detection means to a preferred skin color with reference to the luminance level of the entire image, and the skin color detection means Means for applying an appropriate filtering process to the detected skin color data.

  That is, the present invention provides a color space conversion unit that converts a plurality of types of color systems in order to detect a specific color area for input video data, and a specific color based on the output of the color space conversion unit. A specific color region detection unit for detecting a region, a face detection unit for determining whether a color region detected by the specific color region detection unit is a human face, and for input video data An image filter unit for reducing noise, an HSV color system conversion unit for converting the output of the image filter unit into HSV color system data, a phase for the data converted into the HSV color system data, An image correction apparatus including an image correction processing unit that adjusts saturation and brightness and performs image processing on a specific color region detected by the specific color region detection unit.

  The specific color region detection unit may convert the input video data into a plurality of types of color system data in order to detect the pixel of the specific color in the video data. An image correction apparatus comprising: a specific color detection condition setting unit that sets a condition of pixel information correlated with the specific color with respect to the data converted by the step; and a unit that detects the specific color area based on the setting condition. .

  In the present invention, the face detection unit includes a face detection condition setting unit for setting a condition for detecting a face with respect to output data of the specific color region detection unit, and a face detection condition setting value. The image correction apparatus includes means for detecting pixel information that matches the setting condition in the data of the specific color area detection unit based on the image data.

  Further, according to the present invention, the image filter unit is configured to perform blurring processing on the wrinkles and blurs while leaving the various types of contours of the input video data, and the color region of the output data of the specific color region detection unit It is an image correction apparatus provided with a means to change a filter amount with respect to the total number of.

  Further, according to the present invention, the image correction processing unit adjusts the phase, saturation, and brightness of data obtained by converting the output data from the image filter unit into HSV data by the HSV color system conversion unit, and Based on the specific color area information from the specific color area detection unit, means for masking the input video data other than the specific color area, and the input video data based on the face determination result from the face detection unit On the other hand, means for masking areas other than the face, means for replacing pixels other than the specific color area and face with pixels of the input video data, and specific color areas detected by the specific color area detection unit and the face detection unit And an image correction apparatus comprising means for replacing the pixels of the face part with the pixels of the image filter part.

  According to the present invention, it is possible to distinguish between skin and non-skin color by combining image data converted to each color system, and skin can be easily detected. Further, a combination of these enables detection of a specific color and conversion into a memory color other than the skin color. Further, since data such as the luminance level of the entire image is accumulated, image processing corresponding to the image can be performed, and excessive image processing can be prevented. The data in a specific range with respect to the color system-converted data is the wrinkle, stain, and acne data, and the detected skin color data range is compared with this data according to the detected amount of data such as wrinkles Controls the strength of filtering. In addition, regarding the filter processing, it is possible to control whether or not to perform the filter processing on a specific portion within the skin color range, thereby preventing excessive filter processing and making only the wrinkles and spot portions inconspicuous.

The outline of the present invention is as follows. First, the image data is converted into several kinds of color system data. For example, the RGB data of the image is converted into an HSV color system, an XYZ color system, or an L * u * v * color system. Data within a certain range of these converted data is accumulated for one frame, and a specific color range can be detected by combining each accumulated data. For example, it is known that the skin color falls within the range of 6 to 38 in the HSV color system. However, this alone also detects skin colors that are not human skin. For distinguishing between human skin and other skin colors, the prior art has proposed a method in which a face detecting means is further provided, the positions of the mouth, eyes, eyebrows and the like are specified and image processing is not performed on this portion. However, there are various face orientations, and it is difficult to identify the position and shape of the mouth, eyes, eyebrows and the like as a face, and there is a possibility of misjudgment. In the present invention, the color data of eyes, mouth, and eyebrows are accumulated within the range detected in the skin color range detection, and comprehensive data is compared to determine a face. The color data such as the eyes, mouth, and eyebrows can be discriminated by determining in advance the range in which these data appear and the cumulative number. Filter processing is performed using the peripheral data of the discriminate wrinkle and spot data, and the data range to be used can be several to several tens of pixels + several lines if it is a two-dimensional filter. As a result, it is possible to perform filter processing only on the wrinkles and spots on the skin of the face, arms and body, and image processing close to so-called beautiful skin becomes possible. Further, for example, in the sweater having a skin-colored mesh, data such as eyes, mouth, and eyebrows are not detected from the discrimination process, and therefore, the filtering process is not performed. Therefore, the fine mesh is not lost.
Hereinafter, an embodiment of an image correction apparatus of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image correction apparatus according to an embodiment of the present invention. 1, the image correction apparatus according to the present embodiment includes a color space conversion unit 1, a skin color detection condition setting unit 2, a skin color range detection unit 3, a face detection condition setting unit 4, a face detection unit 5, and an image filter unit 6. HSV color system conversion unit 7 and image processing unit 8.

2 to 6 are diagrams showing the configuration of each function shown in FIG. The overall processing will be described with reference to these drawings.
In FIG. 1, for example, in the case of R (red), G (green), and B (blue), video data input to this system is input to the color space conversion unit 1 for skin color portion detection processing, and various color specifications Converted to system image data. The color space conversion unit 1 is a standard BT. For standard television broadcasting standardized by the International Telecommunications Union (ITU). 601 (hereinafter referred to as SDTV) and the standard BT. 709 (hereinafter HDTV) is converted. The color space conversion unit 1 converts the input RGB data into HSV color system image data represented by hue, saturation, and brightness. H is hue, S is saturation, and V is lightness. In the HSV color system, since hue “H” and brightness “V” are independent, it is possible to examine the hue without worrying about the brightness of the image. Further, the color space conversion unit 1 also converts to the L * a * b * color system. The L * a * b * color system represents a color using three attributes L *, a *, and b *. Since the L * a * b * color system is a uniform color space, the distance in the space becomes the color difference as it is. Further, the color space conversion unit 1 performs conversion to the YCbCr color system. Y represents luminance, Cb represents color difference (blue), and Cr also represents color difference (red) information. These converted image data are transferred to the skin color detection condition setting unit 2.

  The skin color detection condition setting unit 2 refers to the values of the input HSV color system image data 201, L * a * b * color system image data 202, YCbCr color system image data 203, and RGB pixel data, A value for detecting the pixel data range of the skin color portion is set for each color system data.

  The skin color range detection unit 3 refers to the value set by the skin color detection condition setting unit 2 and detects the skin color range for each color system data. The final skin color range is determined by combining these detection conditions.

  The face detection condition setting unit 4 sets conditions for detecting whether the range determined as the skin color portion by the skin color range detection unit 3 is a face or a face. A condition for determining a face is that the skin color portion includes pixel information of eyes, eyebrows, and mouth. Pixels that satisfy the conditions are detected from each color system data.

  The pixel filter unit 6 acts as a blur filter on the image in order to obscure wrinkles, spots, freckles, and the like. Various types of filters have been reported, but a method for removing noise (such as wrinkles, spots, freckles, etc.) without reducing the resolution is used. As a result, the outline (edge) of the image does not blur so much that wrinkles, spots, freckles, etc. are not noticeable.

  The HSV color system conversion unit 7 converts the image filtered by the pixel filter unit 6 into HSV color system data in order to adjust the phase, saturation, and lightness to correct the image.

  The image processing unit 8 independently adjusts H, S, and V of the HSV color system image data filtered by the HSV color system conversion unit 7 to correct the color of the image. The skin or face range information 205 and 207 output from the skin color part detection unit 3 or the face part detection unit 5 is referred to the RGB data 200, and the corrected HSV data 209 is converted to RGB data for the skin or face part. Replace with the data returned to.

  Next, each function will be described in detail. FIG. 2 is a block diagram of the color space conversion unit 1, the skin color detection condition setting unit 2, and the skin color range detection unit 3. The color space conversion unit 1 converts the input RGB data 200 into the HSV color system at the HSV conversion unit 10. Further, since direct conversion is not possible in order to convert to the L * a * b * color system, the RGB data 200 is once converted into XYZ color system data by the XYZ conversion unit 11, and the XYZ color system data is converted to L * a. * B * conversion unit 13 converts to L * a * b * color system data. The YCbCr conversion unit 12 converts the RGB data 200 into YCbCr color system data.

  The skin color detection condition setting unit 2 designates a skin color range for each data converted by the color space conversion unit 1 and detects pixels that meet these conditions. The first skin color detection condition setting unit 20 is a skin color pixel setting for the HSV color system. For example, it is known that the skin color of each race roughly falls within an H (phase) range of 6 ° to 38 ° in FIG. Actually, in addition to this range, 0 ° to 6 ° is a skin color range. Moreover, since about 340 ° to 360 ° may be a skin color, 345 ° to 360 ° (0 °) to 38 ° is set. This set value can be varied according to the situation, and is not limited to this value. The skin color detection second condition setting unit 21 is a skin color pixel setting for the L * a * b * color system. For example, when the value of b * is about −10 to 30 as an example, it is often a skin color. This set value can be varied according to the situation, and is not limited to this value. The skin color detection third condition setting unit 22 is a skin color pixel setting for the YCbCr color system. For example, as an example, the range where Cb is about -10 to -30 and Cr is about 10 to 35 is often a skin color. This set value is variable depending on the situation, and is not limited to this value. The skin color detection fourth condition setting unit 23 is a skin color pixel setting for RGB. For example, when the value of R is 20 or more larger than the larger of the G value or the B value, and the absolute value difference between the G value and the B value is 60 or less, the skin color is often obtained. This set value is variable depending on the situation, and is not limited to this value. The skin color range detection unit 3 detects the range of skin color pixels by combining these conditions. The detected range is provided to the face detection condition setting unit 4 as skin color portion pixel information.

  FIG. 3 is an explanatory diagram of the face detection condition setting unit 4. Since the skin color range pixel information is output as information, the skin color portion pixel information can be determined to be a face by determining how much pixel information of eyebrows, eyes, and mouth is included in this range. Therefore, as with the skin color detection condition, the eye detection condition setting unit 40 sets information corresponding to the eyes from each color system data. For example, it is a setting unit for detecting a range in which pixel information close to black is present while skin color pixels continue, or the number of pixels. For example, the mouth detection condition setting unit 41 can often determine “mouth” when the RGB data is close to the G value and the B value and the R value is larger than the G and B values within the skin color pixel range, as described above. This is a setting unit for detecting a range or the number of pixels. This set value is variable depending on the situation, and is not limited to this value. Similarly, the eyebrow detection condition setting unit 42 is a setting unit for detecting, for example, a range in which pixel information close to the eyebrow exists or the number of pixels while skin-colored pixels continue. The face detection condition setting unit 43 combines the above settings and outputs them as face determination setting values.

  FIG. 4 is an explanatory diagram of the face detection unit 5. In the flesh color pixel histogram 50, for example, as shown in FIG. The eye / mouth / eyebrow histogram 51 accumulates pixel information of eyes, mouths, and eyebrows based on face determination setting values for each color system image data. This operation is performed for one vertical synchronization period, and these results are provided to the face detection unit 52. The face detection unit 52 determines whether the skin color detected with reference to each histogram result is a face. FIG. 11 shows an example of a histogram. 9 is an example in which the total number of skin color pixels and the total number of eyebrow, eye, and mouth pixels are sensuously shown in the skin color range shown in FIG. 8. When the total number of pieces of pixel information of eyebrows, eyes, and mouth is equal to or greater than the specified percentage with respect to the total number of pieces of skin color pixel information, the face is determined. The threshold setting for determination is variable depending on the situation, and is not limited to this value.

  FIG. 5 is an explanatory diagram of an image filter unit for realizing so-called beautiful skin. As an example, an N × N median filter can be considered. This filter is a filter in which the blur of the edge of the image is small. As the value of N increases, the amount of blur increases, but noise or the like becomes inconspicuous. 5 (b) is the 3 × 3 filter 1 with the least blur, FIG. 5 (c) is the 5 × 5 filter 2 with the medium blur, and FIG. 5 (d) is 7 × 7. Filter 3 is the most blurred. If the configuration is such that optimum blur is applied by changing the type of filter depending on the size of the skin color area, for example, if the skin color area is large (the entire screen is a face up, etc.), the blur is increased (7 × 7 filter 3). Operate to make glares and stains less noticeable. On the contrary, when the skin color area is small (an image that looks like the whole person), the blur is reduced (3 × 3 filter 1) and the edges are left. The filter may be selected not by the size of the skin color portion but by expressing wrinkles, spots, freckles as spatial frequencies, and selecting the size according to the size.

  FIG. 6 is an explanatory block diagram of the image processing unit 8 that performs correction of the flesh color portion and preferable flesh color processing. The mask processing unit 80 uses the skin color part pixel information 205 detected by the skin color range detection unit 3 and the face determination result 207 information detected by the face part detection unit 5 to determine the skin color part or face part of the RGB data 200 of the video input. Mask processing is performed on the pixels. The HSV adjustment unit 81 adjusts the H (phase), S (saturation), and V (brightness) of the HSV color system data HSV 209 that has been subjected to the filter processing to adjust the skin color to a preferable skin color. Regarding the adjustment of HSV, in the model of the HSV display system shown in FIG. 10, H (phase) starts from red (0 °), yellow (60 °), green (120 °), light blue (180 °), blue (240 °), purple (300 °), and red again. S (saturation) can be simply expressed as a numerical value. If the minimum value is “0” and the maximum value is “255”, respectively, the saturation S is 0, colorless is 255, and the brightness V is 0. The darkest and 255 can be defined as the brightest. V (brightness) can also be simply expressed as a numerical value. In the brightest state (brightness is 255), about half bright (brightness 128), and completely dark (brightness is 0), the entire circle is black. In the HSV color system, since hue “H” and brightness “V” are independent, it is possible to adjust the hue without worrying about the brightness of the image. After adjusting to a preferable skin color, the RGB conversion unit 82 performs conversion from the HSV color system to RGB. The image composition unit 83 performs processing for replacing the input video data subjected to skin beautification and skin color correction to the mask portion of the input video data subjected to mask processing. As a result, it is possible to make the skin color a preferable color while maintaining the original image except the skin color portion, leaving the outline of the skin color portion, and making the wrinkles and blotches inconspicuous. In addition, since it is possible to determine the face, a scene similar to skin color, for example, a sunset scene, is not determined to be a face, and the filter processing and color adjustment as described above are not performed, so the quality of the original image is maintained. It will be drunk.

Although the present invention has been described with respect to the skin color, the present invention is not limited to the skin color, and for example, forests such as green, blue, and red, the sky, the sunset, and the like can be adjusted to so-called memory colors.
In addition, the present invention is not limited to the above-described embodiment, and various modifications and corrections can be made without departing from the scope of the present invention.

It is a block diagram which shows the structure of the image correction apparatus which concerns on embodiment of this invention. It is a detailed block diagram of the skin color detection condition setting part of this invention. It is a detailed block diagram of the face part detection condition setting part of this invention. It is a detailed block diagram of the face part detection part of this invention. It is a detailed block diagram of the image filter part of this invention. It is a detailed block diagram of an image processing unit of the present invention. It is the figure which showed the pixel information of the face. It is the figure which showed the skin color range of the face. It is the figure which showed the example which detects the pixel information of eyes. It is the figure which showed the model of the HSV color system. It is the figure which showed the histogram of each pixel information within a skin color range.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Color space conversion part 2 ... Skin color detection condition setting part 3 ... Skin color range detection part 4 ... Face detection condition setting part 5 ... Face detection part 6 ... Image filter part 7 ... HSV color system conversion part 8 ... Image processing part 10 ... HSV converter 11 ... XYZ converter 12 ... YCbCr converter 13 ... L * a * b * converter 20 ... skin color detection first condition setting part 21 ... skin color detection second condition setting part 22 ... skin color detection third condition setting Section 23 ... Skin color detection fourth condition setting section 30 ... Each condition setting section 31 ... Detection section 40 ... Eye detection condition setting section 41 ... Mouth detection condition setting section 42 ... Eyebrow detection condition setting section 43 ... Face detection condition setting section 50 ... Skin color pixel histogram section 51... Eye / mouth / brow histogram section 52. Face detection section 60.
61 ... Filter 2
62 ... Filter 3
63 ... selector 80 ... mask processing unit 81 ... HSV adjustment unit 82 ... RGB conversion unit 83 ... image composition unit 200 ... input video data (RGB)
201 ... HSV conversion data 202 ... L * a * b * conversion data 203 ... YCbCr conversion data 204 ... skin color detection setting data 205 ... skin color range information data 206 ... face selection setting data 207 ... face determination result data 208 ... filtered Input video data 209 ... Filtered input video data converted into HSV 210 ... Skin color correction data

Claims (5)

  A color space conversion unit that converts a plurality of types of color systems in order to detect a specific color region of input video data, and a specific color that detects a specific color region based on the output of the color space conversion unit An area detection unit, a face detection unit for detecting whether the color area detected by the specific color region detection unit is a human face, and an image filter for reducing noise in input video data Unit, an HSV color system conversion unit that converts the output of the image filter unit into HSV color system data, and adjustment of phase, saturation, and brightness with respect to the data converted into the HSV color system data And an image correction processing unit that performs image processing on the specific color area detected by the specific color area detection unit.   The specific color area detection unit converts the input video data into data converted by the color space conversion unit that converts the input video data into a plurality of types of color system data in order to detect pixels of the specific color in the video data. The image correction apparatus according to claim 1, further comprising: a specific color detection condition setting unit that sets a condition of pixel information correlated with the specific color; and a unit that detects the specific color area based on the setting condition.   The face detection unit includes a face detection condition setting unit that sets a condition for detecting a face with respect to output data of the specific color region detection unit, and the specific color based on a face detection condition setting value. The image correction apparatus according to claim 1, further comprising means for detecting pixel information that matches the set condition in data of an area detection unit.   The image filter unit filters the input video data with a blurring process for wrinkles and stains while leaving each type of contour, and a filter for the total number of color regions of the output data of the specific color region detection unit The image correction apparatus according to claim 1, further comprising means for changing the amount.   The image correction processing unit includes means for adjusting phase, saturation, and brightness of data obtained by converting output data from the image filter unit into HSV data by an HSV color system conversion unit, and the specific color region detection unit. Means for masking the input video data other than the specific color region based on the specific color region information from the input image data, and other than the face portion for the input video data based on the face determination result from the face detection unit Means for masking, means for replacing pixels other than the specific color area and face with pixels of the input video data, and the specific color area and face pixels detected by the specific color area detection unit and the face detection unit. The image correction apparatus according to claim 1, further comprising means for replacing the pixels of the image filter unit.

Images