JP4111248B2 - Color balance adjustment method and apparatus considering color reproduction of specific color - Google Patents

Description

  The present invention relates to an image color balance adjustment technique.

  For an image captured by an image generation device such as a digital still camera or a video camera, a process called color balance adjustment is performed on the entire image (for example, Patent Document 1 below).

JP 2001-320727 A

  When the color balance adjustment is performed, the color fog generated in the image is reduced, and the image pickup target that was originally white can be accurately reproduced in white.

  However, in some cases, as a result of color balance adjustment, an undesirable color change may occur in a specific imaging target. For example, when color balance adjustment is performed when a pixel close to white is bluish, the blue component is suppressed and the red component is enhanced with respect to the entire image. At this time, if the skin color has a desired color in the image before the color balance adjustment, the skin color is further reddish, and there is a problem that it deviates from the desired color. As described above, the problem that the color balance adjustment causes the color to deviate from the desired color is not limited to the skin color, but may also occur for other chromatic colors such as green and blue. Generally, this problem is common to specific chromatic colors. Met.

  An object of the present invention is to provide a technique that can alleviate a phenomenon that a color near a specific chromatic color in an image is deviated from a desired color by color balance adjustment processing.

  An image processing apparatus according to an aspect of the present invention is an image processing apparatus for adjusting a color balance of a processing target image, and is located near a specific chromatic color among all the pixels present in the processing target image. A specific color pixel ratio calculation unit that calculates a specific color pixel ratio that is a ratio of specific color pixels having a predetermined first range of colors; and a specific color pixel average value that calculates an average value of pixel values of the specific color pixels A calculation unit; a color balance correction value calculation unit having a first calculation mode for calculating a color balance correction value based on the specific color pixel ratio and an average pixel value of the specific color pixel; and the color balance correction value And a color balance adjustment execution unit that executes color balance adjustment on the entire processing target image.

  According to this image processing apparatus, since the color balance correction value is calculated based on the specific color pixel ratio and the average pixel value of the specific color pixel, the color of the pixel having a color close to the specific chromatic color is the color balance. The phenomenon of deviating from a desired color by the adjustment process can be alleviated.

  The color balance correction value calculated in the first calculation mode increases as the difference between the specific color target pixel value predetermined for the specific color and the average pixel value of the specific color pixel increases. In addition, it is preferable that a component that increases as the ratio of the specific color pixel increases is included.

  According to this configuration, the color of a pixel having a color close to a specific chromatic color can be maintained at a more appropriate color.

  The image processing apparatus further includes an average value of pixel values related to a plurality of high-luminance pixels having a predetermined second range of colors with low saturation and high luminance among all the pixels present in the processing target image. A high-brightness pixel average pixel value calculation unit that calculates the color balance correction value calculation unit (i) when the average value of hue values of the plurality of high-brightness pixels is not within a predetermined range close to the specific color (Ii) When the average value of hue values of the plurality of high-luminance pixels is in a predetermined range close to the specific color, the specific color pixel ratio is calculated. The calculation may be executed in a second calculation mode in which a color balance correction value is calculated using an average pixel value related to the plurality of high luminance pixels without using an average pixel value of the specific color pixel. .

  According to this configuration, the color balance correction value can be set to an appropriate value depending on whether or not the average value of the hue values of the high luminance pixels is in a predetermined range close to the specific color. Specifically, in the first calculation mode, a pixel having a color close to a specific chromatic color can be maintained as an appropriate color, and in the second calculation mode, a preferable color balance is realized for the entire processing target image. can do.

  Alternatively, in the image processing device, the specific chromatic color is a skin color, and the image data file representing the processing target image includes shooting scene type information indicating a shooting scene type of the processing target image, The color balance adjustment execution unit executes (i) calculation in the first calculation mode when the shooting scene type information represents a person, and (ii) the shooting scene type information represents a scene other than a person. In this case, in the second calculation mode in which the color balance correction value is calculated using the average pixel value regarding the plurality of high luminance pixels without using the specific color pixel ratio and the average pixel value of the specific color pixel. You may make it perform a calculation.

  According to this configuration, when the shooting scene type represents a person, it is possible to maintain a pixel having a color close to the skin color in an appropriate color. On the other hand, when the shooting scene type information represents a scene other than a person, a preferable color balance can be realized for the entire processing target image.

  An image processing apparatus according to another aspect of the present invention is an image processing apparatus for adjusting a color balance of a processing target image, and is in the vicinity of a specific chromatic color among all the pixels present in the processing target image. A specific color pixel average value calculating unit for calculating an average value of pixel values of specific color pixels having a color in a predetermined first range, and calculating a color balance correction value based on the average pixel value of the specific color pixels A color balance correction value calculation unit having a first calculation mode, and a color balance adjustment execution unit that performs color balance adjustment on the entire processing target image using the color balance correction value.

  According to this image processing apparatus, since the color balance correction value is calculated based on the average pixel value of the specific color pixel, the color of the pixel having a color close to the specific chromatic color is desired by the color balance adjustment process. It is possible to alleviate the phenomenon of falling out of the range.

  The color balance correction value calculated in the first calculation mode increases as the difference between the specific color target pixel value predetermined for the specific color and the average pixel value of the specific color pixel increases. It is preferable to contain the component which becomes.

  According to this configuration, the color of a pixel having a color close to a specific chromatic color can be maintained at a more appropriate color.

  The image processing apparatus further includes an average value of pixel values related to a plurality of high-luminance pixels having a predetermined second range of colors with low saturation and high luminance among all the pixels present in the processing target image. A high-brightness pixel average pixel value calculation unit that calculates the color balance correction value calculation unit (i) when the average value of hue values of the plurality of high-brightness pixels is not within a predetermined range close to the specific color (Ii) When an average value of hue values of the plurality of high-luminance pixels is within a predetermined range close to the specific color, the calculation is performed in the first calculation mode. The calculation may be executed in the second calculation mode in which the color balance correction value is calculated using the average pixel value regarding the plurality of high-luminance pixels without using the average pixel value.

  According to this configuration, the color balance correction value can be set to an appropriate value depending on whether or not the average value of the hue values of the high luminance pixels is in a predetermined range close to the specific color. Specifically, in the first calculation mode, a pixel having a color close to a specific chromatic color can be maintained as an appropriate color, and in the second calculation mode, a preferable color balance is realized for the entire processing target image. can do.

  Alternatively, in the image processing device, the specific chromatic color is a skin color, and the image data file representing the processing target image includes shooting scene type information indicating a shooting scene type of the processing target image, The color balance adjustment execution unit executes (i) calculation in the first calculation mode when the shooting scene type information represents a person, and (ii) the shooting scene type information represents a scene other than a person. In such a case, the calculation is executed in the second calculation mode in which the color balance correction value is calculated using the average pixel value regarding the plurality of high luminance pixels without using the average pixel value of the specific color pixel. May be.

  According to this configuration, when the shooting scene type represents a person, it is possible to maintain a pixel having a color close to the skin color in an appropriate color. On the other hand, when the shooting scene type information represents a scene other than a person, a preferable color balance can be realized for the entire processing target image.

  The present invention can be realized in various forms, for example, an image processing method and an image processing apparatus, a computer program for realizing the function of the method or apparatus, and a recording in which the computer program is recorded. It can be realized in the form of a medium or the like.

Next, embodiments of the present invention will be described in the following order.
A. First embodiment:
B. Second embodiment:
C. Modified example

A. First embodiment:
FIG. 1 is an explanatory diagram showing the configuration of an image processing system as an embodiment of the present invention. This system includes a digital camera 100 as an image generation apparatus, a computer 200 as an image processing apparatus, and a printer 300 as an image output apparatus. The image data file MF generated by the digital camera 100 is transferred to the computer 200 or the printer 300 and output. In this specification, “output” includes display on a monitor and printing.

  FIG. 2 is an explanatory diagram showing the configuration of the image data file MF. The image data file MF includes shooting information, main image data MM, and thumbnail image data TM. The main image data MM and the thumbnail image data TM usually have a format of compressed data (for example, JPEG compressed data). The thumbnail image data TM may not be included in the image data file MF.

  The shooting information includes various conditions and setting values when the digital camera 100 generates the image data file MF. In FIG. 2, “shooting scene type” is exemplified as shooting information. As the shooting scene type, “person”, “landscape”, “night view”, and the like can be set.

  A printer driver 210 for generating print data based on the image data file MF is installed in the computer 200 (FIG. 1). The printer driver 210 includes a color balance adjustment unit 212, a color conversion unit 214, a halftone processing unit 216, and an output processing unit 218. The print data PD generated by the printer driver 210 is supplied from the output processing unit 218 to the printer 300. The print data PD includes dot data that specifies the ink dot recording state for each pixel on the main scan line having the print resolution, and sub-scan feed amount data that specifies the sub-scan feed amount. The printer driver 210 corresponds to a computer program for realizing a function of generating dot data for printing. The color balance adjustment unit 212 corresponds to a computer program for realizing a function of executing color balance adjustment.

  A program for realizing the function of the printer driver 210 may be supplied in a form recorded on a computer-readable recording medium. As such a recording medium, various computer-readable media such as a flexible disk, a CD-ROM, a magneto-optical disk, an IC card, a ROM cartridge, a punch card, and a printed matter on which a code such as a barcode is printed are used. it can.

  FIG. 3 is a block diagram illustrating a configuration of the color balance adjustment unit 212. FIG. 4 is an explanatory diagram showing an example of the processing target image TMG and various color balance adjustment parameters. The color balance adjustment unit 212 includes the following units.

(1) High-brightness pixel average pixel value calculation unit 220:
The calculation unit 220 calculates the following average pixel values related to high luminance pixels based on the processing target image data Dorg (main image data MM in FIG. 2).
a. Color component average values CB_ave (R), CB_ave (R), CB_ave (B);
b. Average luminance value CB_ave (Y);
c. Average hue value CB_ave (H)

  The “high luminance pixel” means a pixel having a color included in a predetermined color range (high luminance range) having a low saturation value and a high luminance value. FIG. 5 shows an example of the high luminance range HLR. Various formulas can be used for calculating the luminance value Y and the saturation value S of each pixel. For example, the following formulas can be used.

Y = 0.587 x G + 0.114 x B + 0.299 x R (1)
S = √ (a * ² + b * ² )… (2)

  Here, a * and b * are coordinate values of the La * b * color system. As the luminance value Y, the lightness value L of the La * b * color system can be used, and the luminance value or lightness value of another color space can also be used. As for the saturation value S, it is possible to use saturation values of other color spaces such as HSL (Hue / Saturation / luminance) color space and HSB (Hue / Saturation / Brightness) color space.

  The color component average values CB_ave (R), (G), and (B) are average values for the respective RGB color components. The average luminance value CB_ave (Y) is an average value of the luminance values Y, and the average hue value CB_ave (H) is an average value of the hue values H. As the hue value H, for example, the angle from the a * axis of the La * b * color system, or the hue value H of the HSL color space or HSB color space can be used.

(2) Skin color pixel ratio calculation unit 222:
The calculation unit 222 calculates the skin color pixel ratio SPratio based on the processing target image data Dorg. The skin color pixel ratio SPratio is given by the following equation (3).

  SPratio = [number of skin color pixels] / [total number of pixels] (3)

  Whether or not a certain pixel is a skin color pixel is determined based on whether or not the color of the pixel is within a predetermined skin color range. The skin color range can be set based on the range of the hue value H and the range of the lightness L, for example.

(3) Skin color pixel average pixel value calculation unit 224:
The calculation unit 224 calculates average pixel values SC_ave (R), (G), and (B) of a plurality of skin color pixels related to the respective RGB color components based on the processing target image data Dorg.

(4) Color balance correction value calculation unit 226:
The calculation unit 226 calculates color balance correction values ΔC (R), (G), and (B) for the respective RGB color components based on the various values described above. In this embodiment, the color balance correction values ΔC (R), (G), and (B) are calculated according to the following equations (4a) to (4c).

ΔC (R) = k1 × {CB_ave (Y)-CB_ave (R)}
+ k2 × {SC_target (R) −SC_ave (R)} × SPratio (4a)
ΔC (G) = k1 × {CB_ave (Y)-CB_ave (G)}
+ k2 × {SC_target (G) −SC_ave (G)} × SPratio (4b)
ΔC (B) = k1 × {CB_ave (Y)-CB_ave (B)}
+ k2 × {SC_target (B) −SC_ave (B)} × SPratio (4c)

  Here, k1 and k2 are predetermined coefficients. SC_target (R), (G), and (B) are skin color target pixel values related to the respective color components, and are predetermined values. As will be described below, the values of the coefficients k1 and k2 are set according to the processing target image, for example. The setting method of the coefficients k1 and k2 and the meaning of the color balance correction values ΔC (R), (G), and (B) will be described later.

  FIG. 6 is a flowchart illustrating a processing procedure of color balance adjustment in the first embodiment. In step S1, the above-described various color balance adjustment parameters CB_ave, SPratio, and SC_ave (excluding correction values ΔC (R), (G), and (B)) are calculated. In step S2, the color balance correction value calculation unit 226 determines whether or not the average hue value CB_ave (H) of the high luminance pixels is included in a predetermined skin color gamut (skin color hue range). When the average hue value CB_ave (H) is not included in the skin color gamut, the color balance correction value calculation unit 226 uses the skin color target values SC_target (R), (G), and (B) to correct the color balance correction value. The calculation is executed in the first calculation mode for calculating (step S3). Specifically, the two coefficients k1 and k2 in the above equations (4a) to (4c) are set to 1.0, for example. At this time, the above equations (4a) to (4c) are rewritten into the following equations (5a) to (5c).

ΔC (R) = {CB_ave (Y)-CB_ave (R)}
+ {SC_target (R)-SC_ave (R)} x SPratio (5a)
ΔC (G) = {CB_ave (Y)-CB_ave (G)}
+ {SC_target (G)-SC_ave (G)} × SPratio (5b)
ΔC (B) = {CB_ave (Y)-CB_ave (B)}
+ {SC_target (B)-SC_ave (B)} x SPratio (5c)

  The first term on the right side of the equations (5a) to (5c) includes the average luminance value CB_ave (Y) of the high luminance pixel and the average pixel value CB_ave (R), (G), (B) of each color component of the high luminance pixel. And the difference. This has the effect of bringing the color of the entire image closer to the achromatic color when the color of the high luminance pixel is shifted from the achromatic color. The second term on the right side of the equations (5a) to (5c) is the skin color target pixel values SC_target (R), (G), (B) and the skin pixel average pixel values SC_ave (R), (G), (B ) Multiplied by the skin color pixel ratio SPratio. This second term has the effect of bringing the target color value closer to the skin color target value when the average color of the pixels having the skin color vicinity deviates from the skin color target value. Therefore, if the color balance adjustment is performed on the entire image to be processed using the correction values ΔC (R), (G), and (B) that can be given by the equations (5a) to (5c), the skin color pixel is set to the skin color target. The color balance of the entire processing target can be realized while approaching the color.

  On the other hand, when it is determined in step S2 that the average hue value CB_ave (H) is included in the skin color gamut, the color balance correction value calculation unit 226 determines the skin color target values SC_target (R), (G), The calculation is executed in the second calculation mode in which the color balance correction value that does not use (B) is calculated (step S4). Specifically, the first coefficient k1 of the above equations (4a) to (4c) is set to a predetermined positive non-zero number, and the second coefficient k2 is set to zero. For example, when the coefficient k1 is set to 1.0 and the coefficient k2 is set to zero, the above equations (4a) to (4c) are rewritten as the following equations (6a) to (6c).

ΔC (R) = {CB_ave (Y) −CB_ave (R)} (6a)
ΔC (G) = {CB_ave (Y) −CB_ave (G)} (6b)
ΔC (B) = {CB_ave (Y) −CB_ave (B)} (6c)

  As described above, the term on the right side has an effect of bringing the color of the entire image closer to the achromatic color when the color of the high-luminance pixel is shifted from the achromatic color. Color balance can be realized. When it is determined that the average hue value CB_ave (H) of the high luminance pixel is included in the skin color gamut, the coefficient k2 of the second term on the right side of the equations (4a) to (4c) is set to zero. This is to prevent the correction value from becoming an excessively large value due to the second term on the right side.

  However, regardless of whether or not the average hue value CB_ave (H) of the high luminance pixel is included in the skin color gamut, the color balance correction value ΔC according to the equations (4a) to (4c) or (5a) to (5c). (R), (G), and (B) may be calculated. At this time, the coefficient k1 of the first term on the right side may be set to zero.

  When the color balance correction values ΔC (R), (G), and (B) are calculated, the color balance adjustment execution unit 228 executes the color balance adjustment in step S5 of FIG. 6, and the adjusted image data Dcr (FIG. 3) is generated. FIG. 7 is a graph showing the contents of color balance adjustment executed using the correction values ΔC (R), (G), and (B). The color balance adjustment is a process for executing gradation correction (tone curve correction) for each of the RGB color components of the processing target image data Dorg. The broken line indicates the non-conversion characteristic. The correction values ΔC (R), (G), and (B) calculated by the above-described procedure correspond to a shift amount for shifting the output value at the input reference value Dref from the non-conversion characteristics. As the input reference value Dref, for example, the above-described average luminance value CB_ave (Y) of the high luminance pixels can be used, or a predetermined constant value (for example, 192) can be used. In the example of FIG. 7, the tone curve shape of each color component is specified by specifying correction values ΔC (R), ΔC (G), ΔC (B) for the input reference value Dref one by one for each color component of RGB. Is determined. However, other setting methods may be adopted.

  As described above, according to the first embodiment, since the color balance correction value of the processing target image is determined using the skin color target value, the color of the pixel that was originally the desired skin color is excessively changed by the color balance adjustment. Can be mitigated.

B. Second embodiment:
FIG. 8 is a flowchart illustrating a processing procedure of color balance adjustment in the second embodiment. The only difference from the processing procedure of the second embodiment shown in FIG. 6 is that step S2 is replaced by two steps S11 and S12, and other configurations and operations are the same as those of the first embodiment. .

  In step S11, the color balance correction value calculation unit 226 acquires the shooting scene type from the image data file MF (FIG. 2). In step S12, it is determined whether or not the shooting scene type represents “person”. If it is “person”, a color balance correction value using the skin color target value is calculated in step S3. On the other hand, if the shooting scene type is not “person”, the color balance correction value is calculated in step S4 without using the skin color target value.

  The shooting scene type is set by the operator when shooting with the digital camera 100. When the shooting scene type is “person”, it is considered that the skin color is regarded as important. Therefore, it is preferable to obtain the color balance adjustment value using the skin color target value in step S3. Note that whether or not the shooting scene is of a person may be determined from information other than the shooting scene type. That is, it may be determined whether or not the shooting scene is “person” from some information indicating the type of shooting scene of the image (this is called “shooting scene type information”).

C. Variation:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

C1. Modification 1:
Instead of the various formulas described above, various forms of calculating the color balance correction value based on the specific color pixel ratio and the average pixel value of the specific color pixel can be employed. For example, the second term on the right side of the equations (4a) to (4c) represents the skin color target pixel values SC_target (R), (G), (B) and the skin color pixel average pixel values SC_ave (R), (G), Although the difference from (B) was multiplied by the skin color pixel ratio SPratio, a format including a squared value or a square root of these values may be used instead. However, the color balance correction value is the skin color target pixel value SC_target (R), (G), (B) and the skin pixel average pixel value SC_ave, as in the second term on the right side of equations (4a) to (4c). It is preferable to have a component that increases as the difference from (R), (G), and (B) increases, and increases as the skin color pixel ratio SPratio increases.
In each of the above embodiments, the specific color pixel ratio is used when obtaining the color balance correction value. However, the color balance correction value may be obtained without using the specific color pixel ratio. For example, the color balance correction value may be obtained based on the average pixel value of the specific color pixel. Further, the color balance correction value may be obtained based on the average pixel value of the specific color pixel and the average pixel value of the high luminance pixel.

C2. Modification 2:
In each of the above-described embodiments, only the color balance adjustment is performed as the image quality adjustment processing. However, in addition to this, other types of image quality adjustment processing may be executed. For example, memory color processing or sharpness enhancement processing may be performed. Note that the memory color process is a process that targets only pixels having colors in the vicinity of the memory color (specific chromatic color) and brings the colors of those pixels close to the target color.

Industrial applicability:
The present invention can be applied to a computer, a camera, an image display device, and the like that perform image color balance adjustment.

It is explanatory drawing which shows the structure of the image processing system as one Example of this invention. It is explanatory drawing which shows the structure of the image data file MF. 3 is a block diagram illustrating a configuration of a color balance adjustment unit 212. FIG. It is explanatory drawing which shows the process target image TMG and various color balance adjustment parameters. It is explanatory drawing which shows an example of the high-intensity range HLR. It is a flowchart which shows the process sequence of the color balance adjustment in 1st Example. It is a graph which shows the content of the color balance adjustment performed using color balance correction value (DELTA) C (R), (G), (B). It is a flowchart which shows the process sequence of the color balance adjustment in 2nd Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Digital camera 200 ... Computer 210 ... Printer driver 212 ... Color balance adjustment part 214 ... Color conversion part 216 ... Halftone processing part 218 ... Output processing part 220 ... High-intensity pixel average pixel value calculation part 222 ... Skin color pixel ratio calculation part 224 ... Skin color pixel average pixel value calculation unit 226 ... Color balance correction value calculation unit 228 ... Color balance adjustment execution unit 300 ... Printer

Claims (5)

An image processing apparatus for adjusting the color balance of an image,
Among all the pixels of the image, a skin color pixel average value calculation unit that calculates an average value of pixel values of skin color pixels;
A high-brightness pixel average pixel calculation unit that calculates an average value of pixel values of a plurality of high-brightness pixels having a predetermined range of colors with low saturation and high luminance among all the pixels of the image;
A color balance correction value calculation unit for calculating a color balance correction values used for the color balance adjustment,
And color balance adjustment execution section for executing the color balance adjustment for the image using a color balance correction value the calculated,
With
The data file representing the image includes shooting scene information representing a shooting scene of the image,
The color balance adjustment execution unit
(I) When the shooting scene information represents a person, a color balance correction value calculated based on a predetermined skin color target pixel value, an average value of the skin color pixels, and an average value of the high luminance pixel values To perform the color balance adjustment using
(Ii) When the shooting scene information represents a scene other than a person, the average pixel value of the plurality of high-luminance pixels without using the predetermined skin color target pixel value and the average value of the skin color pixels The color balance adjustment is performed using the color balance correction value calculated based on
Image processing device. The image processing apparatus according to claim 1,
Color balance correction value is an image processing apparatus has a size when it represents the advance the photographic scene information and the difference is larger of the determined skin color target pixel value and the average pixel value of the skin color pixel is a person. The image processing apparatus according to claim 1 or 2,
The image processing apparatus further includes:
A skin color pixel ratio calculating unit that calculates a skin color pixel ratio that is a ratio of the skin color pixels to all pixels of the image;
When the shooting scene information represents a person , the color balance adjustment unit calculates a color balance correction value calculated based on the skin color pixel ratio, the average value of the skin color pixels, and the average value of the high luminance pixel values. An image processing apparatus that performs the color balance adjustment using a computer. The image processing apparatus according to claim 3,
When the skin color pixel ratio is large, the color balance correction value is an image processing apparatus has a size. An image processing method for adjusting the color balance of an image,
(A) calculating the average value of the skin color pixels among all the pixels of the image;
(B) calculating an average value of pixel values of a plurality of high luminance pixels having a predetermined range of colors having low saturation and high luminance among all the pixels of the image;
(C) calculating a color balance correction values used for the color balance adjustment,
A step of executing the color balance adjustment for the image and (d) using the color balance correction value the calculated,
With
The data file representing the image includes shooting scene information representing a shooting scene of the image,
The step (d)
(I) When the shooting scene information represents a person, a color balance correction value calculated based on a predetermined skin color target pixel value, an average value of the skin color pixels, and an average value of the high luminance pixel values To perform the color balance adjustment using
(Ii) When the shooting scene information represents a scene other than a person, the average pixel value of the plurality of high-luminance pixels without using the predetermined skin color target pixel value and the average value of the skin color pixels Performing the color balance adjustment using a color balance correction value calculated based on
Image processing method.

Images