JP2010062920A - Image processor, image processing method, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent excessive correction and effect optimum tone correction within a range that the impression of an image is not changed too much before and after tone correction. <P>SOLUTION: A first correcting amount producing unit 15 produces a correcting amount improvement of viewability in a dark space region, a second correcting amount producing unit 16 produces a correcting amount for suppression of tone crushing in a highlight region, a third correcting amount producing unit 17 produces a correcting amount for preventing the impression of the image from being changed too much before and after the correction, and a correcting amount determining unit 18 determines the final correcting amount Δ. A correcting tone curve producing unit 19 produces a correcting tone curve in accordance with the final correcting amount Δ while a tone correcting unit 20 applies tone correction to respective pixels of input image data based on the produced correction tone curve. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, a program, and a recording medium for automatically correcting the brightness of an image taken with a digital camera, a film scanner, or the like, and is suitable for an MFP, a printer, software attached to a digital camera, and the like. About.

  Images taken backlit by a digital camera or images taken at night are darkly exposed due to underexposure, and if the subject includes a part that corresponds to a shadow, part of the image is darkened. End up. For such an image, various techniques for automatically correcting the brightness have been studied and put into practical use.

  For example, when correcting an image brightly by gradation correction, it is required to perform correction within a range in which side effects such as gradation collapse, gradation skip, and noise amplification are not noticeable. Patent Document 1 is a technique for appropriately correcting gradation without causing gradation collapse or hue shift, and Patent Document 2 determines a correction amount in consideration of the degree of gradation skip and gradation collapse. Technology.

JP 2002-369004 A JP-A-10-126619

  However, even if the gradation is corrected within a range where the above-mentioned side effects are not noticeable, there are cases where the impression of the image greatly changes before and after the correction due to overcorrection, which is not preferable. For example, in an image obtained by photographing a green lawn against a cloudy sky, the lawn portion may be corrected to the brightness as if it was photographed under fine weather. According to the subjective evaluation, most users did not want it, and it was not evaluated as appropriate correction. As another example, it is unnecessary for the user to correct the image of the building against the background of the sunset sky (possibly intentionally) backlit until the details of the building are visible. Yes, this was not evaluated as an appropriate correction.

The present invention has been made in view of the above problems,
An object of the present invention is to provide an image processing apparatus, an image processing method, a program, and a recording medium that prevent overcorrection and perform appropriate gradation correction within a range in which an image impression does not change significantly before and after gradation correction. It is to provide.

  The present invention includes a subject area determination unit that determines a subject area in an image, a representative color extraction unit that extracts a representative color in the subject area, a correction amount upper limit setting unit that sets a correction amount upper limit in the representative color, A correction amount determining means for determining a gradation correction amount within a range in which a color difference before and after gradation correction in the representative color does not exceed the upper limit of the correction amount, and according to the gradation correction amount, The main feature is that a gradation correction means for correcting the gradation of the image is provided.

  Claims 1, 6 to 8: A subject region in an image is determined, a representative color in the subject region is extracted, a correction amount upper limit is set in the representative color, and a color difference between the representative color before gradation correction and after gradation correction However, since the tone correction amount is determined and the tone correction is performed within the range that does not exceed the upper limit of the correction amount, overcorrection is prevented, and it is appropriate within the range where the impression does not change significantly before and after the tone correction. Gradation correction can be performed.

  Claims 2 to 5: Determine a highlight area in the image, determine a color distribution of the highlight area, and change a correction amount upper limit set by the correction amount upper limit setting unit according to the color distribution of the highlight area. Therefore, in order to perform gradation correction by determining the gradation correction amount within the range where the color difference before and after gradation correction in the representative color does not exceed the upper limit of the correction amount, Accordingly, it is possible to prevent overcorrection more effectively in an image of a scene where it is not particularly desired to change the impression, and to perform appropriate gradation correction within a range in which the impression does not change significantly before and after the gradation correction.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

Example 1
FIG. 1 shows the configuration of Embodiment 1 of the present invention. The luminance conversion unit 10 converts the RGB value of each pixel of the input image data in the bitmap format into luminance Y by the following equation.
Y = 0.30 × R + 0.59 × G + 0.11 × B
The histogram creation unit 11 creates a brightness Y histogram. In the case where RGB has a value of 8 bits each of 0 to 255, Y also has a value of 0 to 255.

  The area extraction unit 12 obtains thresholds th1 and th2 for determining a subject area, a dark area, and a highlight area in the image based on the created histogram. FIG. 2A shows the configuration of the region extraction unit 12. The discriminant analysis method for obtaining a binarization threshold value for adaptively separating the background and the object from the histogram shape is used twice. To briefly explain the discriminant analysis method, the threshold between 0 and the binarization threshold of the histogram is one class, and the binarization threshold to 255 is one class, and the threshold that maximizes (inter-class variance / intra-class variance). To obtain a binarization threshold in which the background and the object are well separated (ie, the variance between classes is large), and the background and the object are well organized (ie, the variance within the class is small) It is.

  The threshold value calculation unit 120 based on the discriminant analysis method obtains the binarization threshold th1 by applying the discriminant analysis method to the target range 1 in FIG. The obtained threshold th1 corresponds to a threshold for separating the subject area and the highlight area. The threshold value calculation unit 121 based on the discriminant analysis method obtains the binarization threshold th2 by applying the discriminant analysis method to the target range 2 in FIG. 2B, which is the subject area of the histogram. The obtained threshold th2 corresponds to a threshold for separating the subject area into the dark part area and the other part. The dark part here is not a dim level, but an area that is so dark that it is not possible to see clearly what is in the dark part by visually observing the image under standard light source and monitor settings. .

  The correction tone curve generation unit 19 generates a correction tone curve according to the final correction amount Δ determined by the correction amount determination unit 18. Based on the input image data, the extracted color information, and the like, the first correction amount generation unit 15 generates a correction amount focused on improving the visibility of the dark region, and the second correction amount generation unit 16 selects the highlight region. The third correction amount generation unit 17 generates a correction amount focusing on not changing the impression of the image largely before and after the correction, and the correction amount determination unit 18 The final correction amount Δ is determined.

  FIG. 3 shows a tone curve shape to be finally created. The correction tone curve is a luminance Y conversion table. The control points are input Y = th1, th2, and Ya except for the start point 0 and the end point 255, and have a linear shape in each section. The control points th1 and th2 are threshold values output from the region extraction unit 12 described above, and the control point Ya is a control point in a dark region that is output from a first correction amount generation unit 15 described later. The correction amount Δ represents an increase in the output Y with respect to the input Y at the control point Ya. The slope α1 of 0 ≦ input Y <Ya in the dark region is determined by Ya and Δ. As a constraint, th2 ≦ input Y <th1 corresponding to the subject area other than the dark area, and the gradient α3 is saved by setting the gradient α3 to 1. The gradient of Ya ≦ input Y <th2 changes abruptly at the section boundary. In order to prevent this, the inclination α2 is set between the inclinations α1 and α3. Thus, when the three control points and the correction amount Δ are determined, the entire correction tone curve is determined. If three control points are determined, the other information is α4 instead of the correction amount Δ, or the input and output data set at one point somewhere through the correction tone curve. It is possible to determine the entire corrected tone curve from the constraints.

  The gradation correction unit 20 performs gradation correction on each pixel of the input image data based on the generated correction tone curve. The luminance Y is calculated from the RGB by luminance conversion, the output Y is obtained by referring to the correction tone curve with the calculated luminance as the input Y, and each signal of RGB is multiplied by (output Y / input Y). RGB after tone correction.

  The area area ratio calculation unit 13 obtains the dark area ratio Pd used by the first correction amount generation unit 15 and the highlight area ratio Ph used by the second correction amount generation unit 16.

  FIG. 4 shows the configuration of the area area ratio calculation unit 13. From the histogram and threshold value th1 in FIG. 2B, the highlight / subject pixel number counting unit 130 counts the pixel number Na of the subject region and the pixel number Nb of the highlight region. The number of pixels in the subject area is obtained by adding all the pixels of 0 ≦ Y <th1 in the histogram, and the number of pixels in the highlight area is obtained by adding all the pixels of th1 ≦ Y ≦ 255 in the histogram. . Similarly, the dark part pixel number counting unit 132 counts the number of pixels Nc in the dark part region from the histogram and the threshold value th2. The number of pixels in the dark area is obtained by adding all the numbers of pixels of 0 ≦ Y <th2 in the histogram.

The highlight area ratio calculation unit 131 obtains the area ratio Ph of the highlight area in the image by the following equation.
Ph = Nb / (Na + Nb)
The dark area ratio calculation unit 133 obtains the area ratio Pd of the dark area in the subject area by the following equation.
Pd = Nc / Na
When the image is larger than a predetermined size, the image sampling unit 30 samples the image with an average value. In the first correction amount generation unit 15, the second correction amount generation unit 16, and the subject representative color extraction unit 14, if the number of pixels of the image data to be used is too large, the processing takes time and the resolution is high in terms of performance. If the image is too large, the correction amount may not be obtained accurately due to the influence of noise. If the image is larger than a predetermined size, the image is sampled with an average value. For example, when the long side is larger than 640 pixels, sampling is performed, and when an image whose number of long side pixels is three times 640 is input, an average value is obtained every 3 × 3 pixels and sampling is performed. The pixel value of the image.

  FIG. 5 shows a configuration of the first correction amount generation unit 15. The luminance conversion unit 150 converts the RGB value of the sampling image into a luminance value, and the dark area determination unit 151 determines that the pixel is in the dark area if the luminance value is less than the threshold th2. The color conversion unit 152 converts the sampled image data to Lab, and the clustering unit 153 performs clustering using the dark region determination result and the sampled image data converted to Lab by the color conversion 152 as input.

  FIG. 6 shows the configuration of the clustering unit 153. The preliminary clustering unit 156 divides the dark region determination result into one or more clusters according to the color difference, and then the cluster integration unit 157 integrates the cluster with a small number of pixels into another cluster.

  FIG. 7 is a processing flowchart of the preliminary clustering unit 156. The Lab of the pixel of interest is sequentially input (step 1561), and clustering is performed. When the pixels in the dark area are input for the first time from the beginning of the image, since the number of clusters N is initialized and N = 0 is set (step 1560), conditional branching due to color difference (step 1565). To go to “No” and add cluster 1 as a new cluster, set the number of clusters to N = 1, set the number of pixels in cluster 1 to n (1) = 0, and focus on the average Lab of cluster 1 Lab values are set (steps 1568 and 1569).

Thereafter, if the Lab value of the pixel of interest that is sequentially input is a pixel in the dark area (step 1570), the color difference from the average Lab of each cluster is obtained (step 1563), and the cluster having the smallest color difference among them is determined. A set of the number j and the color difference dE_min is extracted (step 1564). If the color difference dE_min is less than or equal to a preset threshold value dE_th in the conditional branch by color difference (step 1565), the process proceeds to “Yes” to add the pixel of interest to the cluster j and recalculate the average Lab (steps 1566 and 1567). ). The number of pixels n (j) in cluster j is incremented by 1, and the average value of L * a * b * is recalculated according to the following equation.
(Average L after recalculation) = ((Average L before recalculation) × (n (j) −1) + (L of target pixel)) / n (j)
(Average a after recalculation) = ((Average a before recalculation) × (n (j) −1) + (a of target pixel)) / n (j)
(Average b after recalculation) = ((Average b before recalculation) × (n (j) −1) + (b of target pixel)) / n (j)
If the color difference dE_min is larger than the preset threshold value dE_th in the conditional branch by color difference (step 1565), the process proceeds to “No”, a new cluster is added, the number of clusters N is counted up, and the number of pixels of the new cluster is reduced to 0. And the Lab of the pixel of interest is set to the average Lab (steps 1568 and 1569). DE_th represents the boundary color difference whether or not to add a new cluster, and is set to a small value such as dE_th = 3 in consideration of the fact that the area where the dark part is not clearly visible is targeted. It is appropriate to keep it.

  Clustering ends when the Lab data reaches the rear end of the image and there is no more Lab data to input.

  Of the clusters divided by the preliminary clustering unit 156, a cluster having a small number of pixels is often a cluster generated in response to a noise component, and anyway, it is not necessary to extract a subject in significant chunks. Or it is a less important cluster.

  FIG. 8 is a processing flowchart of the cluster integration unit 157. Each cluster is sequentially viewed, and it is determined whether or not the number of pixels in the cluster is smaller than a predetermined threshold n_th (step 1575). If smaller, the color difference between the average Lab of the cluster i of interest and the average Lab of other clusters is obtained. (Step 1576), the cluster number j corresponding to the smallest color difference obtained for all other clusters is extracted (Step 1577), and the cluster i is integrated into the cluster j (Step 1578). The number of pixels in cluster i is added to the number of pixels in cluster j, and the number of pixels in cluster i is changed to zero. The threshold value n_th, which is the boundary of the number of pixels of the cluster to be integrated, is obtained for each image, for example, a value corresponding to the number of pixels of 15% (suitably set to about 10% to 20%) with respect to the number of pixels in the dark area. That is the threshold value.

  Returning to FIG. 5, the determination unit 154 determines whether to perform gradation correction according to the number of clusters. If the number of clusters is 1, it is determined not to perform gradation correction, and if the number of clusters is other than 1, it is determined to perform gradation correction. As a result, tone correction is not performed when an image that does not need to be brightly corrected is input when there is no object in the night sky corresponding to the dark area, such as an image of fireworks shot against the night sky. Can be processed.

  FIG. 9 shows a configuration of the first correction amount determination unit 155. When the determination unit 154 in FIG. 5 determines not to perform gradation correction, the selection unit 159 selects Δ1b = 0 and outputs Δ1 = Δ1a as a correction amount, thereby substantially correcting gradation. To disable. When the determination unit 154 determines to perform gradation correction, the correction amount Δ1b calculated by the correction amount calculation unit 158 is selected and Δ1 = Δ1b is output as the correction amount.

  FIG. 10 shows the configuration of the correction amount calculation unit 158. The first cluster extraction unit 1580 compares the number of pixels of each cluster and extracts the cluster having the largest number of pixels and the cluster having the next largest number. Of the two clusters, the higher brightness is expressed as cluster A, and the lower brightness is expressed as cluster B. The color difference calculation unit 1582 calculates the color difference dE1 between the average Lab before correction of the cluster A and the cluster B. The correction amount generation unit 1592 sets the correction amount Δ1b, and the temporary correction tone curve generation unit 1591 generates a temporary correction tone curve corresponding to the correction amount. The tone correction unit 1585 performs tone correction using the temporary correction tone curve for RGB_a obtained by converting the average Lab of cluster A into RGB by the color conversion unit 1783 and RGB_b obtained by converting the average Lab of cluster B into RGB similarly. The color conversion unit 1586 converts the corrected value into a Lab signal again. The color difference calculation unit 1587 calculates the color difference dE2 between the average Lab after correction of the cluster A and the cluster B. The color difference ratio calculation unit 1588 calculates and obtains the ratio of the color difference dE1 before correction and the color difference dE2 after correction.

  The second cluster extraction unit 1581 extracts the darkest cluster. The darkest cluster is denoted as cluster C. Incidentally, although the cluster B and the cluster C may point to the same cluster depending on the image, it does not matter. In the gradation correction unit 1585, gradation correction using the temporary correction tone curve is performed on RGB_c obtained by converting the average Lab of the cluster C into RGB by the color conversion unit 1584, and the value after correction is performed by the color conversion unit 1586. It is converted again into a Lab signal.

If the following end condition 1 or end condition 2 is satisfied, the dark portion correction amount determination unit 1589 outputs the correction amount Δ1b at that time and ends. If neither end condition is satisfied, the correction amount generation unit 1592 generates a changed correction amount and repeats the process until the end condition is satisfied.
[End Condition 1] The color difference ratios before and after the correction of cluster A and cluster B are equal to or greater than the target value Xd set in the color difference ratio target value setting unit 1583.
[End Condition 2] The average L after correction of the cluster C is equal to or greater than the upper limit value Lc_th set in the lightness upper limit setting unit 1593 of the darkest cluster.

  The correction so that the color difference between the cluster having the largest number of pixels and the next largest cluster is X times before correction is more visible than the correction by paying attention to the color difference ratio between other clusters having a small area. The improvement is easily recognized by the observer and is very effective. Further, limiting the correction amount so that the darkest cluster is not corrected too brightly is effective in preventing a side effect in which dark part noise is conspicuous.

  FIG. 11 is a diagram for explaining the correction amount limitation of the darkest cluster. (A) shows a dark part, (b) is the figure which expanded the dark part. Yc and Yc_th express the correction amount limitation of the darkest cluster in an easy-to-understand manner, Yc represents the luminance after correction of the cluster C, and Yc_th represents the upper limit value by luminance.

  The correction amount generation unit 1592 in FIG. 10 sets Δ1b = 0 as an initial value, and generates a value obtained by adding 1 to Δ1b as the next correction amount every time a correction amount generation is requested. The temporary correction tone curve generation unit 1591 creates a correction tone curve as shown in FIG. 11 using the luminance Ya obtained by converting the luminance of RGB_a by the luminance conversion unit 1590 as a control point. The control point Ya corresponds to the luminance of cluster A (the brighter of the two clusters having a large area). A correction tone curve is created so that the output at the control point Ya becomes Ya + Δ1b, and the areas other than the dark area are not used by the first correction amount generation unit 15 and may be set in any way, as shown in FIG. , Ya ≦ input Y <255 may be connected by a straight line.

  The color difference ratio target value setting unit 1583 calculates a color difference ratio target value Xd = 3.5 × Pd according to the dark area ratio Pd. The calculation formula is derived from the subjective evaluation result that prepares dozens of sample images and how much is set to correct the brightness to almost all images (at least 80% of the prepared samples). It is. The higher the proportion of the dark area in the subject, the higher the tendency to obtain higher evaluation when the dark area visibility is emphasized, and the lower the proportion of the dark area, the higher the evaluation is given to the subject image quality other than the dark portion. Therefore, there is a tendency that the required level for improving the visibility of dark parts is lowered.

  In the darkest cluster lightness upper limit setting unit 1593, it is appropriate that Lc_th is set to a value of about 15 (between 10 and 18). This may be a predetermined fixed value.

  FIG. 12 shows the configuration of the second correction amount generator 16. The second correction amount generation unit obtains and outputs the inclination α4 ′ in the highlight area of the correction tone curve as the correction amount.

  The luminance conversion unit 1604 converts the RGB value of the sampling image into a luminance value, and the highlight target region determination unit 1605 determines that the pixel is in the highlight target region if the luminance value Y is th1 ≦ Y <255. . Here, an area obtained by excluding the white pixel of Y = 255 from the highlight area is set as a target area.

  The slope α4 ′ lower limit setting unit 1600 calculates the lower limit value Z = 1 / (− 1.4 × Ph + 2.4) of α4 ′ according to the highlight area ratio Ph, and the correction amount generation unit 1601 calculates the correction amount. Z is output as an initial value. This calculation formula is also derived from the subjective evaluation results. As the ratio of the highlight target area in the image increases, the gradation of the highlight (including the fact that it jumps to white) becomes a factor that lowers the evaluation. There is a tendency, and as the proportion of the highlight target area is smaller, the subject image quality tends to be prioritized over the highlight gradation loss. The correction amount generation unit 1601 generates a value obtained by adding +0.03 to α4 ′ as the next correction amount every time there is a subsequent request.

  The temporary correction tone curve generation unit 1602 creates a correction tone curve as shown in FIG. 13 using th1 as a control point. A correction tone curve is created so that the slope is α4 ′ when th1 ≦ input Y ≦ 255, which is the highlight region, and the setting other than the highlight region is not used by the second correction amount generation unit 16 and how it is set. Therefore, it is sufficient to connect 0 ≦ input Y <th1 with a straight line as shown in FIG.

  The gradation correction unit 1603 performs gradation correction based on the created temporary correction tone curve, and the saturated pixel determination unit 1606 is the above-described target region and is determined to be saturated by the gradation correction, It is determined that the pixel is a saturated pixel. Whether or not the image is saturated by the gradation correction is determined with reference to the RGB values before and after the correction. If the number of signals in which the corrected value is saturated to (255−x1) or more among the RGB3 signals is larger than the number of signals saturated in 255 before correction, it is determined that the pixel is saturated by gradation correction. . x1 is a parameter for determining that saturation is included in pixels that are not completely saturated to 255 but close to saturation in order to make the determination of saturation close to human visual judgment, and the value is about x1 = 20 It is appropriate to set it.

The saturated pixel number counting unit 1607 counts the number of pixels Nd determined to be saturated, and the target pixel number counting unit 1608 counts the pixel number Ne of the target region. The saturation calculation unit 1609 obtains the ratio of saturated pixels with respect to the target region as the saturation Pw by the following equation.
Pw = Nd / Ne
The highlight correction amount determination unit 1610 outputs a saturation amount that is equal to or lower than the upper limit value Xh set in the saturation upper limit setting unit 1611 as an end condition, and outputs α4 ′ when the end condition is satisfied as a correction amount. To finish. As long as the end condition is not satisfied, the correction amount generation unit 1601 generates the changed correction amount and repeats the process until the end condition is satisfied.

  In the saturation upper limit setting unit 1611, it is appropriate that Xh is set to a value of about 0.15 (between 0.1 and 0.2). A predetermined fixed value may be used.

The subject representative color extraction unit 14 in FIG. 1 extracts a representative color in the subject region. Two configuration examples are shown.
First Configuration Example of Subject Representative Color Extraction Unit FIG. 14A shows a first configuration example of the subject representative color extraction unit 14. The dark part / subject area determination unit 141 compares the luminance value obtained by the luminance conversion unit 140 with each pixel value of the sampled image data and the thresholds th1 and th2, and determines a subject area other than the dark part region and the dark part region in the image. Then, based on the image data converted into Lab by the color conversion unit 142 and the region determination result of the region determination unit 141, the dark region clustering unit 143 clusters the dark region, and the subject region clustering unit 144 other than the dark region does not include the dark region. Cluster the subject area.

The clustering 143 of the dark area is the same as the clustering unit 153 in the first correction amount generation unit 15 in FIG. 5, and the result of the clustering unit 153 may be referred to without performing clustering again. The clustering unit 144 for the subject area other than the dark part uses the same clustering method as that for the dark part, but sets dE_th, which is a threshold for determining whether or not it belongs to the same cluster, to a value larger than that for the dark part. Set and cluster. Previously, it was stated that it is appropriate to set a value as small as dE_th = 3 in clustering for dark areas, but in contrast, dE_th = 8 is set in clustering for object areas other than dark areas. It is appropriate to do. The representative cluster extraction unit 145 extracts one cluster having the largest number of pixels in the dark region and the subject region other than the dark region, that is, the entire subject region, and outputs the average Lab of the cluster as the representative color Lab_d of the subject.
Second Example Configuration of Subject Representative Color Extraction Unit In a simple method, there is a method in which a median luminance value in a subject area is obtained from a histogram and a representative color is extracted based on the median value. FIG. 14B shows a second configuration example of the subject representative color extraction unit 14. The subject median value extraction unit 146 obtains the median value Ym in the subject region shown in FIG. 15 from the histogram. The number of pixels is added in order from Y = 0, and the luminance value at the time when the number of pixels in the subject area becomes ½ or more is Ym. Next, the Lab value corresponding to Ym is obtained. The median value determination unit 148 extracts the sampled image data whose luminance value is equal to Ym, and the addition calculation unit 149 for each RGB adds the pixel values of the pixels having the luminance value Ym for each RGB, thereby obtaining the entire image. When the addition for each RGB is completed, the RGB average value calculation unit 1400 divides the addition value for each RGB by the number of pixels of the median Ym to calculate the RGB average value of the pixels having the luminance value Ym, and the color conversion unit 1401 The image converted into Lab is output as the representative color Lab_d of the subject.

  FIG. 16A shows the configuration of the third correction amount generation unit 17. As shown in FIG. 16B, the third correction amount generation unit 17 calculates and outputs an increase β as a correction amount with the corrected value in the luminance Yd corresponding to the representative color Lab_d of the subject as Yd + β.

  The color conversion unit 170 converts the representative color Lab_d into RGB, the gradation correction unit 173 performs gradation correction of the RGB value according to the correction amount β generated by the correction amount generation unit 172, and the color conversion unit 174 The corrected value is converted to Lab. The color difference calculation unit 175 calculates the color difference before and after correcting the representative color.

  In the subject representative color correction amount determination unit 176, the end condition is that the color difference before and after the correction of the representative color is equal to or greater than the upper limit value Xm set in the representative color correction amount upper limit setting unit 177, and when the end condition is satisfied Output β as a correction amount together with Yd, and the process ends. As long as the end condition is not satisfied, the correction amount generation unit 172 generates the changed correction amount and repeats the process until the end condition is satisfied.

  The correction amount generation unit 172 sets β = 0 as an initial value, and generates a value obtained by adding 1 to β as the next correction amount every time a correction amount generation is requested.

  In the representative color correction amount upper limit setting unit 177, when a predetermined fixed value is used, Xm is set to a value of about 15 but this value prevents overcorrection of the image and does not greatly change the impression. Appropriate as an upper limit setting.

  FIG. 17 shows the configuration of the correction amount determination unit 18. A correction amount Δ1 generated by the first correction amount generation unit 15, a correction amount Δ2 calculated from α4 ′ representing the slope of the highlight generated by the second correction amount generation unit 16, and a third correction With reference to the correction amount Δ3 calculated from the luminance of the subject representative color generated by the amount generation unit 17 and Yd and β indicating the correction amount, the minimum value selection unit 182 selects and outputs the minimum one ( Δ = min (Δ1, Δ2, Δ3)).

  As the correction amounts Δ2 and Δ3, it is possible to calculate the correction amount Δ = Δ2 at the control point Ya corresponding to the highlight inclination α4 = α4 ′ using the relationship shown in FIG. It is also possible to calculate the correction amount Δ = Δ3 in the correction tone curve passing through Yd + β). This is performed by the correction amount Δ2 calculation unit 180 and the correction amount Δ3 calculation unit 181.

  As described above, according to the present embodiment, in addition to improving the visibility of the dark area of the image, preventing noise, and suppressing the gradation collapse in the highlight area, the correction amount upper limit of the subject representative value is set, and the correction amount upper limit is set. Since the correction amount is determined within the range that does not exceed and the tone correction is performed according to the correction amount, overcorrection is prevented and tone correction is made to an appropriate brightness within a range that does not change the impression of the original image too much. be able to.

Example 2
FIG. 18 shows a configuration of the second embodiment of the present invention. As one of the shooting modes, there is a digital camera having a landscape shooting mode suitable for landscape shooting, and an image processing apparatus in which a user can select a landscape priority processing mode for performing image correction suitable for a landscape image.

  In the present embodiment, the highlight color extraction unit 40 obtains information on the shooting mode and the image processing mode with reference to the bibliographic information and the like, and the highlight color extraction unit 40 selects the highlight region for an image that is assumed to be a landscape scene. The color distribution to which the target color belongs is determined, and the correction amount upper limit setting value of the subject representative color is switched according to the color classification classified from the color distribution. This effectively reduces the amount of correction for scenes where you do not want to change the impression according to the hue or brightness of the sky (for example, an image of the sunset sky as the main image). Make appropriate corrections within the specified range.

  In the second embodiment, the highlight color extracting unit 40 is added to the first embodiment. Further, the processing of the third correction amount generation unit 17 ′ is different from that of the first embodiment.

  FIG. 19 shows a configuration of the highlight color extraction unit 40. The achromatic pixel determination unit 401 determines whether or not the input pixel value is an achromatic pixel. | Max (R, G, B) −Min (R, G, B) | <15, it is determined as an achromatic pixel; otherwise, it is determined as not an achromatic pixel (that is, a chromatic pixel). To do.

  The red phase pixel determination unit 402 determines whether or not the input pixel value is a red phase pixel. Here, simply, if Max (R, G, B) = R, the pixel is determined to be a red phase pixel, and otherwise, it is determined not to be a red phase pixel.

  The input RGB pixel value is converted into luminance Y by the luminance conversion unit 403, and the luminance determination unit 404 determines the magnitude of the luminance. Using a threshold value Ywt_th for determining the magnitude of luminance with respect to an achromatic color pixel, if Y ≧ Ywt_th, it is determined that the pixel is a bright achromatic pixel candidate, and if Y <Ywt_th, it is a dark achromatic pixel candidate. judge. Further, by using a threshold value Ybl_th for determining the luminance level of the blue phase pixel, if Y ≧ Ybl_th, it is determined that it is a bright blue phase pixel candidate, and if Y <Ybl_th, it is a dark blue phase pixel candidate. Is determined. The threshold values Ywt_th and Ybl_th are appropriately set to values of Ywt_th = 220 and Ybl_th = 160 so that Ywt_th> Ybl_th.

The highlight area determination unit 400 determines that the pixel is in the highlight area if the luminance value of the image is equal to or greater than the threshold th1, and the color area determination unit 405 to which the pixel belongs determines that the pixel is in the highlight area. It is determined which color region the selected pixel belongs to.
That is,
If it is an achromatic pixel and a bright achromatic pixel candidate, it is determined that the pixel belongs to a bright achromatic region.
If it is an achromatic pixel and a dark achromatic pixel candidate, it is determined that the pixel belongs to a dark achromatic region.
If it is not an achromatic pixel and a red phase pixel, it is determined that the pixel belongs to the red region.
If the pixel is not an achromatic pixel, is not a red phase pixel, and is a bright blue phase pixel candidate, it is determined that the pixel belongs to a bright blue region.
If the pixel is not an achromatic pixel, is not a red phase pixel, and is a dark blue phase pixel candidate, it is determined that the pixel belongs to a dark blue region.
In this embodiment, pixels that are not achromatic pixels and are not red phase pixels are determined to be blue phase pixels.

A pixel number counting unit 406 for each color area counts the number of pixels belonging to each of the five color areas. The highlight color distribution determination unit 407 calculates the percentage of the area ratio occupied by the pixels of each color region with respect to the highlight region, and determines the color distribution.
That is,
If the number of pixels belonging to the red region is 20% or more, it is determined that the image has a large distribution of red in highlights.
If the pixel does not correspond to the above color distribution and there are 20% or more of pixels belonging to the dark achromatic region, it is determined that the image has many dark achromatic colors distributed in the highlight.
If there is 70% or more of pixels that do not correspond to the color distribution and belong to the bright achromatic region, it is determined that the image has many bright achromatic colors distributed in the highlight.
If the pixel does not correspond to the color distribution and there are 50% or more of pixels belonging to the dark blue region, it is determined that the image has a lot of dark blue in the highlight.
If there is 50% or more of pixels that do not correspond to the color distribution and belong to the bright blue region, it is determined that the image has a lot of bright blue in the highlight.
If it does not correspond to any of the above color distributions, it is determined that the color distribution is unknown.

  Each threshold value for the area ratio used in the color distribution determination is shown as a specific numerical value in the above. However, this is an example, and an image is observed in order to perform a color distribution determination more suited to human senses. However, it is better to adjust the threshold value.

The highlight color classification unit 408 determines the color classification corresponding to the five sky colors assumed to have a color distribution.
That is,
・ Images that are determined to have a large distribution of red are classified as sunset or sunrise sky.
An image determined to be an image in which many dark achromatic colors are distributed is classified as a dark cloudy sky.
An image determined to be an image in which many bright achromatic colors are distributed is classified as a bright cloudy sky.
An image determined to be an image in which many dark blue colors are distributed is classified as a dark blue sky.
An image that is determined to be an image in which a lot of bright blue is distributed is classified as a bright blue sky.
・ Images determined to have unknown color distribution are classified as other.

  The bright cloudy sky and the dark cloudy sky are classified separately because the impression of the original image differs between a gentle white sky and a dark sky like a thundercloud, and the danger of the dark cloudy sky destroying the impression with overcorrection This is because it is considered to be higher. The bright blue sky and the dark blue sky are classified separately, assuming that the morning and evening sky is not rising as a dark blue sky, and the dark blue sky gives a better impression of the image than the bright blue sky. This is because the demands to be held are considered high.

FIG. 20A shows the configuration of the third correction amount generator 17 ′ in the second embodiment. The difference from the third correction amount generation unit 17 of the first embodiment (FIG. 16A) is the processing content of the representative color correction amount upper limit setting unit 178. The representative color correction amount upper limit setting unit 178 sets a correction amount upper limit according to the scene information and the highlight color classification, and determines and outputs a subject representative color correction amount β according to the correction amount upper limit. The correction amount upper limit is set as shown in FIG. 20B in accordance with the highlight color classification when the scene information is regarded as an image of a landscape.
That is,
The bright blue sky correction amount upper limit L1 and the bright cloudy sky correction amount upper limit L2 are set to satisfy the relationship of L2 <L1.
The correction amount upper limit L1 for the bright blue sky and the correction amount upper limit S1 for the dark blue sky are in a relationship of S1 <L1.
The correction amount upper limit L2 for the bright cloudy sky and the correction amount upper limit S2 for the dark cloudy sky have a relationship of S2 <L2.
The correction amount upper limit S3 for sunset and sunrise sky is at least the same as the correction amount upper limit for dark blue sky or dark cloudy sky, or a value smaller than that, but at least the correction amount upper limit L1 for bright blue sky On the other hand, a relationship of S3 <L1 is established.
・ If the color classification is other than that, an intermediate correction amount upper limit is set as the default value. The intermediate correction amount upper limit roughly indicates a value smaller than L1 and L2 and larger than S1 and S2.

  As described above, according to the present embodiment, the color distribution in the highlight region is determined with respect to hue and luminance, and the correction amount upper limit in luminance correction is changed according to the color distribution (corresponding to the color classification). It is possible to effectively suppress the correction amount in an image of a scene where it is not particularly desired to change the impression according to the brightness.

  In addition, the present invention supplies a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and the computer (CPU or MPU) of the system or apparatus is stored in the storage medium. It is also achieved by reading and executing the program code. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments. As a storage medium for supplying the program code, for example, a hard disk, an optical disk, a magneto-optical disk, a nonvolatile memory card, a ROM, or the like can be used. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on an instruction of the program code. A case where part or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included. Further, after the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. This includes the case where the CPU or the like provided in the board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing. Further, the program for realizing the functions and the like of the embodiments of the present invention may be provided from a server by communication via a network.

The structure of Example 1 of this invention is shown. The structure of an area | region extraction part is shown. The shape of the correction tone curve of the present invention is shown. The structure of an area area ratio calculation part is shown. The structure of the 1st correction amount production | generation part is shown. The structure of a clustering part is shown. It is a process flowchart of a preliminary | backup clustering part. It is a process flowchart of a cluster integration part. The structure of the 1st correction amount determination part is shown. The structure of a correction amount calculation unit is shown. It is a figure explaining the correction amount restriction | limiting of the darkest cluster. The structure of the 2nd correction amount production | generation part is shown. The correction | amendment tone curve produced by the temporary correction tone curve production | generation part is shown. The 1st, 2nd structural example of a to-be-photographed object representative color extraction part is shown. It is a figure explaining the process of a to-be-photographed object median value extraction part. The structure of the 3rd correction amount production | generation part is shown. The structure of the correction amount determination part is shown. The structure of Example 2 of this invention is shown. The structure of a highlight color extraction part is shown. The structure of the 3rd correction amount production | generation part in Example 2 is shown.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Brightness conversion part 11 Histogram creation part 12 Area extraction part 13 Area area ratio calculation part 14 Subject representative color extraction part 15 First correction amount generation part 16 Second correction amount generation part 17 Third correction amount generation part 18 Correction Quantity determination unit 19 Correction tone curve generation unit 20 Gradation correction unit 30 Image sampling unit

Claims (8)

  A subject area determining means for determining a subject area in the image; a representative color extracting means for extracting a representative color in the subject area; a correction amount upper limit setting means for setting a correction amount upper limit for the representative color; and Correction amount determining means for determining a gradation correction amount within a range in which a color difference before and after gradation correction does not exceed the upper limit of the correction amount; and a gradation level of the image according to the gradation correction amount. An image processing apparatus comprising gradation correction means for correcting a tone.   A highlight area determination unit that determines a highlight area in the image; and a color distribution determination unit that determines a color distribution of the highlight area, and the upper limit of the correction amount according to the color distribution of the highlight area. The image processing apparatus according to claim 1, wherein the upper limit of the correction amount set by the setting unit is changed.   The color distribution determination means includes determination of an achromatic color distribution state and a blue color distribution state, and when it is determined that many achromatic colors are distributed in the highlight region, compared to a case where a large amount of blue is distributed, The image processing apparatus according to claim 2, wherein an upper limit of the correction amount is reduced.   The color distribution determination means further includes determination of a red distribution state. When it is determined that a large amount of red is distributed in the highlight area, the correction amount upper limit is set as compared with a case where a large number of blue pixels are distributed. The image processing apparatus according to claim 2, wherein the image processing apparatus is made smaller.   The color distribution determination means includes determination of a luminance distribution state, and when it is determined that a large amount of low luminance is distributed in the highlight area, the correction amount upper limit is reduced as compared with a case where a large amount of high luminance is distributed. The image processing apparatus according to claim 2, wherein:   A subject region determination step for determining a subject region in the image; a representative color extraction step for extracting a representative color in the subject region; a correction amount upper limit setting step for setting a correction amount upper limit for the representative color; and A correction amount determining step for determining a gradation correction amount within a range in which a color difference before and after gradation correction does not exceed the upper limit of the correction amount; and a step of the image according to the gradation correction amount. An image processing method comprising a gradation correction step for correcting a tone.   A program for causing a computer to realize the image processing method according to claim 6.   A computer-readable recording medium on which a program for causing a computer to implement the image processing method according to claim 6 is recorded.

Images