JP5218767B2 - Image processing apparatus, image processing method, program, and recording medium - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, a program, and a recording medium that perform gradation correction on an image shot with a digital camera, a film scanner, or the like, and is suitable for an MFP, a printer, software attached to a digital camera, and the like. Regarding technology.

  Images taken backlit by a digital camera or images taken at night are darkly exposed due to underexposure, and if the subject includes a part that corresponds to a shadow, part of the image is darkened. End up. For such an image, various techniques for automatically correcting the brightness have been studied and put into practical use.

  For example, Patent Document 1 assumes a backlight image having two peaks on the high luminance side and the low luminance side of the luminance histogram, obtains a boundary value corresponding to the valley position of the luminance histogram, and calculates the center of gravity on the low luminance side from the boundary value. Is a technology that brightens the image so that the target correction value becomes the target correction value. At the same time, the center of gravity on the low luminance side is corrected to the target correction value, and at the same time, the gradation correction amount is limited by the position of the gravity center on the higher luminance side than the boundary value Technology is also disclosed. Gradation correction is performed according to a correction table created so that the center of gravity on the low-brightness side or the high-brightness side becomes the target correction value (the high-brightness side is the target correction value for limiting the gradation correction amount). The table is created in a polygonal line shape with the centroid on the low luminance side or the high luminance side as the break point.

  Patent Document 2 extracts a main subject area, calculates a tone correction amount as a function of a median value of the main subject area, and on the other hand, sets a lightness range (brightness range brighter than the main subject) to which the main subject does not belong to a control region. Is a technique for preventing over-correction by controlling the gradation correction amount, and creating a polygonal line-shaped luminance correction curve with the central value A and the maximum value C of the luminance histogram of the main subject area as a break point. Is set to 1 so that the gradation of the main subject area is not impaired.

  By the way, in the technique of Patent Document 1, the luminance corresponding to the center of gravity is corrected to the optimal correction target value, but the optimal correction cannot always be performed in the entire image. For example, there is no guarantee that a plurality of subject objects that are present in the dark part of an image and are difficult to identify visually can be sufficiently identified by correction. In addition, the correction amount can be limited by using the center of gravity on the high luminance side, but this is also limited only by the luminance value of the center of gravity. There is a possibility of limiting. Similarly, in the technique of Patent Document 2, since the gradation correction amount is calculated as a function of the median value of the luminance histogram, even if the luminance corresponding to the median value is optimal, the optimal correction can be performed by looking at the entire image. Not necessarily.

  The inventor first extracts a dark area of an image and clusters the dark area according to the color difference. When the number of clusters is 1, brightness correction is not performed. When the number of clusters is greater than 1, the inter-cluster color difference is determined. By correcting the brightness by determining the correction amount based on the above, the technique for correcting the gradation so that the visibility of the dark area is improved only for an image where an object is hidden in the dark area (Japanese Patent Application No. 2008-180337). ) And the subject area in the image is extracted, the representative color in the subject area is extracted, the upper limit of the correction amount in the representative color is set, and the color difference before and after gradation correction in the representative color does not exceed the upper limit of the correction amount Proposed a technique (Japanese Patent Application No. 2008-227086) for determining the correction amount and correcting the gradation.

  In addition, a technique for limiting the correction amount based on the saturation level of the highlight area is also proposed. In the embodiment, the dark area, the subject area, and the highlight area are extracted from the luminance histogram by combining both inventions, and the threshold value of each area is folded. An example is shown in which tone correction is performed by generating a tone correction table (corrected tone curve) having a polygonal line shape as points. In this embodiment, unlike the methods of creating a gradation correction table based on correction target values for one or two luminance values as in Patent Documents 1 and 2, the dark area visibility is improved and noise prevention, In addition, the gradation correction in the highlight area is controlled, and the correction amount is set so that the impression of the image is not changed too much for the subject area, and the gradation is corrected to the optimal brightness that is balanced throughout the image. However, it is necessary to determine at least three areas according to the brightness in order to perform gradation correction to an optimal brightness balanced in the entire image.

The present invention has been made in view of the above problems,
The first object of the present invention is to accurately determine at least three areas according to brightness so that gradation correction can be made to an appropriate brightness as viewed in the entire image, and further, a luminance histogram Image processing that accurately determines at least three areas according to the brightness so that the gradation can be corrected to an appropriate brightness even when the image is extremely biased and dark overall. An apparatus, an image processing method, a program, and a recording medium are provided.

  Further, in the prior art 1, the center of gravity is used as a break point in the tone correction table having a polygonal line shape. However, since the center of gravity often has high-frequency luminance, the gradation is uniform in the high-frequency luminance portion. The image is corrected to an unnatural feeling, and the prior art 2 also uses the median value of luminance as a break point, so that it is similarly corrected unnaturally.

  Therefore, the above-mentioned problem can be solved by using a curve-shaped gradation correction table instead of a polygonal line shape. However, if a curve-shaped gradation correction table reflecting the correction amount is created, the calculation amount increases. In other words, it is not always a practical method because it is difficult to finely control the correction amount. In practice, a polygonal tone correction table is often used.

  The second object of the present invention is to determine at least three regions according to brightness based on the first threshold value and the second threshold value corresponding to the valley position of the histogram, and the first threshold value and the second threshold value. Image processing apparatus, image processing method, and program in which unnatural deterioration of gradation due to image processing is prevented by using as a break point when creating a polygonal line shape gradation correction table as well as area determination And providing a recording medium.

The present invention includes a luminance histogram creating means for creating a luminance histogram of the input image, the analysis range from the minimum value of the input value of the luminance histogram to a maximum value, inter-class variance is large and the intraclass variance is two small classes The first threshold value determining means for determining the first threshold value corresponding to the luminance value to be divided into , the first threshold value is compared with a preset threshold value, and when the first threshold value is less than the threshold value, was determined to be totally dark image input image, when said first threshold is greater than or equal to the threshold value, and the dark image determining unit determines that the input image is not a totally dark image, the input image is If it is determined that the image is not entirely dark, the analysis range of the dark portion from the minimum value of the input value of the luminance histogram to the first threshold is set to have a large interclass variance and an intraclass variance. A second threshold value corresponding to the luminance value to be divided into two classes is determined, and if it is determined that the input image is an overall dark image, the first value of the input value of the luminance histogram A second threshold value determining means for determining a second threshold value corresponding to a luminance value for dividing the analysis range of the bright part from the threshold value to the maximum value into two classes having a large inter-class variance and a small intra-class variance ; When it is determined that the input image is a dark image as a whole, it is determined that the first threshold value is corrected to a bright value and a threshold value correction unit, and the input image is not a dark image as a whole. In this case, a pixel whose luminance is less than the second threshold is determined as a pixel in a dark area, and a pixel whose luminance is higher than the first threshold is determined as a pixel in a highlight region. In the dark area, the A pixel that is not a light region is determined as a pixel in a subject region, and when it is determined that the input image is an overall dark image, a first threshold value in which the luminance of the input image is corrected to the bright value Pixels that are less than the second threshold are determined as pixels in the dark area, pixels in which the luminance of the input image is greater than or equal to the second threshold value are determined as pixels in the highlight area, and pixels that are neither the dark area nor the highlight area are by determining the pixel, and the region determination unit for performing region determination be divided into at least three regions corresponding to the input image to the brightness, on the basis of the determination result of the region determining unit, floor relative to the input image The main feature is to include image processing means for performing tone correction .

  According to the present invention, it is possible to accurately determine at least three areas according to brightness so that gradation correction can be performed to an appropriate brightness as viewed in the entire image. To accurately determine at least three areas according to the brightness, so that the gradation can be corrected to an appropriate brightness even when the image is extremely biased and dark overall. Can do.

The structure of Example 1 of this invention is shown. An example of a histogram and an example of a result of area determination of an image are shown. The structure of a threshold value determination part is shown. The analysis range 1 and the analysis range 2 when the input image is not a dark image, and the threshold value to be determined are shown. The analysis range 1, analysis range 2, and threshold value to be determined when the input image is a dark image are shown. The shape of the correction table created by the present invention is shown. The structure of an area area ratio calculation part is shown. The structure of the 1st correction amount production | generation part is shown. The structure of a clustering part is shown. It is a process flowchart of a preliminary | backup clustering part. It is a process flowchart of a cluster integration part. An example of the clustering result is shown. The other example of a clustering result is shown. The structure of the 1st correction amount determination part is shown. The structure of a correction amount calculation unit is shown. It is a figure explaining the correction amount restriction | limiting of the darkest cluster. The structure of the 2nd correction amount production | generation part is shown. The correction table created by the temporary correction table creation unit is shown. 2 shows a configuration of a subject representative color extraction unit. The structure of the 3rd correction amount production | generation part is shown. The structure of the correction amount determination part is shown. The structure of Example 2 of this invention is shown. The structure of the 1st correction amount production | generation part in Example 2 is shown. The structure of the 2nd correction amount production | generation part in Example 2 is shown.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 shows the configuration of Embodiment 1 of the present invention. The luminance conversion unit 10 converts the RGB value of each pixel of the input image data in the bitmap format into luminance Y by the following equation.
Y = 0.30 × R + 0.59 × G + 0.11 × B
The histogram creation unit 11 creates a brightness Y histogram. In the case where RGB has a value of 8 bits each of 0 to 255, Y also has a value of 0 to 255.

  The threshold value determination unit 12 obtains threshold values th1 and th2 for determining a subject area, a dark area, and a highlight area in the image based on the created histogram. FIG. 2A shows an example of a histogram. The threshold th1 is a threshold corresponding to the boundary between the highlight area and the subject area, and the threshold th2 is a threshold corresponding to the boundary between the dark area and the other area in the subject area. FIG. 2B shows an example of the result of determining the area of the image. The sky is the highlight area, the shadow area of the building that is the subject, the part that is difficult to see what is reflected is the dark area, and the other subject areas are determined as the subject areas other than the dark area . The threshold value determination unit 12 determines threshold values th1 and th2 for performing such area determination.

  FIG. 3A shows the configuration of the threshold value determination unit 12. In FIG. 3A, 120 corresponds to the first threshold value determination unit, and 121 corresponds to the second threshold value determination unit. The threshold value is determined using twice a discriminant analysis method for obtaining a binarization threshold value that adaptively separates the background and the object from the histogram shape.

FIG. 3B illustrates the discriminant analysis method. Briefly, it is an evaluation value of the discriminant analysis method with one class from the minimum value st to the threshold value T of the histogram and one class from the threshold value T to the maximum value en of the input (inter-class variance / class By obtaining a threshold value T that maximizes (internal variance), the background and the object are well separated (ie, the variance between classes is large), and the inside of the background and the object are well organized (ie, the variance within the class is This is a method of obtaining a (small) binarization threshold.
Here, “a luminance value that divides a predetermined range of the luminance histogram into two classes having a large inter-class variance and a small intra-class variance” is defined as a valley position.

  By using the discriminant analysis method, it is possible to find a valley position (not local but global) within the analysis range. Note that the discriminant analysis method is suitable as a means for searching for the threshold value corresponding to the valley position, so in this embodiment, an example using the discriminant analysis method is presented. Any other method may be used as long as it can adaptively search for the valley position.

  In the discriminant analysis method 1201, the discriminant analysis method is applied with the analysis range from the minimum value to the maximum value of the input values of the histogram (analysis range 1), and the binarization threshold TH1 is obtained.

  The dark image determination unit 122 determines whether or not the input image is an overall dark image by comparing the threshold value TH1 with a preset threshold value. When the input luminance value of the histogram is 0 to 255, it is empirically appropriate to set the threshold value for dark image determination to a value of about 50.

  When TH1 <50, it is determined that the image is entirely dark (dark image). When TH1 ≧ 50, it is determined that the image is not entirely dark. In addition, although the example which determines using TH1 was shown here, as another method of determining a dark image, for example, a histogram is added from the dark side to obtain an input luminance value corresponding to 10% of the whole, If an input luminance value corresponding to 10% of the whole is on a darker side than a preset threshold value, it may be determined that the image is a dark image.

  The analysis range 2 setting unit 1211 sets the dark side from the minimum value to the threshold value TH1 among the input values of the histogram as the analysis range 2 when the image is not a dark image, and sets the input value of the histogram when the image is a dark image. The bright side from the threshold value TH1 to the maximum value is set as the analysis range 2.

  In the discriminant analysis method 1212, the discriminant analysis method is applied to the set analysis range 2 to obtain the threshold value TH2. The threshold value correction unit 1202 corrects the threshold value TH1 to a value TH1 'brighter than TH1 when the image is a dark image. The correction method will be described later. If it is not a dark image, TH1 '= TH1 and correction is not performed.

  The ternary threshold determination unit 123 determines the thresholds th1 and th2 as th1 = TH1 ′ (= TH1) and th2 = TH2 when the image is not a dark image, and th1 = TH2 and th2 = TH1 when the image is a dark image. The thresholds th1 and th2 are determined as'.

  FIG. 4 is an example in the case where the input image is not a dark image, (a) is a histogram, (b) is an interclass variance and intraclass variance for the analysis range 1, and (c) is a discriminant analysis method for the analysis range 1. It is a graph of an evaluation value. The horizontal axes of (b) and (c) are the threshold value T for discriminant analysis. The analysis range 1 and the analysis range 2 and the threshold values to be determined are as illustrated. The darker side than TH1 is set as analysis range 2, and TH2 is obtained.

  FIG. 5 is an example in the case where the input image is a dark image, where (a) is a histogram, (b) is inter-class variance and intra-class variance for analysis range 1, and (c) is a discriminant analysis method for analysis range 1. It is a graph of evaluation value of. The analysis range 1 and the analysis range 2 and the threshold values to be determined are as illustrated. The side brighter than TH1 is set as analysis range 2, and TH2 is obtained.

  In the case of a dark image, since it has a histogram shape with no peaks on the highlight side, the threshold value corresponding to the valley position determined by the first threshold value determination unit 120 is the valley position in the area originally determined as the dark area. (Thus, TH1 is less than 50, which is a value that should be clearly in the original dark area). On the other hand, if the same processing as that for an image that is not a dark image is performed without distinguishing whether the image is a dark image, the threshold value TH2 to be determined next is set to a darker value than TH1. The thresholds th1 and th2 are determined in the dark area, and the targeted area cannot be determined.

  Therefore, in the case of a dark image, the analysis range 2 is set to be brighter than TH1, so that TH2 is set to a value brighter than that in the original dark area. In this case, since the magnitude relationship between TH1 and TH2 is opposite to that in the case of not a dark image, the ternary threshold determination unit 123 reverses the correspondence relationship between TH1 and TH2 so that TH2 is the highlight area and the subject area. Threshold value th1 corresponding to the boundary of.

  Since the threshold value TH1 is also set in the original dark part region as described above, the threshold value correcting unit 1202 determines that it is near the boundary between the original dark part region and the other regions as seen from the histogram shape. It is corrected to TH1 '.

  A method for obtaining TH1 'in the threshold correction unit 1202 will be described. In the example of FIG. 5, the number of pixels other than the dark area is essentially very small and most of them are in the dark area. Therefore, in the histogram of FIG. I want to reset TH1 'near the boundary with the luminance on the side (the falling position of the histogram). Comparing the histogram of FIG. 5A and the evaluation value of the discriminant analysis method of FIG. 5C, the falling position of the histogram is the evaluation that appears first when the graph of FIG. 5C is viewed from the highlight side. Corresponds to the inflection point of the value. In addition to the example of FIG. 5, the same correspondence holds for the histograms of other dark images and the evaluation values of the discriminant analysis method.

  Therefore, the position of the inflection point that first appears on the highlight side is obtained and set as the threshold TH1 'after the correction of the threshold TH1. The inflection point in (c) is obtained by connecting the threshold value TH1 and the evaluation value graph at the position of the maximum value in the analysis range 1 with a straight line (dotted line connecting A and B in FIG. 5C), and in a range brighter than TH1. The input luminance value that maximizes the distance between the dotted line connecting A and B and the solid line of the evaluation value graph is found and determined as TH1 ′.

  When the image is larger than a predetermined size, the image sampling unit 30 samples the image with an average value. If the number of pixels in the image data to be used is too large, processing takes time, and if the resolution is too high in terms of performance, the correction amount for gradation correction may not be obtained accurately due to the influence of noise. If is larger than a predetermined size, the image is sampled with an average value. For example, when the long side is larger than 640 pixels, sampling is performed, and when an image whose number of long side pixels is three times 640 is input, an average value is obtained every 3 × 3 pixels and sampling is performed. The pixel value of the image.

The region determination unit 40 determines whether each pixel of the sampling image belongs to the dark region or whether it belongs to the highlight region based on the threshold values th1 and th2. The luminance Y is obtained from the RGB values of the sampled image, and the luminance Y is compared with threshold values th1 and th2.
If Y <th2 and Max (R, G, B) −Min (R, G, B) <50, it is determined that the pixel is in the dark area.
If Y ≧ th1, it is determined that the pixel is in the highlight area.

  The reason for adding Max (R, G, B) −Min (R, G, B) <50 to the determination condition of the pixel in the dark area is that the area having relatively high saturation and visible is excluded from the dark area. This is to make a determination. By determining the dark area and the highlight area, it is obvious that a pixel that does not belong to any one belongs to a subject area other than the dark area, so that the image is brightened as shown in FIG. Thus, the region determination to be divided into the three regions is performed. The region determination result is expressed by 2 bits, and the first 1 bit indicates whether or not the pixel is in the dark region, and the next 2 bits indicate whether or not the pixel is in the highlight region.

  The gradation correction table generation unit 19 generates a gradation correction table according to the final correction amount Δ determined by the correction amount determination unit 18. Based on the input image data, the extracted color information, and the like, the first correction amount generation unit 15 generates a correction amount focused on improving the visibility of the dark region, and the second correction amount generation unit 16 selects the highlight region. The third correction amount generation unit 17 generates a correction amount focusing on not changing the impression of the image before and after correction, and the correction amount determination 18 The final correction amount Δ is determined.

  FIG. 6 shows the shape of the correction table to be finally created. The correction table is a luminance Y conversion table. The control points are input Y = th1, th2, and Ya except for the start point 0 and the end point 255, and have a linear shape in each section. The control points th1 and th2 are threshold values output from the above-described threshold value determination unit 12, and the control point Ya is a control point in the dark area output from the first correction amount generation unit 15 described later. The correction amount Δ represents an increase in the output Y with respect to the input Y at the control point Ya. The slope α1 of 0 ≦ input Y <Ya in the dark region is determined by Ya and Δ. As a constraint, th2 ≦ input Y <th1 corresponding to the subject area other than the dark area, and the gradient α3 is saved by setting the gradient α3 to 1. The gradient of Ya ≦ input Y <th2 changes abruptly at the section boundary. In order to prevent this, the inclination α2 is set between the inclinations α1 and α3. As a result, when the three control points and the correction amount Δ are determined, the entire correction table is determined. Further, if three control points are determined, the other information may be α4 instead of the correction amount Δ, or the input and output data set at some point that passes through the correction table. It is possible to determine the entire correction table from the conditions.

  The gradation correction unit 20 performs gradation correction on each pixel of the input image data based on the generated correction table. The luminance Y is calculated from the RGB by luminance conversion, the output Y is obtained by referring to the correction table using the calculated luminance Y as an input, and each RGB signal multiplied by (output Y / input Y) is a gradation. Let it be RGB after correction.

  The area area ratio calculation unit 13 obtains the dark area ratio Pd used by the first correction amount generation unit 15 and the highlight area ratio Ph used by the second correction amount generation unit 16.

  FIG. 7 shows the configuration of the area area ratio calculation unit 13. From the histogram of FIG. 2A and the threshold th1, the highlight / subject pixel number counting unit 130 counts the pixel number Na of the subject region and the pixel number Nb of the highlight region. The number of pixels in the subject area is obtained by adding all the pixels of 0 ≦ Y <th1 in the histogram, and the number of pixels in the highlight area is obtained by adding all the pixels of th1 ≦ Y ≦ 255 in the histogram. . Similarly, the dark part pixel number counting unit 132 counts the number of pixels Nc in the dark part region from the histogram and the threshold value th2. The number of pixels in the dark area is obtained by adding all the numbers of pixels of 0 ≦ Y <th2 in the histogram.

The highlight area ratio calculation unit 131 obtains the area ratio Ph of the highlight area in the image by the following equation.
Ph = Nb / (Na + Nb)
The dark area ratio calculation unit 133 obtains the area ratio Pd of the dark area in the subject area by the following equation.
Pd = Nc / Na

  FIG. 8 shows a configuration of the first correction amount generation unit 15. The clustering unit 153 acquires information on whether or not the pixel is in the dark region from the region determination result, and performs the clustering using the sampled image data converted into Lab by the color conversion unit 152 as an input.

  FIG. 9 shows the configuration of the clustering unit 153. The preliminary clustering unit 156 divides the dark region determination result into one or more clusters according to the color difference, and then the cluster integration unit 157 integrates the cluster with a small number of pixels into another cluster.

  FIG. 10 is a processing flowchart of the preliminary clustering unit 156. The Lab of the pixel of interest is sequentially input (step 1561), and clustering is performed. When the pixels in the dark area are input for the first time from the beginning of the image, since the number of clusters N is initialized and N = 0 is set (step 1560), conditional branching due to color difference (step 1565). To go to “No” and add cluster 1 as a new cluster, set the number of clusters to N = 1, set the number of pixels in cluster 1 to n (1) = 0, and focus on the average Lab of cluster 1 Lab values are set (steps 1568 and 1569).

Thereafter, if the Lab value of the pixel of interest that is sequentially input is a pixel in the dark area (step 1570), the color difference from the average Lab of each cluster is obtained (step 1563), and the cluster having the smallest color difference among them is determined. A set of the number j and the color difference dE_min is extracted (step 1564). If the color difference dE_min is less than or equal to a preset threshold value dE_th in the conditional branch by color difference (step 1565), the process proceeds to “Yes” to add the pixel of interest to the cluster j and recalculate the average Lab (steps 1566 and 1567). ). The number of pixels n (j) in cluster j is incremented by 1, and the average value of L * a * b * is recalculated according to the following equation.
(Average L after recalculation) = ((Average L before recalculation) × (n (j) −1) + (L of target pixel)) / n (j)
(Average a after recalculation) = ((Average a before recalculation) × (n (j) −1) + (a of target pixel)) / n (j)
(Average b after recalculation) = ((Average b before recalculation) × (n (j) −1) + (b of target pixel)) / n (j)
If the color difference dE_min is larger than the preset threshold value dE_th in the conditional branch by color difference (step 1565), the process proceeds to “No”, a new cluster is added, the number of clusters N is counted up, and the number of pixels of the new cluster is reduced to 0. And the Lab of the pixel of interest is set to the average Lab (steps 1568 and 1569). DE_th represents the boundary color difference whether or not to add a new cluster, and is set to a small value such as dE_th = 3 in consideration of the fact that the area where the dark part is not clearly visible is targeted. It is appropriate to keep it. Clustering ends when the Lab data reaches the rear end of the image and there is no more Lab data to input.

  Of the clusters divided by the preliminary clustering unit 156, a cluster having a small number of pixels is often a cluster generated in response to a noise component, and in any case, when extracting a subject in significant chunks, Unnecessary or less important cluster.

  FIG. 11 is a process flowchart of the cluster integration unit 157. Each cluster is sequentially viewed, and it is determined whether or not the number of pixels in the cluster is smaller than a predetermined threshold n_th (step 1575). If smaller, the color difference between the average Lab of the cluster i of interest and the average Lab of other clusters is obtained. (Step 1576), the cluster number j corresponding to the smallest color difference obtained for all other clusters is extracted (Step 1577), and the cluster i is integrated into the cluster j (Step 1578). The number of pixels in cluster i is added to the number of pixels in cluster j, and the number of pixels in cluster i is changed to zero. The threshold value n_th, which is the boundary of the number of pixels of the cluster to be integrated, is obtained for each image, for example, a value corresponding to the number of pixels of 15% (suitably set to about 10% to 20%) with respect to the number of pixels in the dark area. That is the threshold value.

  Returning to FIG. 8, the determination unit 154 determines whether or not to perform gradation correction according to the number of clusters. FIGS. 12, 13A, and 13C are examples of clustering results. FIG. 12A is an image obtained by shooting fireworks against the background of the night sky. As a result of clustering the dark area, all the dark areas become one cluster (cluster 1), and the number of clusters is 1. FIG. 12B is an image of a building illuminated at night. As a result of clustering the dark area, the background and the building are divided into separate clusters, and the number of clusters is two. FIG. 13A is an image obtained by photographing a person with backlight, and the dark portion is divided into five clusters. FIG. 13B is an image obtained by photographing a flower, and is an image in which a dark part region in which a leaf overlaps and becomes a shadow exists. The result of clustering this is shown in FIG. 13C. The leaves and veins hidden in the shadow are roughly divided into different clusters, and the number of clusters is two.

  The determination unit 154 determines that gradation correction is not performed if the number of clusters is 1, and determines that gradation correction is performed if the number of clusters is other than one. As a result, tone correction is not performed when an image that does not need to be brightly corrected is input when there is no object in the night sky corresponding to the dark area, such as an image of fireworks shot against the night sky. Can be processed.

  FIG. 14 shows a configuration of the first correction amount determination unit 155. When the determination unit 154 determines not to perform gradation correction, the selection unit 159 selects Δ1b = 0 and outputs Δ1 = Δ1a as the correction amount, thereby substantially invalidating the gradation correction. . When the determination unit 154 determines to perform gradation correction, the correction amount Δ1b calculated by the correction amount calculation unit 158 is selected and Δ1 = Δ1b is output as the correction amount.

  FIG. 15 shows a configuration of the correction amount calculation unit 158. The first cluster extraction 1580 compares the number of pixels of each cluster and extracts the cluster with the largest number of pixels and the cluster with the next largest number. Of the two clusters, the higher brightness is expressed as cluster A, and the lower brightness is expressed as cluster B. The color difference calculation unit 1582 calculates the color difference dE1 between the average Lab before correction of the cluster A and the cluster B. The correction amount generation unit 1592 sets the correction amount Δ1b, and the temporary correction table generation unit 1591 creates a temporary correction table corresponding to the correction amount. The gradation correction unit 1585 performs gradation correction using the temporary correction table for RGB_a obtained by converting the average Lab of cluster A into RGB by the color conversion unit 1584 and RGB_b obtained by converting the average Lab of cluster B into RGB similarly. Then, the corrected value is converted again into a Lab signal by the color conversion unit 1586. The color difference calculation unit 1587 calculates the color difference dE2 between the average Lab after correction of the cluster A and the cluster B. The color difference ratio calculation unit 1588 calculates and obtains the ratio of the color difference dE1 before correction and the color difference dE2 after correction.

  The second cluster extraction unit 1581 extracts the darkest cluster. The darkest cluster is denoted as cluster C. Incidentally, although the cluster B and the cluster C may point to the same cluster depending on the image, it does not matter. In the gradation correction unit 1585, gradation correction using the temporary correction table is performed on RGB_c obtained by converting the average Lab of the cluster C into RGB by the color conversion unit 1584, and the value after correction is again performed by the color conversion unit 1586. Convert to Lab signal.

If the following end condition 1 or end condition 2 is satisfied, the dark part correction amount determination unit 1589 outputs the correction amount Δ1b at that time and ends. If neither end condition is satisfied, the correction amount generation unit 1592 generates a changed correction amount and repeats the process until the end condition is satisfied.
[End Condition 1] The color difference ratios before and after the correction of cluster A and cluster B are equal to or greater than the target value Xd set in the color difference ratio target value setting unit 1583.
[End Condition 2] The average L after correction of the cluster C is equal to or greater than the upper limit value Lc_th set in the brightness upper limit setting unit 1590 of the darkest cluster.

  The correction so that the color difference between the cluster having the largest number of pixels and the next largest cluster is X times before correction is more visible than the correction by paying attention to the color difference ratio between other clusters having a small area. The improvement is easily recognized by the observer and is very effective. Further, limiting the correction amount so that the darkest cluster is not corrected too brightly is effective in preventing a side effect in which dark part noise is conspicuous.

  FIG. 16 is a diagram for explaining the correction amount limitation of the darkest cluster. (A) shows a dark part, (b) is the figure which expanded the dark part. Yc and Yc_th express the correction amount limitation of the darkest cluster in an easy-to-understand manner, Yc represents the luminance after correction of the cluster C, and Yc_th represents the upper limit value by luminance.

  The correction amount generation unit 1592 in FIG. 15 sets Δ1b = 0 as an initial value, and generates a value obtained by adding 1 to Δ1b as the next correction amount every time a correction amount generation is requested. The temporary correction table generation unit 1591 creates a correction table as shown in FIG. 16 using the luminance Ya obtained by luminance conversion of RGB_a as a control point. The control point Ya corresponds to the luminance of cluster A (the brighter of the two clusters having a large area). A correction table is created so that the output at the control point Ya becomes Ya + Δ1b, and the first correction amount generation unit 15 other than the dark region is not used by the first correction amount generation unit 15 and may be set in any way, as shown in FIG. Ya ≦ input Y <255 may be connected by a straight line.

  The color difference ratio target value setting unit 1583 calculates a color difference ratio target value Xd = 3.5 × Pd according to the dark area ratio Pd. The calculation formula is derived from the subjective evaluation results to prepare dozens of sample images and how much to set them so that almost all images (at least 80% of the prepared samples) can be corrected to appropriate brightness It is. The higher the proportion of the dark area in the subject, the higher the tendency to obtain higher evaluation when the dark area visibility is emphasized, and the lower the proportion of the dark area, the higher the evaluation is given to the subject image quality other than the dark portion. Therefore, there is a tendency that the required level for improving the visibility of dark parts is lowered.

  In the darkest cluster lightness upper limit setting unit 1590, it is appropriate that Lc_th is set to a value of about 15 (between 10 and 18). This may be a predetermined fixed value.

  FIG. 17 shows a configuration of the second correction amount generation unit 16. The second correction amount generation unit obtains and outputs the inclination α4 ′ in the highlight area of the correction table as the correction amount.

  The highlight target area determination unit 1605 acquires information on whether or not the pixel is a highlight area pixel from the area determination result, determines whether or not the RGB values of the sampling image are all 255, and the RGB value is A highlight area excluding pixels (white pixels) that are all 255 is set as a target area.

  The slope α4 ′ lower limit setting unit 1600 calculates the lower limit value Z = 1 / (− 1.4 × Ph + 2.4) of α4 ′ according to the highlight area ratio Ph, and the correction amount generation unit 1601 calculates the correction amount. Z is output as an initial value. This calculation formula is also derived from the subjective evaluation results. As the ratio of the highlight target area in the image increases, the gradation of the highlight (including the fact that it jumps to white) becomes a factor that lowers the evaluation. There is a tendency, and as the proportion of the highlight target area is smaller, the subject image quality tends to be prioritized over the highlight gradation loss. The correction amount generation unit 1601 generates a value obtained by adding +0.03 to α4 ′ as the next correction amount every time there is a subsequent request.

  The temporary correction table generation unit 1602 creates a correction table as shown in FIG. 18 using th1 as a control point. A correction table is created so that the slope is α4 ′ when th1 ≦ input Y ≦ 255, which is a highlight area, and the area other than the highlight area is not used by the second correction amount generation unit 16 and how it is set. Therefore, it is sufficient to connect 0 ≦ input Y <th1 with a straight line as shown in FIG.

  The gradation correction unit 1603 performs gradation correction based on the created temporary correction table, and the saturated pixel determination unit 1606 is saturated when it is determined that the region is the target region described above and is saturated by the gradation correction. It is determined that it is a pixel. Whether or not the image is saturated by the gradation correction is determined with reference to the RGB values before and after the correction. Of the three RGB signals, if the number of signals saturated after correction is greater than (255−x1) is greater than the number of signals saturated at 255 before correction, the pixel is saturated by gradation correction. to decide. x1 is a parameter for determining that saturation is included in pixels that are not completely saturated to 255 but close to saturation in order to make the determination of saturation close to human visual judgment, and the value is about x1 = 20 It is appropriate to set.

The saturated pixel number counting unit 1607 counts the number of pixels Nd determined to be saturated, and the target pixel number counting unit 1608 counts the pixel number Ne of the target region. The saturation calculation unit 1609 obtains the ratio of saturated pixels with respect to the target region as the saturation Pw by the following equation.
Pw = Nd / Ne

  The highlight correction amount determination unit 1610 outputs a saturation amount that is equal to or lower than the upper limit value Xh set in the saturation upper limit setting unit 1611 as an end condition, and outputs α4 ′ when the end condition is satisfied as a correction amount. To finish. As long as the end condition is not satisfied, the correction amount generation unit 1601 generates the changed correction amount and repeats the process until the end condition is satisfied.

  In the saturation upper limit setting unit 1611, it is appropriate to set Xh to a value of about 0.15 (between 0.1 and 0.2). A predetermined fixed value may be used.

  FIG. 19A shows the configuration of the subject representative color extraction unit 14. The subject median value extraction unit 146 obtains the median value Ym in the subject region shown in FIG. 19B from the histogram. The number of pixels is added in order from Y = 0, and the luminance value at the time when the number of pixels in the subject area becomes ½ or more is Ym. Next, the Lab value corresponding to Ym is obtained. The median value determination unit 148 extracts the sampled image data whose luminance value is equal to Ym, and the addition calculation unit 149 for each RGB adds the pixel values of the pixels having the luminance value Ym for each RGB, thereby obtaining the entire image. When the addition for each RGB is completed, the RGB average value calculation unit 1400 divides the addition value for each RGB by the number of pixels of the median Ym to calculate the RGB average value of the pixels having the luminance value Ym, and the color conversion unit 1401 The image converted into Lab is output as the representative color Lab_d of the subject.

  FIG. 20A shows the configuration of the third correction amount generation unit 17. As shown in FIG. 20B, the third correction amount generation unit calculates and outputs an increase β as a correction amount with the corrected value in the luminance Yd corresponding to the representative color Lab_d of the subject as Yd + β.

  The color conversion unit 170 converts the representative color Lab_d into RGB, the gradation correction unit 173 performs gradation correction of the RGB value according to the correction amount β generated by the correction amount generation unit 172, and the color conversion unit 174 The corrected value is converted to Lab. The color difference calculation unit 175 calculates the color difference before and after correcting the representative color.

  The subject representative color correction amount determination unit 176 uses an end condition that the color difference before and after correction of the representative color is equal to or greater than the threshold value Xm set in the representative color correction amount threshold setting unit 177, and β when the end condition is satisfied. Is output as a correction amount together with Yd, and the process ends. As long as the end condition is not satisfied, the correction amount generation unit 172 generates the changed correction amount and repeats the process until the end condition is satisfied.

  The correction amount generation unit 172 sets β = 0 as an initial value, and generates a value obtained by adding 1 to β as the next correction amount every time a correction amount generation is requested.

  In the representative color correction amount threshold setting unit 177, when a predetermined fixed value is used, Xm is set to a value of about 15 but this value prevents overcorrection of the image and does not change the impression greatly. It is suitable as an upper limit (threshold) setting.

  FIG. 21 shows the configuration of the correction amount determination unit 18. The correction amount Δ2 calculation unit 180 calculates the correction amount Δ when the highlight slope is α4 ′ using the relationship shown in FIG. 6 and outputs it as Δ2. Using the relationship shown in FIG. 6, the correction amount Δ3 calculation unit 181 obtains a correction amount Δ when the correction table passes through the points of the input Yd and the output Yd + β, and outputs it as Δ3. Here, since the data of the control points Ya, th1, and th2 are required from FIG. 6 to calculate Δ2 and Δ3, Ya, th1, and th2 are input to the correction amount Δ2 calculation unit 180 and the correction amount Δ3 calculation unit 181. doing. With reference to the correction amount Δ1 generated by the first correction amount generation unit 15, the correction amount Δ2 generated by the correction amount Δ2 calculation unit 180, and the correction amount Δ3 generated by the correction amount Δ3 calculation unit 181, The minimum value selection unit 182 selects and outputs the minimum one (Δ = min (Δ1, Δ2, Δ3)).

  As described above, according to the present embodiment, the discriminant analysis method is applied twice while changing the analysis range, and the valley position of the histogram is determined as the threshold value, thereby appropriately determining the threshold value for the region determination according to the brightness. It can be carried out. It is determined whether the image is a dark image, and the analysis range of the second discriminant analysis method (second threshold determination) is darker than TH1 when the image is not a dark image, and brighter than TH1 when the image is a dark image. By setting to the side, the threshold value can be set as intended in any case. In addition, a polygonal line-shaped gradation correction table having two threshold values for area determination as a breakpoint is created, and gradation correction is performed using the polygonal line-shaped gradation correction table. It is possible to prevent gradation deterioration.

  According to the present invention, a histogram of an input image is created, the histogram is analyzed, a first threshold value corresponding to a valley position of the histogram is determined, and when the input image is not an overall dark image, The histogram of the portion darker than the first threshold is analyzed to determine a second threshold corresponding to the valley position of the histogram, and the input image is brightened based on the first threshold and the second threshold. In order to perform area determination to be divided into at least three areas according to the area and perform image processing on the input image based on the determination result of the area determination, the input image is at least a dark area, a highlight area, and other areas And appropriate image processing parameters (for example, a gradation correction table) are set on the basis of the characteristics of each region, and suitable image processing can be performed.

  Further, according to the present invention, when the input image is a dark image as a whole, the second threshold value determining means analyzes a histogram of a portion brighter than the first threshold value in the histogram, and Since the second threshold value corresponding to the valley position is determined, even when the input image is a dark image, the threshold value determination for region determination can be performed as intended.

  Further, according to the present invention, when the input image is a dark image as a whole, the first threshold value is corrected to a bright value. Can be reset to an appropriate value, and even when the input image is a dark image, threshold determination for region determination can be performed as intended.

  Furthermore, according to the present invention, the first threshold value or the first threshold value corrected to a bright value and the second threshold value are created as a polygonal line-shaped gradation correction table, and the gradation correction is performed. Since the gradation of the input image is corrected according to the table, unnatural gradation deterioration due to image processing can be prevented.

  In the first embodiment, the area determination unit 40 performs area determination that divides an image into three areas, that is, a highlight area, a dark area, and other areas (subject areas other than the dark area) according to brightness, and the area determination result is set to the first area determination result. The correction amount generation unit 15 and the second correction amount generation unit 16 refer to the example, but the first correction amount generation unit 15 refers only to information on whether or not it is a dark area, and The second correction amount generation unit 16 refers only to information about whether or not it is a highlight area, and it is not always necessary to complete all area determinations in one processing block and to pass the area determination result to a subsequent processing block. Absent.

  FIG. 22 shows a configuration of the second embodiment of the present invention. In this embodiment, not all the region determination is completed in advance and passed to the subsequent stage, but a threshold necessary for region determination is passed to determine whether or not it is a dark region within the first correction amount generation unit 15. It is determined whether or not it is a highlight area inside the second correction amount generation unit 16.

  FIG. 23 shows a configuration of the first correction amount generation unit 15 in the second embodiment. The RGB value of the sampled image is converted into a luminance value Y by the luminance conversion unit 150, and the dark region determination unit 151 converts Y <th2 and Max (R, G, B) −Min (R, G, B) <50. Then, it is determined that the pixel is in the dark area. The clustering unit 153 performs clustering using the dark region determination result and the sampled image data converted into Lab by the color conversion unit 152 as inputs.

  FIG. 24 illustrates a configuration of the second correction amount generation unit 16 in the second embodiment. The RGB value of the sampled image is converted into a luminance value by the luminance conversion unit 1604, and the highlight target region determination unit 1605 determines that the pixel is the highlight target region if th1 ≦ Y <255. Here, an area excluding a pixel (white pixel) with Y = 255 from the highlight area is set as a target area. The subsequent processing is the same as in the first embodiment.

  In the present embodiment, unlike the first embodiment, not all the area determinations are completed in one processing block. The dark area determination section 151 and the second area determination section 151 in the first correction amount generation section 15 are not the same. In the correction amount generation unit 16, the area determination unit 1605 distributes the area to determine whether it is a dark area or a highlight area, and the area determination result obtained as a result is substantially This is the same as in the first embodiment, and the same tone correction amount and corrected image as in the first embodiment can be obtained. In other words, even if the area determination processing blocks are dispersed, the area determination is performed so that the image is substantially divided into three according to the brightness as a whole, and the correction amount is generated based on the image characteristics of each area. In addition, it can be said that the image processing apparatus substantially includes an area determination unit that divides an image into three areas according to brightness. By passing the threshold values th1 and th2 instead of the area determination result to the correction amount generation units 15 and 16, a storage area for holding the area determination result of the entire image becomes unnecessary, which can be realized at low cost. There are benefits.

  As described above, according to the present embodiment, by distributing the area determination processing blocks, a storage area for holding the area determination result of the entire image becomes unnecessary, and can be realized at low cost.

  According to the present invention, a storage medium in which a program code of software for realizing the functions of the above-described embodiments is recorded is supplied to a system or apparatus, and a computer (CPU or MPU) of the system or apparatus is stored in the storage medium. This is also achieved by reading and executing the code. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments. As a storage medium for supplying the program code, for example, a hard disk, an optical disk, a magneto-optical disk, a nonvolatile memory card, a ROM, or the like can be used. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on an instruction of the program code. A case where part or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included. Further, after the program code read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. This includes the case where the CPU or the like provided in the board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing. Further, the program for realizing the functions and the like of the embodiments of the present invention may be provided from a server by communication via a network.

DESCRIPTION OF SYMBOLS 10 Brightness conversion part 11 Histogram creation part 12 Threshold value determination part 13 Area | region area ratio calculation part 14 Subject representative color extraction part 15 First correction amount generation part 16 Second correction amount generation part 17 Third correction amount generation part 18 Correction Amount determining unit 19 Gradation correction table creation unit 20 Gradation correction unit 30 Image sampling unit 40 Area determination unit

JP 2003-69825 A JP 2003-69822 A

Claims (5)

A luminance histogram creating means for creating a luminance histogram of the input image,
A first threshold for determining a first threshold value corresponding to a luminance value for dividing the analysis range from the minimum value to the maximum value of the input value of the luminance histogram into two classes having a large inter-class variance and a small intra-class variance . Threshold determination means;
The first threshold value is compared with a preset threshold value, and when the first threshold value is less than the threshold value, it is determined that the input image is an overall dark image, and the first threshold value is equal to or greater than the threshold value. When,
Dark image determination means for determining that the input image is not entirely dark ,
When it is determined that the input image is not a dark image as a whole, the analysis range of the dark portion from the minimum value of the input value of the luminance histogram to the first threshold is set to have a large inter-class variance and within the class. A second threshold value corresponding to a luminance value to be divided into two classes with small variance is determined, and if it is determined that the input image is an overall dark image, the second value of the input value of the luminance histogram is determined. A second threshold value determining means for determining a second threshold value corresponding to a luminance value for dividing an analysis range of a bright portion from one threshold value to a maximum value into two classes having a large inter-class variance and a small intra-class variance. When,
A threshold value correcting means for correcting the first threshold value to a bright value when it is determined that the input image is an overall dark image ;
When it is determined that the input image is not an entirely dark image, a pixel whose luminance is lower than the second threshold is determined as a pixel in a dark area, and the luminance of the input image is the first luminance. A pixel that is equal to or greater than one threshold is determined as a pixel in the highlight area, a pixel that is neither the dark area nor the highlight area is determined as a pixel in the subject area,
If it is determined that the input image is an overall dark image, the input image is determined to be a pixel in a dark area region, where the luminance of the input image is less than the first threshold value corrected to the bright value, and the input A pixel whose image brightness is equal to or higher than the second threshold is determined as a pixel in the highlight area, and a pixel that is neither the dark area nor the highlight area is determined as a pixel in the subject area. Region determining means for performing region determination to divide into at least three regions according to,
An image processing apparatus comprising: an image processing unit that performs gradation correction on the input image based on a determination result of the region determination unit.   The image processing means is a gradation correction means for correcting the gradation of the input image, and the first threshold value or the modified first threshold value and the second threshold value are broken line-shaped floors. The image processing apparatus according to claim 1, further comprising a gradation correction table creating unit configured to create a tone correction table, wherein the gradation of the input image is corrected according to the gradation correction table. A luminance histogram generation step of generating a luminance histogram of the input image,
A first threshold for determining a first threshold value corresponding to a luminance value for dividing the analysis range from the minimum value to the maximum value of the input value of the luminance histogram into two classes having a large inter-class variance and a small intra-class variance . A threshold determination step;
The first threshold value is compared with a preset threshold value, and when the first threshold value is less than the threshold value, it is determined that the input image is an overall dark image, and the first threshold value is equal to or greater than the threshold value. A dark image determination step of determining that the input image is not a dark image as a whole ;
When it is determined that the input image is not a dark image as a whole, the analysis range of the dark portion from the minimum value of the input value of the luminance histogram to the first threshold is set to have a large inter-class variance and within the class. A second threshold value corresponding to a luminance value to be divided into two classes with small variance is determined, and if it is determined that the input image is an overall dark image, the second value of the input value of the luminance histogram is determined. A second threshold value determining step for determining a second threshold value corresponding to a luminance value for dividing an analysis range of a bright portion from one threshold value to a maximum value into two classes having a large inter-class variance and a small intra-class variance. When,
A threshold value correcting step of correcting the first threshold value to a bright value when it is determined that the input image is an overall dark image ;
When it is determined that the input image is not an entirely dark image, a pixel whose luminance is lower than the second threshold is determined as a pixel in a dark area, and the luminance of the input image is the first luminance. A pixel that is equal to or greater than one threshold is determined as a pixel in the highlight area, a pixel that is neither the dark area nor the highlight area is determined as a pixel in the subject area,
If it is determined that the input image is an overall dark image, the input image is determined to be a pixel in a dark area region, where the luminance of the input image is less than the first threshold value corrected to the bright value, and the input A pixel whose image brightness is equal to or higher than the second threshold is determined as a pixel in the highlight area, and a pixel that is neither the dark area nor the highlight area is determined as a pixel in the subject area. An area determination step for determining an area to be divided into at least three areas according to
An image processing method comprising: an image processing step of performing gradation correction on the input image based on a determination result of the region determination step. A program for causing a computer to realize the image processing method according to claim 3 . A computer-readable recording medium recording a program for causing a computer to implement the image processing method according to claim 3 .

Images