JP2005130484A - Gradation correction apparatus and gradation correction program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent gradation correction from becoming unnatural by local property distortion or inclination change that is specific to equalizing of histograms. <P>SOLUTION: The gradation correction apparatus for converting gradations of an original image comprises a histogram creating part, a first property creating part, a second property creating part, a property compositing part, and a gradation correcting part. The histogram creating part creates a histogram from lightness of the original image. The first property creating part finds a first gradation conversion property for making approximately equal fineness and coarseness of probability densities of the histogram. The second property creating part holds a second gradation conversion property limited to a gradation change, wherein a gradation skip does not occur. The characteristic compositing part performs weighted compositing of the first gradation conversion property and the second gradation conversion property, to create a composited gradation conversion characteristics. The gradation correcting part finds the ratio of input/output gradations of the composited gradation conversion characteristics, using the lightness as an input parameter and multiplies the color components of the source image with this ratio to correct the gradations of the original image. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

The present invention relates to a gradation correction apparatus and a gradation correction program for converting the gradation of an original image.
In particular, the present invention relates to a technique for obtaining a histogram of an original image and creating gradation conversion characteristics so as to reduce the density of the histogram.

Conventionally, a technique for performing gradation correction on image data is known.
For example, when a main subject is photographed with backlight using an electronic camera, the background may be photographed extremely brightly. Further, when flash photography is performed, a background located far away may be photographed extremely dark.

These images have strong gradation deviations and lack subtle gradation changes. As a technique for correcting the gradation deviation, a technique called histogram equalization is known.
In this histogram equalization, a luminance component is extracted from the original image to be corrected in units of pixels, the appearance frequency is obtained for each value of the luminance component, and a histogram is created. Next, after accumulating the frequencies of this histogram, the accumulated frequencies are normalized in accordance with the output gradation range to obtain a characteristic curve for gradation correction. In the histogram equalization, gradation correction of the original image is performed according to the characteristic curve thus obtained.

  By this gradation equalization gradation correction, gradations having a high appearance frequency in the original image are corrected in a direction in which the gradation change is expanded. Conversely, for gradations with a low appearance frequency in the original image, correction is made in a direction that narrows the gradation change. As a result, an image with uniform gradation can be obtained.

However, in this case, there is a problem that the slope of the characteristic curve becomes remarkably steep at the peak portion of the appearance frequency, and the image after gradation correction becomes unnatural. Patent document 1 is proposed as an effective method for such a problem.
In Patent Document 1, first, the frequency distribution of the histogram is limited to a predetermined upper limit value. Next, after accumulating the frequency after the upper limit, it is normalized according to the gradation range to create a characteristic curve. By adding such upper limit processing, it is possible to limit the slope of the characteristic curve to be created, and to avoid excessive gradation correction appearing in the image after gradation correction.
US Pat. No. 5,923,383

  However, in the upper limit of Patent Document 1, since the effect is limited to the frequency peak portion, the steep slope of the characteristic curve cannot be suppressed over the entire area, and a steep portion of the characteristic curve appears. Due to such a steep characteristic curve, a gentle gradation change is excessively emphasized, and an unnatural image is still unsolved.

  Furthermore, in a special captured image, there is a case where a specific gradation area does not originally exist in the original image (hereinafter, a gradation area where the appearance frequency is almost zero in the original image is referred to as a missing gradation area). On the histogram, the appearance frequency of zero is arranged in this missing gradation area. As a result, in the histogram equalization, the cumulative frequency hardly changes in this missing gradation region, and the slope of the characteristic curve becomes almost zero. For this reason, after the gradation correction, the gradation of the missing gradation area is compressed to almost zero. In this case, the upper and lower gradations are extremely close to each other across the missing gradation area, and the original gradation difference is lost.

In particular, in a captured image such as a night view, a wide range of missing gradations occurs on the high luminance side. In the histogram equalization, the missing gradation area on the high luminance side is subjected to gradation compression, so that the low luminance side is unnaturally brightly corrected for gradation. As a result, the night view impression is greatly impaired.
Further, when gradation correction is performed brightly in this way, there is a problem that any of the color components (RGB components, etc.) is over the range and changes to an unnatural hue.

  Such a problem of the missing gradation area cannot be solved at all by the upper limit of Patent Document 1. In other words, the upper limit is for dealing with a gradation range in which the appearance frequency exceeds the upper limit value, and it is not possible to deal with a missing gradation range whose appearance frequency is close to zero.

Therefore, the present invention aims to improve the gradation correction for histogram equalization so that the image after gradation correction becomes more natural than before, with the above-mentioned problem of the unresolved missing gradation area as a common problem. Objective.
Another object of the present invention is to improve characteristic changes that appear locally in the gradation conversion characteristics of histogram equalization (particularly too steep slopes).
In particular, another object of the invention according to claim 1 is to enable brightness control which is difficult to achieve without being flexible only by histogram equalization.
Furthermore, another object of the invention according to claims 5 and 6 is to create a lightness value particularly suitable as a reference value for gradation correction.

<Claim 1>
The invention according to claim 1 is a gradation correction device for converting the gradation of an original image, and includes a histogram creation unit, a first property creation unit, a second property creation unit, a property synthesis unit, and a tone correction unit. Is provided.
The histogram creation unit creates a histogram for the lightness (a value that comprehensively represents color density and brightness) of the original image.
The first characteristic creation unit obtains a first gradation conversion characteristic that makes the probability density of the histogram closer to equal.
The second characteristic creation unit holds the second gradation conversion characteristic limited to gradation change that does not cause gradation skip. The second gradation conversion characteristic is preferably created on condition that there is no change in the characteristic curve that is too steep to cause gradation skipping.
The characteristic combination unit weights and combines the first gradation conversion characteristic and the second gradation conversion characteristic to create a combined gradation conversion characteristic.
The gradation correction unit obtains a ratio of input / output gradations by comparing the lightness with a combined gradation conversion characteristic for each pixel constituting the original image, and multiplies the color component of the original image by the ratio. Tone-correct the image.
For example, it is preferable to use the maximum value of the color component for each pixel as the brightness that comprehensively represents the color density and brightness.

Preferably, the second gradation conversion characteristic held by the second characteristic creation unit is a characteristic including a power curve. That is, at least a part of the second gradation conversion characteristic is a relation “y−yo = C (x−xo) ^a ” where output y is a power of input x (where a, C, yo, and xo are adjustment parameters). Matches.

<Claim 2>
According to a second aspect of the present invention, in the gradation correction apparatus according to the first aspect, the histogram creation unit extracts a partial image from the original image and creates a histogram from the partial image.

<Claim 3>
According to a third aspect of the present invention, in the gradation correction device according to the second aspect, the histogram creating unit performs at least one feature determination of brightness, color of the original image, and “subject discrimination such as face discrimination”. A region of the partial image is determined on the basis of this.

<Claim 4>
According to a fourth aspect of the present invention, in the gradation correction device according to any one of the first to third aspects, the second characteristic creating unit generates the second gradation according to the average luminance of the original image. The conversion characteristic is adjusted and changed.

<Claim 5>
According to a fifth aspect of the present invention, there is provided a gradation correction device for converting a gradation of an original image, the first brightness extraction unit, the second brightness extraction unit, the brightness synthesis unit, the characteristic creation unit, and the tone correction unit. Is provided.
The first lightness extraction unit extracts a value that comprehensively represents the color density and brightness from the original image to obtain the first lightness.
The second lightness extraction unit extracts the second lightness by performing weighted addition of the amounts related to the color components constituting the original image.
The lightness combining unit combines the first lightness and the second lightness to obtain the third lightness.
The gradation correction unit obtains a ratio of input / output gradations using the third brightness as a parameter for each pixel constituting the original image, and multiplies the color component by the ratio to correct the gradation of the original image.
For example, it is preferable to use the maximum value of the color component for each pixel as the first brightness that comprehensively represents the color density and brightness.

<Claim 6>
According to a sixth aspect of the present invention, in the gradation correction apparatus according to the fifth aspect, the lightness synthesis unit includes:
In the dark part of the original image, the weighting ratio of the second brightness is set higher than in the bright part of the original image.

<Claim 7>
A gradation correction program according to a seventh aspect includes a program code for causing a computer to function as the gradation correction apparatus according to any one of the first to sixth aspects.

<< Claims 1-4 >>
According to the first to fourth aspects of the present invention, two types of gradation conversion characteristics are weighted and combined to create a combined gradation conversion characteristic.
Among these, one first gradation conversion characteristic is a characteristic created so as to make the brightness histogram closer to each other. That is, in the first gradation conversion characteristic, when the density density change of the lightness histogram is extreme, a steep slope occurs locally. Further, the first gradation conversion characteristic is a characteristic that relatively compresses the gradation of the missing area if there is a missing area in the brightness of the original image.

  On the other hand, the second gradation conversion characteristic is a characteristic in which gradation change is limited to a range in which gradation skip does not occur. That is, even in a portion having a steep inclination in the first gradation conversion characteristic, the second gradation gradation conversion characteristic maintains an inclination that does not cause gradation skipping. Further, the second gradation conversion characteristic is a characteristic for performing gradation correction in a range in which gradation skip does not occur in the entire gradation range regardless of whether or not it is in the missing gradation range.

  By weighting and combining these two types of gradation conversion characteristics, the locally steep characteristics of the first gradation conversion characteristics are effectively relaxed by the second gradation conversion characteristics that do not cause gradation skipping. As a result, a “smooth characteristic curve synthesized tone conversion characteristic” is obtained, in which the probability density of the lightness histogram is evenly approached to some extent and the unnaturalness caused by locally steep slopes and missing areas is suppressed. .

Further, the lightness is given to this combined gradation conversion characteristic as an input parameter, and the ratio of the input / output gradation of this combined gradation conversion characteristic is obtained. The gradation correction is performed by multiplying this ratio by the color component. This lightness is a total value of color depth and brightness, in other words, the size of the color solid in the color coordinate space (virtual cube set so that the coordinate point of the pixel color is on the surface). The value roughly corresponds to.
Therefore, the gradation correction based on the brightness is performed by proportionally expanding or reducing a color solid (a cube with the color coordinate point of each pixel on the surface) centering on the origin of the color coordinate space according to the above-described ratio. Is almost equal. Even if the color solid is expanded to the maximum by this proportional expansion, the coordinate point of the color of each pixel is less likely to protrude beyond the maximum color solid determined by the output gradation range of the color component. There is a low possibility of over-range (color saturation).
Due to the effect of preventing saturation, the hue change due to color saturation is eliminated, and the hue of the image after gradation correction can be maintained in a natural impression.
Therefore, it is possible to obtain a corrected image with little deterioration in image quality due to color saturation and smooth and rich gradation change.

Conventionally, in order to create the above-described excellent gradation conversion characteristics, a veteran image processing engineer has to create a gradation correction curve for each individual image by trial and error. It was a time consuming work. Moreover, since it requires work such as experience and sense, it was impossible to mechanize as it is.
However, the inventions of claims 1 to 4 are very excellent in that this work, which has been difficult in the past, can be accurately realized with a simple algorithm and a gradation conversion characteristic of stable quality can be created according to the original image. Invention.

<< Claims 5 and 6 >>
As described above, the color saturation problem can be improved by performing the gradation correction based on the first brightness (maximum value of the color component).
However, in a captured image of an electronic camera, there may be a lot of noise only for a specific color component (for example, R component). For this reason, the S / N of the first brightness is lowered at a location where the specific color component has the maximum value. Therefore, when tone correction is performed based on the first brightness, noise is noticeable after tone correction depending on conditions.

Accordingly, the inventions of claims 5 and 6 have been made in order to improve the noise problem while improving the problems such as color saturation after gradation correction.
That is, the second lightness (for example, luminance Y) is obtained by weighted addition of the amounts related to the color components. In such second brightness, the noise of a specific color component decreases according to the weighted addition ratio.

  By combining the second lightness and the first lightness to obtain the third lightness, and performing the gradation correction based on the third lightness, the problem of color saturation after the gradation correction is improved compared to the conventional case. Noise problems can be improved.

  In particular, according to the sixth aspect of the present invention, as the third brightness, the first brightness is prioritized in a bright place and the second brightness is prioritized in a dark place. By performing histogram equalization based on the third lightness, the first lightness reliably improves the saturation problem that tends to occur in bright places, while the second lightness ensures that noise problems that are prominent in dark places are noticeable. It becomes possible to improve. By isolating the problem by brightness in this way, it is possible to appropriately improve both the saturation and noise problems.

<< First Embodiment >>
[Description of Configuration of this Embodiment]
FIG. 1 is a block diagram illustrating the configuration of the gradation correction device 11.
The tone correction device 11 is roughly configured to include a histogram creation unit 12, a first property creation unit 14, a second property creation unit 13, a property synthesis unit 15, and a tone correction unit 16.
Among these, the histogram creation unit 12 includes a partial image extraction unit 21, a lightness extraction unit 22, a lightness analysis unit 23, and a smoothing unit 24.
The second characteristic creation unit 13 includes an average luminance calculation unit 31 and a gamma determination unit 32.
The gradation correction device 11 having such a configuration is preferably realized by hardware or software and mounted on an electronic camera.
Further, it is possible to cause the computer to function as the gradation correction device 11 using a gradation correction program describing a processing flow described later.

[Create gradation conversion characteristics]
FIG. 2 is a flowchart for explaining the gradation conversion characteristic creation processing by the gradation correction apparatus 11.
In the following, the process of creating gradation conversion characteristics will be described along the step numbers in FIG.

Step S1: An original image is input to the gradation correction device 11 from the outside. Each pixel of the original image is composed of RGB color component data. First, this original image is input to the partial image extraction unit 21.
The partial image extraction unit 21 determines whether the main pattern of the original image is a person or a landscape by identifying brightness or a skin color region from the original image or by performing a known face identification method. To do.

  An electronic camera is known in which a user appropriately switches a shooting mode (landscape mode, portrait mode, etc.) according to a shooting target. With respect to the original image captured by such an electronic camera, information on the shooting mode can be acquired from the MPU for system control of the electronic camera. It is also possible to acquire shooting mode information from the data file of the original image. The partial image extraction unit 21 may determine whether the main pattern of the original image is a person or a landscape from this shooting mode.

Step S2: When the partial image extraction unit 21 determines that the main pattern of the original image is a landscape, the operation proceeds to step S3. On the other hand, when the partial image extraction unit 21 determines that the main pattern of the original image is a person, the operation proceeds to step S4.

Step S3: When the main pattern is a landscape, the partial image extraction unit 21 sets a histogram population to be described later over the entire screen. After the setting, the partial image extraction unit 21 shifts the operation to step S5.

Step S4: When the main design is a person, the partial image extraction unit 21 sets a histogram population to be described later as a partial image including a skin color area and a face area. After the setting, the partial image extraction unit 21 shifts the operation to step S5.

Step S5: The lightness extraction unit 22 extracts the maximum RGB value for each pixel of the selected population and sets it as lightness L. In this case, in order to increase the speed, it is preferable to extract the lightness L by repeated sampling. Further, the lightness L noise may be reduced in advance by locally adding the lightness L on the image space.

Step S6: The lightness analysis unit 23 analyzes the appearance frequency of the extracted lightness L and obtains a histogram of the lightness L. FIG. 3A is an explanatory diagram showing the state of the brightness histogram.

Step S7: The smoothing unit 24 smoothes the distribution shape of the histogram.
As such smoothing, processing for obtaining a moving average (or moving weighted average) of frequencies is preferable. Further, a soft limit smoothing process such as taking a power of power (0.5 to 0.1 power, etc.) is also preferable. Further, both moving average and power may be performed.
FIG. 3B is an explanatory diagram showing the state of the histogram after smoothing.

Step S8: The first characteristic creation unit 14 obtains the cumulative frequency from the low brightness side for the smoothed histogram.

Step S9: The first characteristic creating unit 14 obtains the first gradation conversion characteristic by normalizing the cumulative frequency so that the reached value of the cumulative frequency becomes the maximum value of the output gradation range after gradation correction.
The solid line curve shown in FIG. 3C is an example of the first gradation conversion characteristic.

Step S10: On the other hand, the average luminance calculation unit 31 calculates and averages the luminance of the original image to determine the average luminance.

Step S11: The gamma determination unit 32 determines adjustment parameters a, xo, and yo for gamma conversion (y = C (x−xo) ^a + yo) based on the average luminance.
In this gamma conversion, if the gradation skip occurs beyond the allowable range, the conversion curve of the gamma conversion is limited. As such a limiting method, for example, a part of the conversion curve may be replaced with a straight line or the like so that gradation skip does not occur. For example, the origin of the conversion curve may be shifted by setting (xo, yo) so that gradation skip does not occur.

Further, brightness adjustment can be realized by changing the adjustment parameter a corresponding to the average luminance. For example, it is preferable to decrease the adjustment parameter a as the average brightness is darker (for example, a = 0.1 to 0.3). As a result, the darkness of the original image can be corrected appropriately.
The second gradation conversion characteristic created in this way has the effect of relaxing the steep slope portion of the first gradation conversion characteristic, and at the same time, it can have the effect of correcting the brightness of the original image. it can.
The adjustment parameter C for gamma conversion may be calculated according to the maximum value of the output gradation range.
By determining such adjustment parameters, the second gradation conversion characteristics can be obtained.

Step S12: The characteristic synthesis unit 15 performs weighted synthesis of the first gradation conversion characteristic obtained in Step S9 and the second gradation conversion characteristic obtained in Step S11, and obtains a synthesized gradation conversion characteristic. The generated combined gradation conversion characteristic is set in the gradation correction unit 16.
Usually, the weighted composition ratio (first: second) is fixed to a predetermined ratio such as 1: 1 or 2: 1.

When fine adjustment according to the user's preference is possible, it is preferable to present a manual slide bar or the like to the user so that the user can set and input the weighted composition ratio. In this case, the user can adjust the taste of the image by changing the weighted composition ratio. Specifically, the scenery of autumn leaves can greatly change the impression. Also, with a pattern such as a pond pond, the impression of water surface reflection can be greatly changed, and an impression can be obtained as if the polarizing filter was attached and detached.
FIG. 4 is a diagram showing the synthesized gradation conversion characteristics obtained in this way.

[Explanation of gradation correction operation]
FIG. 5 is a diagram for explaining the operation of gradation correction by the gradation correction apparatus 11.
Hereinafter, the operation of gradation correction will be described along the step numbers shown in FIG.

Step S21: The gradation correction unit 16 selects a pixel to be subjected to gradation correction from the original image. For the pixel to be processed, the maximum value is obtained from the RGB values constituting the pixel, and the brightness L is obtained.

Step S22: The gradation correction unit 16 collates the extracted lightness L with the set gradation conversion characteristic (in the above-described case, the combined gradation conversion characteristic), and acquires the input / output gradation ratio. . Note that it is preferable to omit the ratio calculation process by preparing a gradation conversion characteristic (formula or look-up table) in which “input gradation” and “input / output gradation ratio” are associated with each other.

Step S23: The gradation correction unit 16 multiplies the acquired ratio by the RGB color component of the pixel to be processed to obtain a gradation-corrected RGB color component. The multiplication processing here is preferably executed at high speed by referring to a table or the like.
By performing such processing on the entire original image, the gradation correction operation is completed. Note that saturation correction may be performed as necessary after completion of gradation correction.
Note that the processing time may be shortened by preventing the lightness extraction in steps S5 and S21 from overlapping. For example, the brightness L may be extracted first for all pixels of the original image and stored as a brightness image, and the brightness L of the partial image may be extracted from the brightness image and used as the population of the histogram.

[Effects of this embodiment, etc.]
As described above, in the present embodiment, the first gradation conversion characteristics that make the probability density of the histogram of lightness L close to equal density and the second gradation conversion characteristics that do not cause gradation skip are weighted and combined. At this time, the local excessively steep inclination of the first gradation conversion characteristic is relieved by the second gradation conversion characteristic. As a result, it is possible to create a combined gradation conversion characteristic that suppresses a sharp change in brightness while equalizing the brightness histogram. Furthermore, the second effect is that the brightness of the image can be appropriately controlled by appropriately determining the gamma a of the second gradation conversion characteristic.

In addition, in the present embodiment, the ratio of input / output gradations of the combined gradation conversion characteristics is obtained based on the lightness L of the original image, and the original image is subjected to gradation correction by multiplying this ratio by the color component.
FIG. 6A is a diagram for explaining the effect of gradation correction based on this lightness.
In this figure, the color of the pixel having the lightness L1 (position on the color coordinate space) is located on the surface of the color solid having the lightness L1. In the present embodiment, in the gradation correction, the input / output gradation ratio (L2 / L1) is obtained by referring to the lightness L1 of the processing target pixel to the combined gradation conversion characteristics, and this ratio is calculated as RGB of the processing target pixel. Multiply by color component. In this case, as shown in FIG. 6A, the RGB color components in the color coordinate space move on the surface of the color solid, so that the maximum color solid M never protrudes. That is, there is no adverse effect such as generation of an unnatural color due to saturation of a specific color component by gradation correction.

Incidentally, FIG. 6B shows a case where gradation correction is performed on the basis of linear combination P1 and P2 of color components as a comparison target of the present embodiment. The linear combination here is, for example, a linear combination [P = αR + βG + (1−α−β) B] with a predetermined ratio of RGB color components.
In this case, an input / output tone ratio (P2 / P1) is obtained by referring to the linear combination P1 of the pixels to be processed to the tone conversion characteristics, and this ratio is multiplied by the RGB color component of the pixel to be processed. Then, as shown in FIG. 6B, there is a possibility that the vector in the color coordinate space exceeds the maximum color solid M. As a result, the specific color component (in this case, the R component) exceeds the range, and an unnatural color due to saturation of the specific color appears in a high-luminance portion. On the other hand, in the present embodiment, as described above, such saturation does not occur.

In addition, the second gradation conversion characteristic of the present embodiment mainly includes a power curve. Since the shape of the power curve changes smoothly, a local excessively steep inclination change as seen in histogram equalization does not occur. Therefore, the second gradation conversion characteristic of the present embodiment is a preferable characteristic for alleviating the negative effects of histogram equalization.
The second tone conversion characteristic is not limited to this power function. For example, the second gradation conversion characteristic is preferably a monotonically increasing type characteristic (where there may be an area with zero inclination) with little local inclination change. For example, as the second gradation conversion characteristic, a power curve, an S-shaped curve, a Bezier curve, or the like is preferable.

Furthermore, in this embodiment, the population of the histogram is limited to a partial range of the image according to the situation. By limiting the population in this way, the possibility of an extreme gradation deviation occurring in a part of the original image appearing in the histogram is further reduced. As a result, it is possible to improve an adverse effect caused by a local sudden change in inclination in histogram equalization.
In the present embodiment, the brightness and color corresponding to the skin color are characterized and the partial range including the person who is the main subject is used as the histogram population. As a result, it is possible to accurately detect the histogram bias in the skin portion of the person. Further, the gradation conversion characteristic can be created so that the bias of the histogram of the skin portion is corrected to a histogram distribution that is preferable as a preset skin portion. As a result, good gradation conversion characteristics can be created without being affected by a portion other than the skin portion. (For example, it is possible to appropriately brighten a dark shadow portion or the like under a clear sky without being affected by other than the skin portion, and a portrait image with less dark portion crushing can be obtained.)

Note that, when the range of the histogram population is limited, the partial image extraction unit 21 may perform the following processes (1) to (3).
(1) First, an original image is divided into multi-areas, and the difference between the luminance (or color component) and the maximum luminance (or maximum color component) of all areas is obtained as a parameter for each divided area.
(2) On the other hand, the partial image extraction unit 21 stores in advance a correspondence relationship between this parameter and an optimal divided area (which may be a combination of a plurality of divided areas). This correspondence is prepared on the basis of a large number of actual shooting results.
(3) The partial image extraction unit 21 determines an optimum divided area (or weighting according to the priority order of the divided areas) by referring to the correspondence obtained with the parameter obtained from the original image. With such a range setting, it is possible to set an optimal divided area in situations such as backlighting, and the adverse effects of histogram equalization can be further prevented.

  Further, if the photographing intention can be determined from the photographing mode or user input, the method for limiting the histogram population may be flexibly changed according to the photographing intention. For example, in the autumn leaves photographing mode, the population may be limited by identifying red, yellow, or brightness areas corresponding to autumn leaves.

  Furthermore, in the present embodiment, the second gradation conversion characteristic is changed according to the average luminance of the original image. Therefore, it is possible to shift the final combined gradation conversion characteristic in a direction in which the brightness of the original image is appropriately corrected. As a result, it is possible to add an adjustment factor for making the brightness of the original image moderate to the mechanical gradation correction called histogram equalization. As a result, tone conversion characteristics that take into account a plurality of adjustment factors that could not be created by an experienced image processor can be created with a simple algorithm.

  In addition, it can be recognized from the average luminance of the original image (preferably the average luminance for each divided area) that the night scene shooting, starry night shooting, or fireworks shooting. In this case, the second gradation conversion characteristic can be limited so that the dark part is not corrected brighter than necessary. In this case, it is possible to perform tone correction of a night scene image or the like that has been corrected unnaturally in histogram equalization while preserving the impression of the night scene.

Furthermore, in this embodiment, the distribution shape of the histogram is smoothed. By this smoothing process, extreme undulations in the histogram are gently suppressed. As a result, the extreme frequency change of the histogram is reduced, and it is possible to effectively suppress the steep slope that appears locally in the histogram equalization characteristic curve. By such an operation, as shown in FIG. 3C, the first gradation conversion characteristic in which the local inclination change is suppressed can be created.
Incidentally, FIG. 3C shows, as comparison objects, the conversion characteristics obtained by the conventional histogram equalization and the conversion characteristics obtained by the conventional upper limit. Compared to these conventional conversion characteristics, the first gradation conversion characteristics of the present embodiment have a small local inclination change and good gradation conversion characteristics.
In particular, the smoothing of the histogram reduces the frequency difference between the missing gradation area and other gradation areas (including areas with a moderate appearance frequency), improving the negative effects of the missing gradation area in histogram equalization. You can also
It should be noted that smoothing such as moving average (or moving weighted average) that fills the missing gradation area with surrounding frequency values is particularly preferable. Such smoothing not only lowers the large frequency peak portion on the histogram distribution but also has an effect of lifting the low frequency valley portion in a well-balanced manner. As a result, while suppressing the steep slope portion of the characteristic curve for histogram equalization, the suppressed portion can be distributed in a well-balanced manner to the gentle slope portion of the characteristic curve. As a result of such an inclination distribution effect, a locally steep spot does not occur on the characteristic curve.
Further, in the moving average, the missing gradation area can be fundamentally removed, the width of the missing gradation area can be narrowed, and the frequency change between the missing gradation area and its adjacent area can be continued smoothly. As a result, it is possible to reliably improve many of the conventional problems caused by the missing gradation area.

  Note that the upper limit of Patent Document 1 is a hard limit, which is clearly different from the smoothing of the present embodiment. In other words, the upper limit has only a function of locally cutting the peak portion of the histogram distribution almost unrelated to the missing gradation area. Therefore, the upper limit of Patent Document 1 cannot achieve the above-described effects of the present embodiment.

  In the present embodiment, this first gradation conversion characteristic is weighted and combined with the second gradation conversion characteristic. However, the present invention is not limited to this, and the tone correction may be performed using the first tone conversion characteristic obtained using the smoothed histogram as it is. In this case as well, it is possible to obtain a gradation correction result with improved conventional problems.

Alternatively, the first gradation conversion characteristic may be created by conventional histogram equalization and weighted with the second gradation conversion characteristic. In this case as well, it is possible to obtain a gradation correction result with improved conventional problems.
Next, another embodiment will be described.

<< Second Embodiment >>
FIG. 7 is a diagram illustrating the configuration of the gradation correction device 40 according to the second embodiment. The gradation correction device 40 may be realized by hardware. Further, the computer may be caused to function as a gradation correction device by a gradation correction program.
A structural feature of the second embodiment is that a curve approximation unit 45 is arranged in place of the characteristic synthesis unit 15 of the first embodiment. In addition, about the same component as 1st Embodiment (FIG. 1), the same referential mark is shown in FIG. 7, and duplication description here is abbreviate | omitted.

FIG. 8 is a flowchart showing a process of creating gradation conversion characteristics in the second embodiment.
In FIG. 8, the operations in steps S1 to S9 are the same as those in the first embodiment (FIG. 2), and thus redundant description is omitted here.
The operational feature in the second embodiment is step S30. That is, the approximate gradation conversion characteristic is obtained by approximating the gradation conversion characteristic (here, the first gradation conversion characteristic) obtained by the equalization of the histogram or its modification method with a curve function. As an approximation method, the least square method or the like is preferable.

The curve function in this case is preferably a monotonically increasing type curve function with little local inclination change (however, there may be an area with zero inclination). For example, a power function, an S-shaped curve function, a Bezier curve, or the like is preferable. For example, for a curve function, it is preferable to replace a part of the curve with a straight line in order to prevent gradation skipping.
Alternatively, curve approximation may be performed for each using a plurality of curve functions, and the one with the smallest approximation error may be selected. Alternatively, curve approximation may be performed by sampling a plurality of points from the gradation conversion characteristics and selecting a curve function that matches the magnitude relationship between these points.

In the approximate tone conversion characteristic, the local inclination change is effectively removed by reflecting the overall tendency of the histogram equalization as shown in FIG. As a result, it is possible to satisfactorily perform tone correction on the overall tendency of tone bias while suppressing unnatural tone correction unique to histogram equalization.
In this approximate gradation conversion characteristic, many of the conventional problems due to the missing gradation area and the steep slope are improved. In particular, it is possible to appropriately reduce “gradation steps (false contours)” and “missing gradation differences” that are generated by locally distorting the gradation correction characteristic curve.

  Note that the synthesized gradation conversion characteristic may be created by performing weighted combination of the approximate gradation conversion characteristic thus obtained and the above-described second gradation conversion characteristic. In this case, various requests for tone correction can be flexibly added by adjusting and changing the second tone conversion characteristics.

  Further, the approximate gradation conversion characteristic may be created by approximating the gradation conversion characteristic created by the conventional histogram equalization to a curve. In this case as well, it is possible to obtain a gradation correction result with improved conventional problems.

<< Third Embodiment >>
A feature of the third embodiment is that the third lightness created by the creation procedure shown in FIG. 10 is used instead of the lightness L of the above-described embodiment. In addition, about the usage method of 3rd brightness, since it is the same as the brightness L of embodiment mentioned above, description here is abbreviate | omitted.
Hereinafter, the procedure for creating the third brightness will be described with reference to FIG.

Step S41: The maximum value is obtained from the RGB color components of the original image for each pixel, and the first brightness L is obtained.
Incidentally, the lightness L of the above-described embodiment may be used as it is as the first lightness L.

Step S42: The RGB color components of the original image are weighted and added in units of pixels according to the following formula to obtain the second brightness P.
P = αR + βG + (1-α-β) B
The weighting ratio of the above formula may be 1: 1: 1 or 1: 2: 1, or may be matched with the weighting ratio of luminance Y (such as 3: 6: 1).

In addition, for example, it is also preferable to reduce the influence of noise by reducing the weighting ratio of the above equation for a color component with a large amount of noise.
Here, the weighted addition may be an amount related to the color component, for example, an amount obtained by linearly converting the color component, or an amount obtained by performing gamma correction on the color component.
Further, the second brightness P may be obtained for the positions sampled on the screen in order to increase the processing speed.

Step S43: The first lightness L and the second lightness P are synthesized for each pixel to obtain the third lightness M. In this case, it is preferable to increase the ratio of the second lightness P included in the third lightness M in darker places. For example, the third brightness M satisfying such a requirement can be generated by the following expression.
M = S · L + (1−S) · P
Or
M = (1 + S) .L / 2 + (1-S) .P / 2
However, the weight ratio S may be an evaluation value of the brightness of the target pixel. For example, as S, the first lightness L, the second lightness M, or an average value or a composite value thereof may be normalized to the numerical range 0 to 1 and used.

Further, the weight ratio S may be made non-linear by taking a power to such a normalized brightness evaluation value. By making the weighting ratio S non-linear, it becomes possible to increase the ratio of the second lightness P included in the third lightness M in a non-linear manner in a dark place where noise is a problem.
In this case, as a weighting function w (S) normalized to 0 ≦ w (S) ≦ 1,
M = w (S) * L + (1-w (S)) * P
Or
M = (1 + w (S)) * L / 2 + (1-w (S)) * P / 2
It becomes.

  Further, for example, when the weight ratio S is changed into a polygonal line, the second lightness P is selected as the third lightness M as it is in a dark place where noise is a problem, and the light component is bright enough to cause color component saturation Thus, the first lightness L can be selected as it is as the third lightness M.

  In the first and second embodiments described above, when the third lightness M is employed instead of the lightness L, noise and saturation problems in gradation correction can be changed flexibly according to the brightness of the portion. It becomes possible to improve.

<< Additional items of embodiment >>
It should be noted that the gradation correction according to the present invention may of course be performed in combination with requantization of pixel values (such as conversion of 12 bits to 8 bits). At this time, if the gradation correction is not based on lightness, a value after re-quantization may be set in advance as the output value of the data table of gradation conversion characteristics. In gradation correction based on lightness, it is preferable to perform requantization when multiplying the input / output gradation ratio by the color component. That is, the value after requantization may be set as the output value of the multiplication data table used for the multiplication process. By such processing, in the present invention, it is possible to execute a plurality of processings of gradation correction and requantization together.

  In the above-described embodiment, the case where the gradation correction is performed on the RGB color components has been described. However, the present invention is not limited to this. For example, it is of course possible to apply the present invention when tone correction is performed for a color component such as a complementary color component or a YCbCr component (Y is also a color component in the present invention).

  In the above-described embodiment, a person / landscape is identified, and the histogram population is switched. However, such a population switching process may be omitted. Also, a certain range (such as the center screen 1/4) may always be set in the histogram population. Further, for example, when information on a focus detection area at the time of photographing or ROI (region of interest) at the time of image compression can be acquired, the vicinity of the region may be selected as the population of the histogram.

  Further, in the above-described embodiment, the maximum value of the color component is extracted from each pixel to obtain the lightness L, and gradation equalization for histogram equalization is performed based on the lightness. However, the present invention is not limited to this. For example, instead of the lightness L, gradation correction for histogram equalization may be performed using luminance Y or a specific color component (for example, G color component) as a reference. Even in the histogram equalization based on the luminance Y or the like, the above-described embodiment (combination of the first gradation conversion characteristic and the second gradation conversion characteristic, etc.) reduces locally steep gradation changes. There are effects such as being able to.

  As described above, the present invention is an invention that can be used in the industrial field for performing gradation correction of an image.

2 is a block diagram illustrating a configuration of a gradation correction device 11. FIG. It is a flowchart explaining the production | generation operation | movement of a gradation conversion characteristic. It is a figure explaining the operation | movement which implements equalization of a histogram, after smoothing a histogram. It is a figure which shows the synthetic | combination gradation conversion characteristic. It is a figure explaining the operation | movement of gradation correction | amendment. It is a figure explaining the gradation correction based on brightness. 3 is a diagram illustrating a configuration of a gradation correction device 40. FIG. It is a flowchart which shows the preparation process of a gradation conversion characteristic. It is a figure which shows an approximate gradation conversion characteristic. It is a flowchart which shows the example of creation of a brightness.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Gradation correction apparatus 12 Histogram creation part 13 2nd characteristic creation part 14 1st characteristic creation part 15 Characteristic composition part 16 Gradation correction part 21 Partial image extraction part 22 Brightness extraction part 23 Brightness analysis part 24 Smoothing part 31 Average brightness | luminance Calculation unit 32 Gamma determination unit 40 Gradation correction device 45 Curve approximation unit

Claims (7)

A gradation correction device for converting the gradation of an original image,
A histogram creating unit that creates a histogram for the brightness of the original image (a value that comprehensively represents color intensity and brightness);
A first characteristic creating unit that obtains a first gradation conversion characteristic that uniformly approximates the density of probability density of the histogram;
A second characteristic creation unit that outputs a second gradation conversion characteristic limited to gradation changes that do not cause gradation skip;
A characteristic synthesizer that weights and synthesizes the first gradation conversion characteristic and the second gradation conversion characteristic to create a combined gradation conversion characteristic;
For each pixel constituting the original image, the lightness is compared with the combined gradation conversion characteristic to obtain an input / output gradation ratio, and the ratio is multiplied by a color component of the original image. A gradation correction apparatus comprising: a gradation correction unit that performs gradation correction on the image. The gradation correction apparatus according to claim 1,
The histogram creation unit
A gradation correction apparatus that extracts a partial image from the original image and creates the histogram from the partial image. The gradation correction apparatus according to claim 2,
The histogram creation unit
A gradation correction apparatus that determines the area of the partial image based on at least one feature discrimination of brightness, color of the original image, and “subject discrimination such as face discrimination”. The gradation correction device according to any one of claims 1 to 3,
The second characteristic creating unit
The gradation correction apparatus according to claim 1, wherein the second gradation conversion characteristic is adjusted and changed according to an average luminance of the original image. A gradation correction device for converting the gradation of an original image,
A first lightness extraction unit that extracts first lightness (a value that comprehensively represents color intensity and lightness) from the original image;
A second lightness extraction unit for extracting a second lightness by weighted addition of an amount related to a color component constituting the original image;
A lightness composition unit for combining the first lightness and the second lightness to obtain a third lightness;
A gradation correction unit that performs gradation correction on the original image by obtaining a ratio of input / output gradations for each pixel constituting the original image using the third brightness as a parameter and multiplying the ratio by a color component; A gradation correction apparatus comprising: The gradation correction apparatus according to claim 5,
The brightness composition unit
The gradation correction apparatus according to claim 1, wherein a weighting ratio of the second brightness is higher in a dark portion of the original image than in a bright portion of the original image.   A gradation correction program for causing a computer to function as the gradation correction apparatus according to any one of claims 1 to 6.

Images