JP4402994B2 - Image processing method, apparatus, and program - Google Patents

Description

  The present invention relates to an image processing method and apparatus for determining a threshold value for discriminating edges and noise in a digital image, particularly a natural image such as a snapshot image, and a program therefor.

  In recent years, digital still cameras (hereinafter referred to as digital cameras) are rapidly spreading. Instead of imaging light information on a silver salt film, a digital camera forms an image on a digital device (CCD, photomultiplier tube, etc.) and stores the image on a recording medium as digital data (hereinafter referred to as a digital image). Since it can be used directly on a computer, it is convenient for image processing and storage.

  Also, photoelectric conversion of images obtained by imaging light information on silver halide films by conventional imaging methods and images obtained by printing on printing media such as photographic paper and paper using a reader such as a scanner The digital data read and stored in the same manner as a digital image acquired by a digital camera, image processing, and the like are also performed.

  Such a digital image (hereinafter abbreviated as an image) is reproduced by being displayed on a display device such as a monitor or being printed on a photographic paper. As expected, it is necessary to perform various image processing on the digital image.

  Among these image processing, image structure conversion processing such as improving the sharpness of an image or suppressing noise has a great influence on the image quality of the processed image. Has been proposed. In addition, simply suppressing noise has the problem that the edges in the image are also suppressed and the processed image is blurred. Simply increasing the sharpness also enhances the noise, Since there is a problem that the image quality of the image is not good, it is desired to improve the sharpness of the image and suppress noise.

  In Patent Document 1, as a method for suppressing noise without losing edges, a signal having a high frequency component is extracted from an image, and if the absolute value of the extracted signal is equal to or smaller than a predetermined threshold, the signal is determined as noise. On the other hand, a method has been proposed in which if the absolute value of the extracted signal is larger than a predetermined threshold value, this signal is judged to be an edge and not removed (coring processing).

  Patent Document 2 also proposes a method for sharpening an image and suppressing noise. In this method, first, a smoothing process using a smoothing filter such as a low-pass filter is performed on an image to obtain a smooth image in which noise is suppressed (an edge is also suppressed), and the pixel of interest is a central pixel, The differential value in the direction from the center pixel to the surrounding pixels is obtained, and if the value is less than or equal to a predetermined threshold value, it is regarded as noise and removed by coring processing. The sum is taken and multiplied by a constant of 1.0 or more, and the last emphasized part is added to the smooth image.

These methods make use of the fact that edges in an image have a large amplitude whereas noise appears as fluctuations in a small amplitude, and the amplitude of noise that can be taken in the image (hereinafter referred to as the noise level) is set as a threshold value. The threshold is used to identify and separate edges and noise, enhance the separated edges to improve sharpness, smooth noise, and suppress noise. is there.
JP2000-175078 JP-T-57-50031

  However, the image structure conversion processing method described above uses a unique threshold for an image when discriminating between edges and noise. On the other hand, an image, particularly a natural image such as a snapshot, has a brightness over a wide range, and it is known that the noise level is lower in a tendency of a brighter portion in such an image. Therefore, if a threshold value for discriminating between edge and noise is used for an image, a unique value is used regardless of the brightness of the image, the discrimination between edge and noise is not appropriate. However, there is a problem that a processed image having a high quality image quality cannot be obtained even if image structure conversion processing is performed by separating the image data.

  Therefore, a method of acquiring a noise level for each brightness level in an image and setting a threshold value for each brightness level is conceivable. However, in the case of a natural image, the amount of information for acquiring the noise level is small. Because there may be brightness or noise level acquisition may be affected by different patterns, the reliability of the noise level directly obtained for each brightness level is low, and the consistency of the entire image is also low There is a problem that it is difficult to remove.

  The present invention has been made in view of the above circumstances, and an image processing method and apparatus capable of appropriately determining a threshold for discriminating between edge and noise for each level of image brightness, and a program therefor Is intended to provide.

A first image processing method of the present invention is an image processing method for obtaining a threshold value for discriminating edges and noise in a digital image from a digital image.
Calculating a noise level of the entire digital image from the digital image;
Determining the threshold for the entire digital image based on the noise level of the entire digital image;
For each brightness level of the digital image, a threshold value for each brightness level of the digital image is obtained by adjusting a threshold value for the entire digital image according to the brightness level. It is.

A second image processing method of the present invention is an image processing method for obtaining a threshold value for discriminating edges and noise in a digital image from a digital image.
Calculating a noise level of the entire digital image from the digital image;
For each brightness level of the digital image, obtain a noise level for each brightness level of the digital image by adjusting the noise level of the entire digital image according to the brightness level;
The threshold value for each brightness level of the digital image is obtained based on a noise level for each brightness level.

  Here, the “digital image” means not only a digital image obtained by imaging a subject with a digital camera or the like but also an image on a silver halide photographic film or printed matter (for example, a print) by a reading device such as a scanner. The obtained digital image is also included. Hereinafter, for convenience of explanation, an image simply means a digital image.

  As described above, it is known that when the brightness level is different in the image, the noise level of the part having the brightness is also different, and an edge and noise are added for each brightness level in the image. Although a threshold value for discrimination is desired, there is a problem that it is difficult to obtain an appropriate noise level due to the influence of a pattern or the like if the noise level is directly calculated for each brightness level. Therefore, in the first image processing method of the present invention, first, the noise level of the entire image is calculated, and a threshold value for the entire image is obtained based on the overall noise level. Then, the threshold value for each brightness level is obtained by adjusting the threshold value for the entire image according to the brightness level. In the second image processing method of the present invention, first, the noise level of the entire image is calculated, and an adjustment corresponding to the brightness level is applied to the noise level of the entire image, so that the brightness level for each brightness level is calculated. Get the noise level. Then, a threshold value for each brightness level is obtained based on the noise level for each brightness level.

  Here, the “noise level” is not limited as long as it can represent the magnitude of noise amplitude, and for example, a noise variance value (also referred to as RMS granularity) can be used.

  An image has different channels depending on the space (RGB, YCrCb, LC1C2, etc.) representing it. For example, an image is represented by each component of three channels of R, G, and B in the RGB space, and is represented by each component of three channels of luminance and color difference in the luminance color difference space (YCrCb, LC1C2, etc.). . In the present invention, for which channel the noise level is to be calculated and how many channels the noise level is to be calculated are the types of space representing the image and the object of image structure conversion processing performed after discriminating edges and noise. The channel for calculating the noise level may be determined depending on the channel. For example, in an apparatus for acquiring an image such as a digital camera or a scanner, it is unlikely that the amount of noise generated in each color pixel of the photoelectric conversion element is greatly different, that is, the noise level in each color of R, G, B is almost the same Therefore, it is only necessary to calculate the noise level for only one channel in the RGB space, and in the luminance color difference space, the noise level can be calculated for each channel.

  Or, for example, in each channel of the luminance color difference space, the luminance component greatly contributes to human vision, and image structure conversion processing (processing that improves sharpness, that is, sharpening processing and / or noise suppression processing) is performed on the luminance component. ), A great image quality improvement effect can be obtained. Therefore, when performing the image structure conversion process in the luminance color space, if only the luminance component is to be processed, the noise level may be calculated only for the luminance component.

  In addition, as a method for obtaining the noise level of the entire image, various conventional methods can be applied. For example, a flat portion with little change in shading (that is, with few edges) is extracted from the image, and the flat portion is extracted. The noise variance value may be obtained and used as the noise level of the entire image. The noise level of the entire image obtained by this method is only the noise level of the flat part, and it is insufficient to correctly evaluate the noise level of the part other than the flat part such as a part with a pattern. It is preferable to determine the noise level of the image. As a method for obtaining the noise level from the entire image, for example, a method of obtaining the color difference noise level from the entire image and estimating the luminance noise level from the color difference noise level can be used. In the luminance component of the image, edges and noise are mixed, and it is difficult to distinguish between the edge and noise, and it is difficult to directly determine the luminance noise level from the luminance component. On the other hand, an image, particularly a natural image such as a snapshot image, has a characteristic that there are few edges although noise exists in the color difference component. In the image processing method of the present invention, it is preferable to calculate the noise level in the luminance color difference space as follows. First, the noise level in the color difference component in the image, that is, the color difference noise level is calculated. And the image acquired by an imaging device such as a digital camera or a reading device such as a scanner has the same cause of noise of luminance and color difference in principle, and therefore, using the relationship between luminance and color difference, The luminance noise level is estimated from the color difference noise level. Specifically, for example, a YCrCb space as a space for luminance and color difference is defined by the following equation (1).

Y = 0.299 × R + 0.587 × G + 0.114 × B
Cr = 0.500 × R−0.419 × G−0.081 × B (1)
Cb = −0.169 × R−0.331 × G + 0.500 × B
Y: luminance
Cr, Cb: Color difference
R, G, B: R value, G value, B value

Here, for example, if the noise level of each component is the RMS value, that is, the standard deviation σ of that component, the relationship shown in the following equation (2) is established.

(ΣY) ² = (0.299 × σR) ² + (0.587 × σG) ²
+ (0.114 × σB) ²
(ΣCr) ² = (0.500 × σR) ² + (0.419 × σG) ² (2)
+ (0.081 × σB) ²
(ΣCb) ² = (0.169 × σR) ² + (0.331 × σG) ²
+ (0.500 × σB) ²
Where σ: standard deviation

On the other hand, as described above, in an apparatus for acquiring an image, it is unlikely that the amount of noise generated in each color pixel of the photoelectric conversion element is greatly different, that is, the noise level in each color of R, G, B is set to σR, σG , ΣB, it is considered that the noise amount of each color is substantially the same as shown in the following equation (3).

σR≈σG≈σB (3)
Where σR, σG, σB: Noise amount

The following equation (4) can be further deduced from the equations (2) and (3).

σY≈1.02σCr≈1.07σCb (4)
Where σY: Luminance noise level
σCr, σCb: Color difference noise level

That is, if the color difference noise level σCr or σCb is known, the luminance noise level σY can be estimated according to the equation (4).

  Further, for example, when the LC1C2 space defined by the following equation (5) is used as the luminance color difference space, the relationship between the luminance noise level and the color difference noise level as in the equation (6) as in the above estimation. Can be obtained.

L = 0.299 × R + 0.587 × G + 0.114 × B
C1 = −0.299 × R−0.587 × G + 0.886 × B (5)
C2 = 0.701 × R−0.587 × G−0.114 × B
Where L: Luminance
C1, C2: Color difference

σL≈σC1 / 1.65≈σC2 / 1.38 (6)
Where σL: Luminance noise level
σC1, σC2: Color difference noise level

Further, the above equations (4) and (6) for estimating the luminance noise level from the color difference noise level are based on the assumption that the components of each color of R, G, and B receive no gain or the same gain. R, G, and B differ in the case where white balance processing is performed or the sensitivity setting of each color pixel is different in an image acquisition device. Since there is a possibility of receiving a gain, when estimating the luminance noise level from the chrominance noise level, rather than estimating the luminance noise level from one chrominance noise level, as shown in equations (7) and (8) It is preferable to estimate the luminance noise level using the average value of the two color difference noise levels.

σY≈ (1.02 × σCr + 1.07 × σCb) / 2 (7)
Where σY: Luminance noise level
σCr, σCb: Color difference noise level


σL≈ (σC1 / 1.65 + σC2 / 1.38) / 2 (8)
Where σL: Luminance noise level
σC1, σC2: Color difference noise level

Further, for example, if the gain received by each of the R, G, and B colors of the image is determined as shown in Expression (9) from the transmittance ratio of the color filter of the apparatus that acquires the image and the image processing conditions such as white balance processing, More preferably, the luminance noise level is estimated from the color difference noise level as shown in the following formulas (10) and (11) according to the gain.

R: G: B = g _r : 1.0: g _b (9)
Where g _r , g _b : gain

Here, the YCrCb space and the LC1C2 space are taken as examples of the luminance / color difference space, but the luminance / chrominance space such as the L * a * b space similarly has a predetermined relationship between the color difference noise level and the luminance noise level. The luminance noise level can be estimated from the color difference noise level even for the luminance color difference space.

  In general, noise in an image has a different influence depending on the frequency band so that the S / N ratio is lower as the frequency band is higher, and the S / N ratio is higher as the frequency band is lower. When the S / N ratio is high, the degree of mixing of the original signal and noise of the image becomes high and it becomes difficult to evaluate the noise level. Therefore, it is desirable to calculate the noise level in the high frequency band, but the image is acquired. Since most of the images are output after being subjected to image processing such as JPEG compression processing and sharpness correction processing in the apparatus, information for acquiring the noise level is lost in the highest frequency band. In many cases, the noise level acquired in this frequency band is not always appropriate. Therefore, in the image processing method of the present invention, when calculating the noise level from the image, it is preferable to calculate the noise level in a high frequency band other than the highest frequency band, preferably in the frequency band next to the highest frequency band. .

    Furthermore, the image output from the image acquisition device may be subjected to enlargement / reduction processing, that is, scaling processing, and the frequency band suitable for obtaining the noise level also changes in the scaling processing image. The image processing method of the present invention determines a frequency band for obtaining a noise level according to a scaling factor of an image, creates a band-limited image representing a component of the frequency band of the image based on the image, The noise level is preferably obtained from the band limited image.

  Further, the first image processing method of the present invention adjusts the threshold for the entire image obtained based on the noise level of the entire image according to the brightness level of the image, and sets the threshold for each brightness level. In contrast, the second image processing method of the present invention adjusts the noise level for each brightness level by adjusting the noise level of the entire image according to the brightness level. The threshold value for each brightness level is obtained from the noise level for each brightness level. When the noise level itself is used as the threshold value, the noise level of the entire image is adjusted, Since the adjustment of the threshold value for the whole is the same, the first image processing method and the second image processing method of the present invention in this case are the same.

  Also, adjusting according to the brightness level of the image (adjusting the threshold for the entire image or adjusting the noise level of the entire image) adjusts according to the distribution of the noise level corresponding to the brightness level of the image. Means that. For example, using a tendency that the noise level tends to be lower as the brighter part of the image (that is, the part where the brightness level is higher), the threshold value for the entire image or the noise level of the entire image is adjusted to decrease as the brightness increases. Thus, a threshold value for each brightness level or a noise level for each brightness level can be obtained.

  In addition, a digital image to be processed by the present invention is often subjected to some image processing before being processed by the image processing method of the present invention. Such image processing may be performed inside an image acquisition device such as a digital camera or may be performed during a process of transferring an image, for example, a process of being transmitted via a server on a network. . Among these image processes, there is a brightness change process for changing the brightness in addition to the above-described scaling process and white balance process. The brightness change process means a process in which the brightness of an image is changed by the process, and examples include exposure correction (AE correction), gamma correction, and hypertone processing.

  When the original image is subjected to the brightness conversion process, the magnitude of the noise amplitude in the processed image (corresponding to the digital image to be processed of the present invention) by the brightness change process is the same as the magnitude of the noise in the original image. And will be different. Here, the case where the digital image of the present invention is obtained by performing exposure correction on an original image acquired by a digital camera will be described as an example.

  Normally, exposure correction is performed in the density space, and FIG. 17 shows the relationship between the density and the QL value of the digital camera. The QL value corresponds to the amount of light that has entered the image sensor of the digital camera, and increases as the amount of light increases until the image sensor becomes saturated. Therefore, the larger the QL value, the greater the brightness of the digital image. The level is also great.

  When exposure correction is performed in the density space, the density is corrected with the same correction amount for the entire original image, and the QL value is changed by this correction. FIG. 16 shows the change brought about in the QL value by the exposure correction. FIGS. 16A and 16B show changes in the QL value in the case of exposure correction by subtracting 0.5 and 0.7 from the density, respectively. The degree of change in the QL value in the image varies depending on the exposure correction, and as shown in the gain curve on the right side of FIG. 4, the larger the QL value, the smaller the degree of change (gain in the figure). In the example of FIG. 16A, if the QL value before correction is 50, the QL value after correction is 100, and if the QL value before correction is 100, the QL value after correction is 180. That is, the gain received by the QL value by this exposure correction is about 2.5 and 2 when the corrected QL values are 50 and 100, respectively.

  On the other hand, a change in the amplitude of noise in the image corresponds to a change in the QL value, and the noise amplitude receives the same gain as the gain received by the QL value. For example, in the example of FIG. 16A, since the corrected QL value of 50 receives a gain of about 2.5, the amplitude of noise is also 2 with respect to the corrected QL value of 50. It will be amplified 5 times.

  That is, the gain received by the amplitude of the noise varies depending on the magnitude of the QL value in the digital image to be processed according to the present invention (for example, the image subjected to exposure correction shown in FIG. 16A).

  Here, the exposure correction has been described as an example, but the gain curve indicating the gain received by the QL value is shown in FIG. 16 even when other processing such as gamma correction is performed to change the brightness of the image. Although different from the gain curve shown, the QL value and the gain received by the noise amplitude differ depending on the magnitude of the QL value.

The image processing method of the present invention pays attention to this, and the original image resulting from the brightness change process is applied to a digital image obtained by performing a brightness change process for changing the brightness of the original image. The degree of change in the amplitude of the noise in (for example, the gain received by the amplitude of the noise) is determined for each brightness level of the digital image,
It is preferable to adjust the threshold for the entire digital image or the noise level of the entire digital image according to the degree.

A first image processing apparatus of the present invention is an image processing apparatus for obtaining a threshold value for discriminating edges and noise in a digital image from a digital image,
An overall noise level calculating means for calculating a noise level of the entire digital image from the digital image;
An overall threshold acquisition means for determining the threshold for the entire digital image based on the noise level of the entire digital image;
Adjusting means for acquiring the threshold value for each brightness level of the digital image by adjusting a threshold value for the whole digital according to the brightness level for each brightness level of the digital image; It is characterized by.

A second image processing apparatus of the present invention is an image processing apparatus for obtaining a threshold value for discriminating edges and noise in a digital image from a digital image,
An overall noise level calculating means for calculating a noise level of the entire digital image from the digital image;
For each brightness level of the digital image, adjusting means for obtaining a noise level for each brightness level of the digital image by adjusting a noise level of the entire digital image according to the brightness level;
Threshold acquisition means for obtaining the threshold value for each brightness level of the digital image based on the noise level for each brightness level is provided.

The adjustment means in the image processing apparatus of the present invention is configured to reduce noise in the original image caused by the brightness change process on a digital image obtained by performing a brightness change process for changing the brightness of the original image. The degree of change in the amplitude of each digital image is determined for each brightness level,
It is preferable that the threshold for the entire digital image or the noise level of the entire digital image is adjusted according to the degree.

  The image processing method of the present invention may be provided as a program that causes a computer to execute the image processing method.

  According to the first image processing method and apparatus of the present invention, in calculating a threshold value for discriminating between edges and noise in an image for each brightness level of the image, first, the noise level of the entire image is calculated. To do. Then, a threshold value for each brightness level is acquired by adjusting a threshold value for the entire image obtained based on the noise level of the entire image according to the brightness level of the image. By doing this, the amount of information for calculating the noise level varies depending on the brightness of the image, or the local picture of the image is different from the method of calculating the noise level directly for each brightness level of the image and obtaining the threshold value. It is possible to prevent the reliability from being lowered due to factors such as being affected, and an appropriate threshold value can be acquired.

  In addition, for a digital image that has been subjected to brightness change processing on the original image, the degree of change in noise amplitude in the original image due to the brightness change processing, that is, the noise amplitude in the original image For each brightness level, the degree of change in magnitude of the amplitude of noise in the digital image of the present invention obtained by performing brightness change processing is obtained from the magnitude, and the threshold for the entire image is determined according to this degree. If adjusted, the threshold value for each brightness level can be obtained more accurately.

  The same applies to the second image processing method and apparatus of the present invention.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus of the present invention performs correction processing for correcting noise in an image on an image, and a correction processing program read into an auxiliary storage device is stored in a computer (for example, a personal computer). Realized by executing above. The correction processing program is stored in an information storage medium such as a CD-ROM or distributed via a network such as the Internet and installed in a computer.

  Further, since the image data represents an image, the following description will be given without particularly distinguishing the image from the image data.

  As shown in FIG. 1, the image processing apparatus of the present embodiment inputs an image D0 (R0, G0, B0) that has been subjected to enlargement / reduction processing and brightness change processing (here, exposure correction processing as an example). The input unit 1, the YCC conversion unit 5 that performs YCC conversion on the image D 0 to obtain the image S 0 (Y 0, Cb 0, Cr 0), and a plurality of blurred images S 1, S 2, ... Sn (n: integer of 3 or more) for each of luminance Y, color difference Cb, color difference Cr, and a plurality of blur images using the image S0 and the blur images S1, S2,. Band-limited image creating means 20 for creating band-limited images T1, T2,... Tn for each luminance / color difference component, and band-limited images Tpb and Tpr for color difference components for analyzing noise from each band-limited image are determined. And A threshold setting unit 30 that analyzes a noise level in the image S0 using the determined band-limited images Tpb and Tpr and sets a threshold λ for discriminating an edge and noise in the image S0, and a threshold setting unit 30 Based on the obtained threshold value λ, noise correction means 40 that performs noise correction on the image S0 to obtain a noise-corrected image S′0 is provided. Hereinafter, details of each component of the present embodiment will be described.

  The image input means 1 inputs the image D0 (R0, G0, B0), and from the additional information of the image D0, the enlargement / reduction ratio α applied to the image D0 and the correction amount of the exposure correction process H is also acquired and output to the threshold setting means 30.

  The YCC conversion means 5 converts the R, G, and B values of the image data D0 into the luminance value Y and the color difference values Cb and Cr according to the above-described equation (1).

Y = 0.299 × R + 0.587 × G + 0.114 × B
Cr = 0.500 × R−0.419 × G−0.081 × B (1)
Cb = −0.169 × R−0.331 × G + 0.500 × B
Y: luminance
Cr, Cb: Color difference
R, G, B: R value, G value, B value

The blurred image creating unit 10 creates a plurality of blurred images for each luminance color difference component of the image S0 using the image S0 obtained by the YCC converting unit 5. Note that the blurred image creating unit 10 creates a blurred image for each component by the same method. Here, the operation of the blurred image creating unit 10 will be described by taking a luminance component as an example. FIG. 2 is a block diagram illustrating a configuration of the blurred image creation unit 10. As shown in the figure, the blurred image creation means 10 includes a filtering means 12 that performs filtering processing to obtain filtered images B1, B2,... Bn, an interpolation means 14 that performs interpolation processing on each filtering image, and filtering. The control means 16 which controls the means 12 and the interpolation means 14 is provided. The filtering means 12 performs a filtering process using a low-pass filter. As the low-pass filter, for example, a filter F substantially corresponding to a 5 × 1 grid-shaped one-dimensional Gaussian distribution as shown in FIG. 3 is used. be able to. This filter F has σ = 1 in the following equation (12).

  The filtering unit 12 performs a filtering process on the entire image to be processed by performing a filtering process on the image to be processed in the x direction and the y direction of the pixel using such a filter F.

FIG. 4 shows details of processing that the control unit 16 of the blurred image creation unit 10 causes the filtering unit 12 and the interpolation unit 14 to perform. As illustrated, the filtering unit 12 first performs a filtering process on the luminance component Y0 of the image S0 every other pixel using the filter F shown in FIG. A filtering image B1 (Y1) is obtained by this filtering process. The size of the filtered image B1 is 1/4 of the size of the image S0 (1/2 in the x direction and y direction, respectively). Next, the filtering unit 12 performs the filtering process by the filter F every other pixel on the filtering image B1 (Y1) to obtain the filtering image B2 (Y2). The filtering means 12 repeats the filtering process by such a filter F, and obtains n filtered images Bk (k = 1 to n). The size of the filtering image Bk has become a ^{1/2 2K} of the size of the image S0. FIG. 5 shows frequency characteristics of each filtered image Bk obtained by the filtering unit 12 when n = 3 as an example. As shown in the figure, the response of the filtered image Bk is such that the higher the k, the higher the frequency component is removed.

  In the image processing apparatus according to the present embodiment, the filtering unit 12 performs the filtering process on the x direction and the y direction of the image by the filter F shown in FIG. A filtering process may be performed on the image S0 and the filtered image Bk at a time using a × 5 two-dimensional filter.

The interpolation unit 14 performs an interpolation process on each filtering image Bk obtained by the filtering unit 12 so that the size of each filtering image Bk is the same as that of the image S0. There are various interpolation processing methods such as a B-spline method. In this embodiment, the filtering unit 12 uses a filter F based on a Gaussian signal as a low-pass filter. A Gaussian signal is also used as an interpolation coefficient for performing the interpolation calculation, and this interpolation coefficient is approximated to σ = 2 ^K−1 in the following equation (13).

When interpolating the filtering image B1, since k = 1, σ = 1. The filter for performing interpolation when σ = 1 in the above equation (13) is a 5 × 1 one-dimensional filter F1 as shown in FIG. The interpolating unit 14 first expands the filtering image B1 to the same size as the image S0 by interpolating the pixels having a value of 0 every other pixel one by one with respect to the filtering image B1, and then the enlarged image A blurred image S1 is obtained by performing a filtering process using the filter F1 shown in FIG. The blurred image S1 has the same number of pixels as the image S0, that is, the same size as the image S0.

  Here, the filter F1 shown in FIG. 7 is a 5 × 1 filter. However, before applying the filter F1, pixels having a value of 0 are interpolated with respect to the filtered image B1 every other pixel. The interpolation processing by the means 14 is substantially performed by two types of filters: a 2 × 1 filter (0.5, 0.5) and a 3 × 1 filter (0.1, 0.8, 0.1). Equivalent to filtering.

  When the interpolation means 14 performs interpolation on the filtered image B2, k = 2, so σ = 2. In the above equation (13), the filter corresponding to σ = 2 is an 11 × 1 one-dimensional filter F2 shown in FIG. The interpolation means 14 first expands the filtered image B2 to the same size as the image S0 by interpolating three pixels each having a value of 0 every other pixel with respect to the filtered image B2, and then expands the image into an enlarged image. On the other hand, a filtering process is performed by the filter F2 shown in FIG. 8 to obtain a blurred image S2. The blurred image S2 has the same number of pixels as the image S0, that is, the same size as the image S0.

  Similarly, the filter F2 shown in FIG. 8 is an 11 × 1 filter, but before applying the filter F2, three pixels each having a value of 0 are interpolated with respect to the filtered image B2 by three. Therefore, the interpolation processing by the interpolation means 14 is substantially performed by a 2 × 1 filter (0.5, 0.5) and a 3 × 1 filter (0.3, 0.65, 0.05), ( This is equivalent to filter processing using four types of filters (0.3, 0.74, 0.13) and (0.05, 0.65, 0.3).

In this way, the interpolation means 14 interpolates (2 ^K −1) pixels each having a value of 0 every other pixel for each filtering image Bk, so that the filtering image Bk has the same size as the image S0. And filtering by a filter having a length of (3 × 2 ^K −1) created based on the above equation (13) with respect to the filtered image Bk interpolated with pixels having a value of 0 Processing is performed to obtain a blurred image Sk.

  FIG. 9 shows the frequency characteristics of each blurred image Sk obtained by the blurred image creating means 10 when n = 3 as an example. As shown in the figure, the blurred image Sk is such that the higher the k, the higher the frequency component of the image S0 is removed.

  The band limited image creating means 20 uses the blurred images S1, S2,... Sn obtained by the blurred image creating means 10 to calculate components for a plurality of frequency bands of the image S0 according to the following equation (14). Band-limited images T1, T2,.

Tm = S (m−1) −Sm (14)
Where m is an integer between 1 and n

FIG. 10 shows the frequency characteristics of each band limited image Tm obtained by the band limited image creating means 20 when n = 3 as an example. As illustrated, the band limited image Tm represents a component in the low frequency region of the image S0 as m increases.

  In this way, the blurred image creating unit 10 and the band limited image creating unit 20 create band limited images T1, T2,... Tn for each luminance Y, color difference Cr, and color difference Cb from the image S0.

  The threshold value setting means 30 calculates a threshold value λ for discriminating between edges and noise in the image S0. The threshold λ may be obtained for each component of the image S0. However, in this embodiment, since the luminance component contributes greatly to human vision, noise correction is performed only on the luminance component. Therefore, the threshold value setting means 30 obtains the threshold value λ of the luminance component. FIG. 11 is a block diagram showing the configuration of the threshold setting means 30. As shown in FIG.

  As shown in FIG. 11, the threshold setting means 30 is a band-limited image for analyzing the noise level according to the enlargement / reduction ratio α of the image D0 obtained from the image input means 1 (here, the band limit of the color difference component). Image) Obtained by band determining means 31 for determining Tpb, Tpr, color difference noise level calculating means 35 for calculating color difference noise levels σCr, σCb using the determined band limited image, and color difference noise level calculating means 35 The luminance noise level estimator 38 for estimating the luminance noise level σY using the color difference noise level, and the luminance noise level σY obtained by the luminance noise level estimator 38 are adjusted to obtain a noise level λ for each luminance. And adjusting means 39 for obtaining (Y).

  The band determining means 31 determines a band-limited image for calculating the noise level so that the frequency band in the selected frequency region is lowered as the expansion / contraction ratio α increases as much as possible. FIG. 12 shows an example of a mode in which the band determining unit 31 in the present embodiment determines a band limited image. In the example shown in the figure, for each scaling factor α that is 0.75 or less and greater than 0.75 to 1.25 and 1.25, the band determining means 31 selects noise T1, T2, and T3, respectively. Band-limited images Tpb and Tpr for calculating the level are determined. The band determining unit 31 outputs the band limited images Tpb and Tpr to the color difference noise level calculating unit 35, and calculates the color difference noise level for the band limited image TpY of the luminance component in the frequency band corresponding to the band limited images Tpb and Tpr. It outputs to the means 35.

  The color difference noise level calculation means 35 obtains the color difference noise levels σCb and σCr using the band limited images Tpb and Tpr determined by the band determination means 31. FIG. 13 is a block diagram showing the configuration of the color difference noise level calculation means 35. As shown in the figure, the color difference noise level calculation means 35 includes a luminance edge pixel removal means 32, a component removal means 34 that seems to be a color difference edge, and a standard deviation calculation means 36. The luminance edge pixel removing unit 32 selects pixels having a luminance value equal to or higher than a predetermined threshold based on the luminance value of each pixel of the luminance component band limited image TpY output together with the color difference component band limited images Tpb and Tpr. As luminance edge pixels, pixels corresponding to pixels that are likely to be luminance edges are removed from the band-limited images Tpb and Tpr of the color difference component. Note that the “threshold value” here corresponds to a magnitude that is impossible as the amplitude of noise, and is a sufficiently small value to remove pixels that can only be considered as edges.

  Next, the component removal unit 34 that seems to be a color difference edge removes a component that seems to be a color edge from the band limited images Tpb and Tpr after the luminance edge pixel is removed. Each unit of the color difference noise level calculation unit 35 performs the same operation with respect to Tpb and Tpr. Therefore, the band-limited images Tpb and Tpr are not particularly distinguished below and will be described as Tp.

  The component removal means 34 that seems to be a color difference edge creates a histogram of the absolute value of the color difference for the band limited image Tp from which the luminance edge pixels have been removed, and a predetermined percentage (for example, 80%, 90%, etc.) in this histogram. Color difference values having a greater frequency are removed as components that are likely to be color difference edges.

  The standard deviation calculation means 36 calculates the standard deviation of the color difference using the color difference value after the color difference edge-like component is removed by the color difference edge-like component removal means 34 in the color difference absolute value histogram and calculates the color difference noise level σCr. , ΣCb is obtained. Here, a value obtained by correcting the standard deviation of the calculated color difference according to the percentage used in the component removing unit 34 that seems to be a color difference edge may be used as the color difference noise level. This makes it possible to obtain an appropriate noise level even if the color difference value removed as a color difference edge-like component is a noise color difference value. Specifically, assuming that the histogram has a complete normal distribution, the standard deviation calculated using color difference values having a frequency of 0% to 90% is an abbreviation of the standard deviation calculated using all color difference values. For example, when a color difference value having a frequency of 90% or more is removed as a component that seems to be a color difference edge, the standard deviation calculated using a color difference value other than the component that seems to be a color difference edge is 0. 9 times. What is necessary is just to correct | amend so that 1.27 which becomes the reciprocal number of 79 may be multiplied. Similarly, when 80% or 70% is used as the threshold value, the standard deviation may be multiplied by 1.51 or 1.78, respectively.

  In this way, the color difference noise level calculation unit 35 acquires the color difference noise levels σCr and σCb of the entire image S0.

  The luminance noise level estimation unit 38 estimates the luminance noise level of the entire image S0 from the color difference noise levels σCr and σCb obtained by the color difference noise level calculation unit 35. In this embodiment, since YCrCb is used as the luminance / color difference space, the luminance noise level estimation means 38 estimates the luminance noise level σY from the color difference noise levels σCr, σCb according to the above-described equation (7).

σY≈ (1.02 × σCr + 1.07 × σCb) / 2 (7)
Where σY: Luminance noise level
σCr, σCb: Color difference noise level

The adjusting means 39 adjusts the exposure correction amount H and the brightness level (here, the brightness noise level σY (luminance noise level of the entire image) obtained by the brightness noise level estimating means 38 from the image input means 1. Then, the noise level for each luminance value is acquired by performing adjustment according to the luminance value in the band limited image TpY, and is set as the threshold value λ (Y) for each brightness level of the image S0.

  As described above, the gain received by the noise amplitude in the image by the exposure correction varies depending on the magnitude of the QL value. In addition, when the exposure correction amount H is different, the gain curve indicating the gain received by the noise amplitude is also different. FIG. 14 is a gain curve showing the gain received by the amplitude of noise when the density is subtracted from 0.5 and 0.7 as exposure correction, and the dotted line in the figure is when the density is subtracted from 0.5. The solid curve in the figure is the gain curve when the density of 0.7 is subtracted.

  The adjustment unit 39 selects a gain curve corresponding to the correction amount H obtained from the image input unit 1 from gain curves that differ for each correction amount H as shown in FIG. 14, and uses the selected gain curve. Then, the luminance noise level σY obtained by the luminance noise level estimating means 38 is adjusted to obtain a noise level λ (Y) for each luminance value, and the noise level λ (Y) for each luminance is obtained as an edge and noise in the image S0. Is output to the noise correction means 40 as a threshold value for discriminating. 14 shows the gain received by the noise amplitude for each QL value, the gain received by the noise amplitude for each luminance value may be converted from the correspondence between the QL value and the luminance value. .

  The adjusting means 39 adjusts the luminance noise level σY using a gain curve as shown in FIG. 14. As a specific method, for example, as shown in FIG. 15, the noise level σY is adjusted in the image. No adjustment is made corresponding to the representative QL value (the adjustment multiple is 1), and the ratio between the gain received by the QL value other than the representative QL value and the gain received by the representative QL value corresponds to the corresponding QL value. Adjusted multiples. As shown in FIG. 15, such adjustment lowers the threshold value (the same as the noise level in the example of the present embodiment) of a brighter part from the noise level (here, luminance noise level σY) of the entire image, The darker part increases the threshold.

  The representative QL value can be an intermediate value of the dynamic range of the luminance Y of the image or a 50% point of the cumulative histogram.

  The threshold setting unit 30 outputs λ (Y) obtained in this way to the noise correction unit 40 as a threshold for each luminance value for discriminating between edges and noise in the luminance component of the image S0.

  The noise correction unit 40 performs noise correction processing on the luminance component of the image S0 using the threshold value λ (Y) for each luminance value obtained by the threshold setting unit 30. The noise correction processing here is not limited to the methods described in Patent Document 1 and Patent Document 2, but uses a threshold value for discriminating between edges and noise, suppresses a component having an amplitude smaller than the threshold value as a noise component, and / or Alternatively, any noise correction processing method that emphasizes a component having an amplitude equal to or greater than a threshold as an edge component may be used. When noise correction processing is performed by these methods, a threshold value for each brightness level (in this embodiment, for each luminance value) may be applied.

  As mentioned above, although preferred embodiment of this invention was described, this invention is not limited to embodiment mentioned above, Unless it deviates from the main point of this invention, various increase / decrease and change can be performed.

  For example, in this embodiment, since the luminance component contributes the most to human vision, noise correction processing is performed only on the luminance component, but the threshold value for each channel of luminance and color difference is obtained. The noise correction process may be performed on each channel.

  Also, the method for creating a blurred image and the method for creating a band limited image are not limited to the methods used in the present embodiment, and various conventional methods may be applied.

  Also, not only for images that have undergone exposure correction processing, but also for images that have undergone processing that changes the brightness of the image, such as gamma correction and hypertone, the gain of the amplitude of noise in the image received by that processing The threshold value for each brightness can be obtained by adjusting the noise level of the entire image or the threshold value for the entire image according to the mode.

  Also, whether the image has been subjected to brightness change processing, what brightness change processing has been performed, how much the brightness has changed (the correction amount of the brightness change processing), etc. If it cannot be grasped, the brightness level of the image is adjusted so that the lighter part becomes smaller with respect to the threshold for the whole image or the noise level of the whole image according to the distribution tendency of the noise level in the image. A threshold value may be obtained.

1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. 1 is a block diagram showing the configuration of a blurred image creation unit 10 in the image processing apparatus shown in FIG. The figure which shows the example of the one-dimensional filter F which the filtering means 12 in the blur image creation means 10 shown in FIG. 2 uses The figure which shows the process performed in the blurred image preparation means 10 shown in FIG. The figure which shows the frequency characteristic of the filtering image Bk produced by the filtering means 12 in the blur image creation means 10 shown in FIG. The figure which shows the example of the two-dimensional filter which the filtering means 12 in the blur image creation means 10 shown in FIG. 2 uses The figure which shows the example of the filter F1 which the interpolation means 14 in the blur image creation means 10 shown in FIG. 2 uses for interpolation of the filtering image B1 The figure which shows the example of the filter F2 which the interpolation means 14 in the blur image creation means 10 shown in FIG. 2 uses for interpolation of filtering image B2 The figure which shows the frequency characteristic of the blur image Sk created by the blur image creation means 10 shown in FIG. The figure which shows the frequency characteristic of the band limited image Tk produced by the band limited image preparation means 20 in the image processing apparatus of embodiment of this invention shown in FIG. 1 is a block diagram showing the configuration of threshold setting means 30 in the image processing apparatus according to the embodiment of the present invention shown in FIG. The figure for demonstrating the band determination means 31 in the threshold value setting means 30 shown in FIG. FIG. 11 is a block diagram showing the configuration of the color difference noise level calculation means 35 in the threshold setting means 30 shown in FIG. The figure which shows the gain which the amplitude of the noise received in the density-corrected image The figure for demonstrating the process by the adjustment means 39 in the threshold value setting means 30 shown in FIG. The figure which shows the gain which QL value received by exposure correction Diagram showing the relationship between concentration and QL value

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image input means 5 YCC conversion means 10 Blurred image creation means 12 Filtering means 14 Interpolation means 16 Control means 20 Band-limited image creation means 30 Threshold setting means 31 Band determination means 32 Luminance edge pixel removal means 34 Component removal means that seems to be color difference edge 35 Color difference noise level calculation means 36 Standard deviation calculation means 38 Luminance noise level estimation means 39 Adjustment means

Claims (3)

In an image processing method for obtaining a threshold value for discriminating edges and noise in a digital image from a digital image that has been subjected to exposure correction processing for changing the brightness of the original image ,
Calculating a noise variance value of the entire digital image from the frequency component of the digital image;
For each brightness level of the digital image , adjustment of the noise variance value of the entire digital image according to the brightness level, and the entire digital image using a gain curve that differs for each correction amount in exposure correction processing Obtaining the noise variance value for each brightness level of the digital image by adjusting the noise variance value of
Set the noise variance value for each adjusted brightness level as a threshold,
An image processing method, wherein a frequency component of the digital image that is smaller than the threshold is suppressed as a noise component. An image processing apparatus that obtains a threshold value for discriminating edges and noise in a digital image from a digital image that has been subjected to exposure correction processing for changing the brightness of the original image ,
An overall noise level calculating means for calculating a noise variance value of the entire digital image from the frequency component of the digital image;
For each brightness level of the digital image , adjustment of the noise variance value of the entire digital image according to the brightness level, and the entire digital image using a gain curve that differs for each correction amount in exposure correction processing Adjusting the noise variance value to obtain the noise variance value for each level of brightness of the digital image;
Threshold acquisition means for setting a noise variance value adjusted for each brightness level as a threshold, and noise correction means for suppressing a component smaller than the threshold set by the threshold acquisition means as a noise component. A featured image processing apparatus. A program for causing a computer to execute processing for obtaining a threshold value for discriminating edges and noise in a digital image from a digital image that has been subjected to exposure correction processing for changing the brightness of the original image ,
For each brightness level of the digital image , adjustment of the noise variance value of the entire digital image according to the brightness level, and the entire digital image using a gain curve that differs for each correction amount in exposure correction processing Processing for obtaining the noise variance value for each brightness level of the digital image by adjusting the noise variance value of
A process for setting the noise variance value for each adjusted brightness level as a threshold;
A program for executing a process of suppressing a component smaller than the set threshold as a noise component .

Images