JP2018174470A - Noise reduction device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce noise with high accuracy, by considering the feature of an amplifier built in each pixel of image pick-up device.SOLUTION: A noise reduction device 1 includes a wavelet packet disassembly section 11 for generating frequency resolution coefficients by performing wavelet packet disassembly processing for an input image, a clustering section 12 for generating multiple clusters for the frequency resolution coefficients in a low frequency band, and determining the cluster phase position, a noise level calculation section 13 for calculating noise level from the values of frequency resolution coefficients at a cluster phase position, out of the frequency resolution coefficients in a high frequency band for each cluster, a degeneration processing section 14 for generating degeneration coefficients, where the frequency resolution coefficients below noise level are degenerated stronger than the frequency resolution coefficients exceeding noise level, for the frequency resolution coefficients at the cluster phase position, for each cluster, and a wavelet packet reconstitution section 15 for generating noise reduction image by wavelet packet reconstitution of the degeneration coefficients.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a noise removal apparatus and program for removing noise from an input image.

  Conventionally, a method has been proposed in which a frequency decomposition coefficient is generated by wavelet transform of an image, and noise is removed by coring a component having a noise level or less among the frequency decomposition coefficient (for example, Non-Patent Document 1). reference). Further, in noise removal using wavelet degeneracy, the spatial high-frequency band component is almost a noise component, so the noise level is obtained from the dispersion value of the power value in the spatial high-frequency band.

David L. Donoho, Iain M. Johnstone, Gerard Kerkyacharian and Dominique Picard, “Wavelet Shrinkage: Asymptopia?”, Journal of the Royal Statistical Society. Series B (Methodological), Vol. 57, No. 2, pp. 301-369 , 1995.

  In recent digital cameras and video cameras, a CMOS sensor is used as an image sensor. The CMOS sensor has an amplifier for each pixel, and amplifies the photoelectrically converted charge amount. Since the signal-to-noise ratio is lower in the dark pixel than in the bright pixel, the noise level is often higher in the dark pixel than in the bright pixel because the amplification characteristics of the amplifier are not linear. For this reason, it is desirable that the amount of degeneration at the time of noise removal be larger for dark pixels than for bright pixels. However, conventionally, such noise level characteristics have not been taken into consideration when removing noise.

  An object of the present invention made in view of such circumstances is to provide a noise removal device and a program capable of removing noise with high accuracy in consideration of the characteristics of an amplifier incorporated in each pixel of an image sensor. It is in.

  In order to solve the above problems, a noise removal device according to the present invention is a noise removal device that removes noise from an input image, and performs wavelet packet decomposition processing on the input image to generate a frequency decomposition coefficient. For each of the clusters, a decomposing unit, a clustering unit that generates a plurality of clusters by clustering in order of magnitude for the frequency resolving coefficients in a low frequency band, and determines a cluster phase position that is a phase position for each cluster A noise level calculation unit that calculates a noise level from a value of a frequency resolution coefficient at the same phase position as the cluster phase position among the frequency resolution coefficients in a high frequency band; and for each cluster, the same as the cluster phase position For the frequency resolution coefficient of the phase position, with the noise level as a threshold, A degeneration processing unit that generates a degeneration coefficient that is generated by degenerating the frequency decomposition coefficient more strongly than the frequency decomposition coefficient that exceeds the noise level, and a wavelet packet reconfiguration that reconstructs the degeneration coefficient as a wavelet packet and generates a noise-removed image. And a constituent part.

  Furthermore, in the noise removal device according to the present invention, the clustering unit generates the cluster for the frequency resolution coefficient in the lowest frequency band for each of the four adjacent frequency bands divided by the quadtree, and the noise level The calculation unit calculates the noise level from the value of the frequency resolution coefficient at the same phase position as the cluster phase position among the frequency resolution coefficients in the highest frequency band for each of the four frequency bands. .

  Furthermore, in the noise removal apparatus according to the present invention, the degeneration processing unit degenerates the frequency decomposition coefficient having a lower frequency band more strongly in the same cluster.

  Furthermore, in the noise removal device according to the present invention, the noise level calculation unit uses the root mean square value or the average value of the frequency resolution coefficients at the same phase position as the cluster phase position as the noise level. To do.

  In order to solve the above problems, a program according to the present invention causes a computer to function as the noise removal device.

  According to the present invention, the noise removal amount (degeneration rate) can be controlled for each phase position and each power level in consideration of the characteristics of the amplifier incorporated in each pixel of the image sensor, and the noise removal of the image can be performed with high accuracy. Can be done.

It is a block diagram which shows the structural example of the noise removal apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows a mode that the 2nd floor wavelet packet decomposition | disassembly of the input image was carried out. It is a figure which shows the 1st example of the frequency band which performs a clustering process, and the frequency band which performs a noise level calculation process. It is a figure which shows the 2nd example of the frequency band which performs a clustering process, and the frequency band which performs a noise level calculation process. It is a figure which shows the example of the degeneracy function used with the noise removal apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the noise removal apparatus which concerns on the 2nd Embodiment of this invention. It is a figure explaining the process in the case of correct | amending a noise level (and maximum noise level) in the noise removal apparatus which concerns on the 2nd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
The noise removal apparatus according to the first embodiment of the present invention will be described below. FIG. 1 shows a configuration example of a noise removal apparatus according to the first embodiment of the present invention. The noise removal apparatus 1 shown in FIG. 1 includes a wavelet packet decomposition unit 11, a clustering unit 12, a noise level calculation unit 13, a degeneration processing unit 14, and a wavelet packet reconstruction unit 15.

  The noise removal device 1 is a device that removes noise from an image (input image) input from an image sensor and outputs the noise-removed image to the outside.

  The wavelet packet decomposition unit 11 performs a wavelet packet decomposition process that performs octave decomposition on the low frequency band side and the high frequency band side of the input image to generate a frequency decomposition coefficient (frequency decomposition component). The clustering unit 12 and the degeneration processing unit 14 for output. It is assumed that the Parseval equation holds between the frequency decomposition coefficients of each floor in wavelet packet decomposition.

  FIG. 2 is a diagram illustrating how frequency decomposition is performed when an input image is subjected to second-order wavelet packet decomposition by the wavelet packet decomposition unit 11. When an input image having a resolution of 8K horizontal and vertical 4K is subjected to second-order wavelet packet decomposition, as shown in FIG. 2, the frequency band of the input image is divided into 16, and the wavelet packet decomposition unit 11 performs frequency decomposition within each frequency band. Output coefficients. In the following description, the spatial frequency of the image is represented by a resolution corresponding to the frequency. In the following description, “phase position” refers to the coordinate position (x, y) of the frequency resolution coefficient within each divided frequency band. That is, in the example of FIG. 2, 0 ≦ x <2K and 0 ≦ y <1K, and there are 16 frequency resolution coefficients at the same phase position.

The clustering unit 12 generates a plurality of clusters by clustering the frequency resolution coefficients in the low frequency band in order of magnitude in order to enable noise control according to the luminance of the input image. Hereinafter, in this embodiment, it is assumed that the number of clusters is 4, and clusters C ₁ , C ₂ , C ₃ , and C ₄ are generated. As the number of clusters increases, finer control is possible according to the luminance level of the input image. However, even if the number of clusters is about 3 to 5, noise removal considering the characteristics of the amplifier of the image sensor is sufficiently possible in experiments. I know.

For example, when a 12-bit depth input image is clustered into four clusters, the clustering unit 12 sets the effective level of 4064 gradations (0 to 15 and 4080 to 4095 gradations) of an SDI (Serial Digital Interface) signal. Dividing into four equal parts, the cluster C ₁ is composed of 16 to 1031 gradations, the cluster C ₂ is composed of 1032 to 2047 gradations, the cluster C ₃ is composed of 2048 to 3063 gradations, and the cluster C ₄ is composed of 3064 to 4079 gradations. Alternatively, 256 gray levels of 0% -black and 3760 gray levels of 100% -white are equally divided into four, and 16 to 255 gray levels are in cluster C ₁ and 3761 to 4079 gray levels are in cluster C ₄ . It may be assigned. In addition, clustering may be performed using a k-means method or the like. With this clustering process, each frequency resolution coefficient in the low frequency band is classified into _one of the clusters C _{1 to} C ₄ .

  Then, the clustering unit 12 determines a phase position (cluster phase position) for each cluster with respect to the frequency resolution coefficient in the low frequency band, and outputs it to the noise level calculation unit 13.

  The noise level calculation unit 13 calculates a noise level from the value of the frequency decomposition coefficient at the same phase position as the cluster phase position among the frequency decomposition coefficients in the high frequency band for each cluster, and outputs the noise level to the degeneration processing unit 14. The noise level is a root mean square (RMS) value or an average value of the frequency resolution coefficient value. Since the high frequency band has a small frequency resolution coefficient value (the signal level is small), the ratio of noise is high, which is useful for estimating the noise level.

FIG. 3 is a diagram illustrating a first example of a frequency band for performing clustering processing and a frequency band for performing noise level calculation processing. In the example illustrated in FIG. 3, the clustering unit 12 clusters the frequency resolution coefficients in the lowest frequency band LLLL indicated by diagonal lines into clusters C _{1 to} C _4, and the phase positions of the frequency resolution coefficients belonging to the cluster C ₁ (cluster C ₁ phase position), the phase position of the frequency decomposition coefficients belonging to the cluster C ₂ (cluster C ₂ phase position), the phase position of the frequency decomposition coefficients belonging to the cluster C ₃ (cluster C ₃ phase position), and frequency resolution which belongs to the cluster C ₄ The phase position of the coefficient (cluster C ₄ phase position) is determined.

Then, the noise level calculation unit 13, among the frequency decomposition coefficients in the highest frequency band HHHH indicated by vertical lines, and calculates the noise level N ₁ from the value of the frequency decomposition coefficients of the same phase position as the cluster C ₁ phase position. Similarly, noise levels N ₂ , N ₃ , and N ₄ are respectively calculated from the values of the frequency resolution coefficients at the same phase position as the cluster C ₂ phase position, the cluster C ₃ phase position, and the cluster C ₄ phase position. Further, the noise level calculation unit 13 may set the maximum value of the frequency resolution coefficient value at the same phase position as the phase position for each cluster (cluster C _n phase position) as the maximum noise level N _n _max. Note that n is a symbol for distinguishing clusters, and in this embodiment, n = 1 to 4.

FIG. 4 is a diagram illustrating a second example of a frequency band for performing clustering processing and a frequency band for performing noise level calculation processing. As shown in FIG. 4, the clustering unit 12 performs frequency decomposition coefficients in the lowest frequency band for each of the four adjacent frequency bands that are divided into quadtrees (hereinafter referred to as “quadtree-divided frequency band group”). The phase position for each cluster (cluster C _n phase position) may be determined. Here, the quadtree division frequency band group means four frequency bands surrounded by a thick frame shown in FIG. That is, the first quadtree split frequency band group is composed of frequency bands LLLL, LLLH, LLHL, and LLHH, and the second quadtree split frequency band group is composed of frequency bands LHLL, LHLH, LHHL, and LHHH, and the third The quad-tree split frequency band group consists of frequency bands HLLL, HLLH, HLHL, and HLHH, and the fourth quad-tree split frequency band group consists of frequency bands HHLL, HHLH, HHHL, and HHHH.

In the example illustrated in FIG. 4, the clustering unit 12 uses the phase position (cluster) for each cluster for the frequency resolution coefficients in the lowest frequency bands LLLL, LHLL, HLLL, and HHLL in each quadtree divided frequency band group indicated by diagonal lines. determining C _n phase position).

For the first quadtree division frequency band group, the noise level calculation unit 13 uses the phase for each cluster in the lowest frequency band LLLL indicated by diagonal lines among the frequency resolution coefficients in the highest frequency band LLHH indicated by vertical lines in FIG. The noise level N _n is calculated from the value of the frequency decomposition coefficient at the same phase position as the position (cluster C _n phase position). For the second quadtree split frequency band group, among the frequency resolution coefficients in the highest frequency band LHHH indicated by vertical lines in FIG. 4, the phase position (cluster C _n) for each cluster in the lowest frequency band LHLL indicated by diagonal lines. The noise level N _n is calculated from the value of the frequency resolution coefficient at the same phase position as the (phase position). For the third quadtree split frequency band group, among the frequency resolution coefficients in the highest frequency band HLHH indicated by vertical lines in FIG. 4, the phase position (cluster C _n) for each cluster in the lowest frequency band HLLL indicated by diagonal lines. The noise level N _n is calculated from the value of the frequency resolution coefficient at the same phase position as the (phase position). For the fourth quadtree split frequency band group, among the frequency resolution coefficients in the highest frequency band HHHH indicated by vertical lines in FIG. 4, the phase position (cluster C _n) for each cluster in the lowest frequency band HHLL indicated by diagonal lines. The noise level N _n is calculated from the value of the frequency resolution coefficient at the same phase position as the (phase position).

The degeneration processing unit 14 uses the noise level N _n calculated by the noise level calculation unit 13 as a threshold and degenerates the frequency decomposition coefficient _{equal to} or lower than the noise level N _n more strongly than the frequency decomposition coefficient exceeding the noise level N _n. A coefficient is generated and output to the wavelet packet reconstruction unit 15.

In the example illustrated in FIG. 3, the degeneration processing unit 14 degenerates the frequency resolution coefficient at the same phase position as the cluster C _n phase position with the noise level N _n as a threshold value for all frequency bands. In the example shown in FIG. 4, the frequency resolution coefficient at the same phase position as the cluster C _n phase position is degenerated using the noise level N _n as a threshold for each quadtree division frequency band group.

The degeneration processing unit 14 may perform the degeneration process using a degeneration function that represents an output value with respect to an input value of the frequency decomposition coefficient. FIG. 5 shows an example of the degeneration function. When the degeneration function of FIG. 5A is used, the frequency decomposition coefficient whose absolute value is lower than the noise level N _n is degenerated to 0, and the frequency decomposition coefficient whose absolute value exceeds the noise level N _n is output as it is without degeneration. The

In addition, processing when the degeneration processing unit 14 inputs the maximum noise level N _n _max in addition to the noise level N _n from the noise level calculation unit 13 will be described. In this case, erosion process unit 14, among frequency decomposition coefficients, the first frequency decomposition coefficients absolute value is less than or equal to the noise level N _n, and the largest absolute value noise level N _n _max less beyond the noise level N _n The second frequency resolution coefficient is degenerated so that the absolute value becomes smaller, and the degeneration rate of the absolute value of the first frequency resolution coefficient is made larger than the degeneration rate of the absolute value of the second frequency resolution coefficient. .

When using degenerate function of FIG. 5 (b), the absolute value the absolute value of the following frequency decomposition coefficients N _n are converted to 0, the maximum noise level N _n _max following absolute value exceeds the noise level N _n The absolute value of the frequency resolution coefficient is converted to a value larger than 0 and smaller than the original value.

  The wavelet packet reconstruction unit 15 performs wavelet packet reconstruction processing on the degeneration coefficient generated by the degeneration processing unit 14 to generate a noise-removed image, and outputs it to the outside. If the wavelet packet decomposition unit 11 has performed wavelet packet decomposition processing with the k-th floor, the wavelet packet reconstruction unit 15 performs wavelet packet reconstruction processing with the k-th floor.

  Note that a computer can be suitably used to function as the noise removal device 1 described above, and such a computer stores a program describing processing contents for realizing each function of the noise removal device 1. This program can be realized by reading out and executing this program by the CPU of the computer.

As described above, in the first embodiment, wavelet packet decomposition processing is performed on the input image to generate frequency decomposition coefficients, and a plurality of clusters are obtained by clustering the frequency decomposition coefficients in the low frequency band in order of magnitude. Generate and determine the cluster phase position. Then, for each cluster, a noise level N _n is calculated from the value of the frequency resolution coefficient at the same phase position as the cluster phase position among the frequency resolution coefficients in the high frequency band, and the frequency resolution coefficient at the same phase position as the cluster phase position Is generated by degenerating a frequency decomposition coefficient _{equal to} or lower than the noise level N _n more strongly than a frequency decomposition coefficient exceeding the noise level N _n . Finally, the degenerate coefficient is reconstructed as a wavelet packet to generate a noise-removed image. With this configuration, according to the present invention, it is possible to control the amount of noise removal (degeneration rate) for each phase position and for each power level in consideration of the characteristics of the amplifier incorporated in each pixel of the CMOS sensor. Noise in the input image can be removed with high accuracy.

(Second Embodiment)
Next, a noise removal apparatus according to the second embodiment of the present invention will be described. As noise superimposed on the camera system, there are thermal noise and shot noise. Thermal noise has white Gaussian characteristics, and the noise level is almost constant over the entire frequency band. On the other hand, shot noise becomes dominant in the spatial low frequency band, and the noise level is inversely proportional to the frequency f (becomes 1 / f). For this reason, the noise level due to thermal noise and shot noise has a certain degree of color, is high in the low frequency band, and is almost constant in the high frequency band above a certain value. Therefore, in the second embodiment, the lower the frequency band, the stronger the degeneration during noise removal.

  FIG. 6 is a block diagram illustrating a configuration example of a noise removal device according to the second embodiment. The noise removal apparatus 2 shown in FIG. 6 includes a wavelet packet decomposition unit 11, a clustering unit 12, a noise level calculation unit 13, a degeneration processing unit 14, a wavelet packet reconstruction unit 15, and a noise level correction unit 16. Prepare. The noise removal device 2 according to the second embodiment is different from the noise removal device 1 according to the first embodiment in that a noise level correction unit 16 is further provided. Since other configurations are the same as those of the first embodiment, the same reference numerals are given and description thereof is omitted as appropriate.

The noise level calculation unit 13 differs from the first embodiment only in the output destination, and outputs the calculated noise level N _n to the noise level correction unit 16.

The noise level correcting unit 16 corrects the noise level N _n calculated by the noise level calculating unit 13 by multiplying the noise level N _n by a larger coefficient in the lower frequency band, and the corrected noise level N _n ′ is supplied to the degeneration processing unit 14. Output. When using the maximum noise level N _n _max by degeneration processing unit 14, similarly with respect to the maximum noise level N _n _max, corrected by multiplying a greater coefficient as low frequency bands, corrected maximum noise level N _n ′ _max is output to the degeneration processing unit 14.

FIG. 7 is a diagram for describing processing in the case of correcting the noise level N _n (and the maximum noise level N _{n —} max). For example, in the frequency band (LLLL) indicated by diagonal lines in FIG. 7, the coefficient α = 1.5, in the frequency band group indicated by vertical lines (LLLLH, LLHL, LLHH), the coefficient α = 1.3, and the frequency band indicated by the horizontal line In the group (LHLL, HLLL, LHHL, HLLH, HHLL), the coefficient α = 1.1, and in the remaining frequency band groups (LHLH, HLHL, LHHH, HLHH, HHLH, HHHL, HHHH) indicated by dots, the coefficient α = 1. To do.

The degeneration processing of the degeneration processing unit 14 is the same as that of the first embodiment, and the noise level N _n ′ (and the maximum noise level N _n ′ _max) corrected by the noise level correction unit 16 is used as a threshold value, for example, a degeneration function is used. Use to degenerate frequency resolution coefficients. However, in the first embodiment, the values of the noise level N _n and the maximum noise level N _{n —} max are the same between the frequency bands in the same cluster, but in this embodiment, the noise level N _n ′ and the maximum The value of the noise level N _n '_max becomes larger as the frequency band is lower in the same cluster. Therefore, the degeneration processing unit 14 of the present embodiment degenerates more strongly in the same cluster as the frequency resolution coefficient having a lower frequency band.

  Note that a computer can be suitably used to function as the noise removal device 2 described above, and such a computer stores a program describing processing contents for realizing each function of the noise removal device 2. This program can be realized by reading out and executing this program by the CPU of the computer.

  As described above, in the second embodiment, the frequency decomposition coefficient having a lower frequency band is degenerated more strongly in the same cluster. Therefore, according to the present invention, in addition to the characteristics of the amplifier incorporated in each pixel of the CMOS sensor, the characteristics of thermal noise and shot noise can be considered, and noise can be removed with higher accuracy.

  Although the above embodiment has been described as a representative example, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, it is possible to combine a plurality of constituent blocks described in the configuration diagram of the embodiment into one, or to divide one constituent block.

  The program according to the present invention may be recorded on a computer readable medium. If a computer-readable medium is used, it can be installed on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be a recording medium such as a CD-ROM or a DVD-ROM.

DESCRIPTION OF SYMBOLS 1, 2 Noise removal apparatus 11 Wavelet packet decomposition part 12 Clustering part 13 Noise level calculation part 14 Degeneration process part 15 Wavelet packet reconstruction part 16 Noise level correction part

Claims (5)

A noise removal device for removing noise from an input image,
A wavelet packet decomposition unit that generates a frequency decomposition coefficient by performing wavelet packet decomposition processing on an input image;
A clustering unit that generates a plurality of clusters by clustering in order of magnitude for the frequency resolution coefficient in a low frequency band, and determines a cluster phase position that is a phase position for each cluster;
A noise level calculation unit that calculates a noise level from a value of a frequency resolution coefficient at the same phase position as the cluster phase position among the frequency resolution coefficients in a high frequency band for each cluster;
For each of the clusters, with respect to the frequency resolution coefficient at the same phase position as the cluster phase position, the frequency resolution coefficient equal to or lower than the noise level is degenerated more strongly than the frequency resolution coefficient exceeding the noise level with the noise level as a threshold. A degeneration processing unit that generates a degeneration coefficient;
A wavelet packet reconstruction unit that reconstructs the degeneration coefficient into a wavelet packet to generate a noise-removed image;
A noise removal apparatus comprising: The clustering unit generates the cluster for the frequency decomposition coefficient in the lowest frequency band for every four adjacent frequency bands that are divided into quadtrees,
The noise level calculation unit calculates the noise level from the value of the frequency resolution coefficient at the same phase position as the cluster phase position among the frequency resolution coefficients in the highest frequency band for each of the four frequency bands. The noise removal device according to claim 1, wherein   The noise reduction apparatus according to claim 1, wherein the degeneration processing unit degenerates the frequency decomposition coefficient having a lower frequency band more strongly in the same cluster.   4. The noise level calculation unit according to claim 1, wherein the noise level is a root mean square value or an average value of frequency decomposition coefficient values at the same phase position as the cluster phase position. 5. The noise removing device according to item. The program for functioning a computer as a noise removal apparatus as described in any one of Claim 1 to 4.

Images