JP4478981B2 - Color noise reduction method and color imaging apparatus - Google Patents

Description

  The present invention relates to a method and apparatus for reducing color signal noise from a color image signal without causing color blur, and more particularly to a color signal noise reduction method for a color imaging apparatus.

In recent years, color solid-state imaging devices such as digital still cameras have been remarkably miniaturized. To this end, downsizing of the solid-state image sensor greatly contributes. The downsizing of this solid-state image sensor has realized the miniaturization of pixels while maintaining the sensitivity by improving the microlens and improving the condensing rate and the charge-voltage conversion rate by the progress of semiconductor microfabrication technology. .

However, in the sensitivity improvement that relies on the charge-voltage conversion rate, the number of signal charges itself has decreased, so the noise of photodiode leakage current called shot noise or dark current, which is quantum noise, increases. The S / N of the solid-state imaging device is lowered, and particularly, the increase in color noise is one of the main causes of image quality degradation because the glossiness of the image is impaired.

To compensate for this reduction in S / N, recent color video cameras use frame memory to control the noise by controlling afterimages in areas where the input image signal is stationary and weakly in areas where there is motion. Some are equipped with a cyclic noise reduction circuit that reduces noise. This noise reduction method is effective for noise that fluctuates randomly, such as light shot noise, which is a fluctuation in the amount of incident light, but it does not reduce fixed pattern noise such as variations in dark current. It cannot be used for a still camera that can obtain only one image signal.

As a technique for reducing noise with only one image signal, there is a noise reduction technique using the region growing method of Patent Document 1. This is because the signal value of the target pixel starts from the target pixel and the neighboring pixel signal value is sequentially expanded to a region including pixels within a predetermined level difference to obtain the neighboring region. Is the signal value from which the noise of the pixel of interest has been removed. By repeating this operation for each pixel, image noise can be removed. In addition, since pixels having a large signal level difference are not included in the calculation of the average value, color bleeding at the image edge portion does not occur.
U.S. Pat. No. 6,731,806

In cyclic noise reduction using a frame memory, there is a problem that fixed pattern noise and single image noise cannot be reduced. In addition, the noise reduction method using the region growing method has a problem in that it requires a huge amount of calculation because neighboring regions having close signal values for each pixel are determined and an average value is calculated.

In order to solve the above-described problem, in the present invention, a gradient signal is generated by taking an absolute value of a value obtained by first-order differentiation of a brightness signal included in an input color image signal. Pixels with the generated gradient signal value larger than the gradient signal value of half or more of adjacent pixels are extracted as region boundary pixels. A plurality of color difference signals included in the input color image signal are subjected to a two-dimensional smoothing process divided by the region boundary pixels to generate smoothed color difference signals. The chrominance signal value of the input image and the corresponding generated smoothed chrominance signal value are compared for each pixel, and the smoothed chrominance signal is mainly displayed when close, and the chrominance signal of the input image is mainly displayed when separated. To produce an output color difference signal.

Since the color difference signal is generated by subjecting the color difference signal to two-dimensional smoothing divided at the region boundary pixels, the smoothed color difference signal does not cause color blur at the image edge portion. Therefore, smoothing can be performed using a wide range of color difference signals, and a smoothed color difference signal with little fluctuation can be generated. By comparing the color difference signal value of the input image corresponding to the smoothed color difference signal value for each pixel and changing the composition ratio, the smoothed color difference signal is mainly used when the signal value recognized as a difference due to color noise is close. Is used to suppress color noise. When the noise of the input image is large, the region boundary pixels are divided due to the noise, and even if color blurring occurs in a part of the smoothed color difference signal, the difference from the color difference signal value of the input image is mainly large. Since the color difference signal is output, color bleeding does not occur.

In addition, when used for a color video camera, since it takes time to generate a smoothed color difference signal, the color difference signal and the smoothed color difference signal included in the input image are separate frames or field images, so that the motion in the image is not changed. There is a large difference between the chrominance signal value and the smoothed chrominance signal value in a certain part, and the color difference signal of the input image is mainly output, so there is no time delay of the display color, and the chrominance signal value and smoothing in the still part of the image Since the difference between the color difference signal values is small and the color difference signal smoothed in the image is mainly output, the color noise of the fixed pattern can be suppressed.

Since the smoothed color difference signal is obtained simply by obtaining a region boundary with a ridge line of the absolute value of the value obtained by first-order differentiation of the brightness signal and performing a two-dimensional smoothing process divided at the region boundary, the amount of calculation is small. Further, since the smoothed color difference signal is a narrow band signal, there is no problem even if the smoothed color difference signal is calculated after converting the input color image to an image with a small number of thinned pixels, and the amount of calculation can be further reduced.

In the present invention, a gradient signal is generated by taking an absolute value of a value obtained by first-order differentiation of a brightness signal included in an input color image signal. As the brightness signal, when the input color image is composed of a luminance signal and a color difference signal, the luminance signal is used, and when the input color image is an R (red), G (green), and B (green) primary color signal. May be a luminance signal or G signal synthesized by weighted addition. Further, since the smoothed color difference signal is a narrowband signal, an image of a brightness signal with a small number of pixels obtained by thinning out the input color image may be created. As the absolute value of the first-order differentiated value, the square root of the sum of squares of the difference signal of pixels adjacent in the vertical and horizontal directions is isotropic and desirable. There is no problem with the sum of the absolute values of the signals. By selecting a pixel whose generated gradient signal value is larger than the gradient signal value of half or more of adjacent pixels, a pixel whose continuous brightness signal changes suddenly is selected, and the boundary of the region in the image can be determined.

Next, a two-dimensional smoothing process in which a plurality of color difference signals included in the input color image signal are divided by the region boundary pixels is performed. When the input color image is composed of luminance signals and color difference signals, the color difference signal is used as it is. When the input color image is the RGB primary color signal, the BY / R / Y and B / G / RG signals are used. The color difference signal such as is generated and is taken into the image memory. The two-dimensional smoothing process repeats the image memory scan several times in the order of left to right, top to bottom, right to left and bottom to top, while holding the signal value at the region boundary pixel and the next pixel, For other pixels, the signal value of the pixel and the signal value of the previous pixel are updated to a synthesized value. Thereby, the smoothing process is divided at the region boundary pixels, and a value synthesized from the color difference signal values of a wide range of pixels in the region surrounded by the region boundary pixels is obtained, and the color noise is removed.

Finally, the color difference signal value of the input image and the generated smoothed color difference signal value corresponding to each other are compared for each pixel to synthesize the output color difference signal. This is because the color difference signal value of the input image is subtracted from the smoothed color difference signal value to generate a color difference error signal. If the amplitude of the color difference error signal is smaller than the predetermined level, the amplitude is not changed. Is applied to the color difference signal value of the input image to obtain a combined color difference signal output. As a result, a smoothed color difference signal is output when the amplitude of the color difference error signal is small, and a signal that approaches the color difference signal of the input image is output as the amplitude of the color difference error signal increases. Therefore, color noise can be reduced when the difference in the color difference signal due to smoothing is at a level that can be regarded as noise fluctuation, and even if the smoothed color difference signal is blurred due to the division of the region boundary pixels due to noise or the like, the color difference signal of the input image Therefore, when the color difference signal changes greatly in a portion where the color video signal is moving, the color difference signal of the input image is output and no color afterimage is generated. As a result, it is possible to obtain a high-quality color image signal with no color blur and no color afterimage and reduced color noise.

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a functional block diagram of a color noise reduction method according to the first embodiment of the present invention, FIG. 2 is a functional block diagram of inclination detection, and FIG. 3 is a functional block of edge detection. The input color image is composed of a luminance signal Y and two color difference signals Cr and Cb. The luminance signal Y is input to the inclination detection function 1 to generate a gradient signal DET. In the inclination detection function shown in FIG. 2, signal values in the vicinity 11 of the target pixel are collected using a memory, a delay unit, and the like, and the difference between the upper and lower adjacent pixels U and D and the left and right adjacent pixels L and The difference between R is output and combined by the absolute value functions 14 and 15 and the addition function 16 to output a gradient signal DET. The ridge line detection function 2 detects a region boundary pixel from the gradient signal as follows. Gather the gradient signal values of the pixel of interest and its neighboring pixels 21 using memory, delay means, etc., and compare the pixels of interest U, D, L, and R adjacent in the vertical and horizontal directions with the comparison functions 22-1-4, In consideration of the noise level, 1 is output when it is determined that the gradient signal value of the target pixel is clearly higher, and 0 is output otherwise. The outputs of the comparison functions 22-1 to 4 are added by the addition function 23, and 1 is output as a region boundary signal BRDR when the determination function 24 is more than half (2), and 0 is output otherwise.

The region boundary signal BRDR and the color difference signals Cr and Cb thus generated are input to the two-dimensional smoothing functions 3 and 4, respectively. FIG. 4 is a functional block diagram of the two-dimensional smoothing function. The input color difference signal and area boundary signal are stored in the color difference memory 25 and the boundary memory 26 controlled by the scanning address generation function 27, respectively. The color difference memory 25 and the boundary memory 26 have three screens, and the write address, smoothing process address, and read address are switched for each screen by the scanning address generation function 27 as shown by the hatched area in FIG. However, by appropriately controlling, a two-dimensional smoothed signal for a continuous image can be obtained continuously. The delay function 28 gives a delay of one pixel to the region boundary signal, the logical sum is taken by the OR function 32, and the switch 31 is switched to the b side at the region boundary pixel and the next pixel. When the switch 31 is switched to the b side, the pixel signal stored in the color difference memory 25 is input to the delay function 29 and the color difference memory 25 as it is. Therefore, the region boundary pixel and the next pixel are not subjected to the smoothing process. In other pixels, the switch 31 is on the 0 side, so that the signal values of the color difference memory 25 and the delay function 29 are mixed by the addition function 30 and input to the delay function 29 and the color difference memory 25. In this way, smoothing processing divided at the region boundary pixels in one scanning direction can be realized.

FIG. 6 illustrates the scanning directions of the color difference memory 25 and the boundary memory 26. FIGS. 6A, 6B, 6C, and 6D illustrate scanning from left to right, top to bottom, left to right, and bottom to top, respectively. A dotted line indicates a return line. By setting the region boundary signal to the peripheral pixels, it is possible to prevent the color difference signals of the pixels at both ends from being mixed. FIG. 7 shows a procedure for realizing the two-dimensional smoothing process using a flowchart. FIG. 8 shows how the signal spreads when the signal value 4096 is present at the pixel position at the center of the hatching by this smoothing process in the flat portion of the image. Smoothing is repeatedly performed while changing the smoothing direction, and a smooth value can be synthesized using a wide range of pixel values of 500 pixels or more by simple processing. The contribution ratio of each pixel to the smooth value is 1% or less, and randomly generated noise is reduced to 1/10 or less. FIG. 9 shows how the signal spreads when there is a signal value 4096 at the hatched pixel position by the smoothing process when there is a region boundary. A black portion indicates a region boundary pixel. The contribution ratio of each pixel to the smooth value increases more in the region boundary than in the flat region, and rapidly decreases outside the region boundary, although a signal oozes out from the breakpoint of the region boundary pixel. Therefore, even if the region boundary pixel is divided due to the influence of noise or the like, the influence on the smoothed color difference signal is slight. In this way, the smoothed color difference signal generated in the color difference memory 25 is controlled by the scanning address generation function 27 and is output in time with the color difference signal of the input color image.

The generated smoothed color difference signal value corresponding to the color difference signal value of the input image is subtracted by the subtraction functions 5 and 6 to generate a color difference error signal. The non-linear processing functions 7 and 8 perform non-linear processing for gradually decreasing the amplitude when the amplitude of the color difference error signal is smaller than a predetermined level, and when the amplitude is larger than the predetermined level, and the addition functions 9 and 10 It is added to the color difference signal value to obtain a combined color difference signal output. As a result, a smoothed color difference signal is output when the amplitude of the color difference error signal is small, and a signal that approaches the color difference signal of the input image is output as the amplitude of the color difference error signal increases. Therefore, color noise can be reduced when the difference between the color difference signals due to smoothing is at a level that can be regarded as noise fluctuation. Even if the smoothed chrominance signal is blurred due to the division of the area boundary pixels due to noise, etc., the chrominance signal of the input image is output, so it does not bleed, and the chrominance signal in the part where the color video signal moves When the value changes greatly, the color difference signal of the input image is output, so that no color afterimage occurs. As a result, it is possible to obtain a high-quality color image signal with no color blur and no color afterimage and reduced color noise.

FIG. 10 is a block diagram of a color noise reduction apparatus according to the second embodiment of the present invention, and parts having the same functions as those in FIG. The input signal is G BR of the three primary color signals of light, and the thinning circuit 33 outputs an average value signal when the input image signal is divided into a plurality of blocks each having a size of vertical n pixels × horizontal m pixels. The G signal and the R signal that have a great influence on the resolution of the human eye are respectively input to the inclination detection circuit 1 and added by the adder 35 to output a gradient signal. The subtracter 34 subtracts the G signal thinned out from the thinned B signal and R signal, and outputs color difference signals BG and RG. Based on the region boundary detected by the ridge line detection circuit 22, the color difference signals BG and RG are converted into smoothed color difference signals by the two-dimensional smoothing circuits 3 and 4, respectively.

FIG. 11 is a block diagram of a smoothing circuit according to the second embodiment of the present invention, and FIG. 12 is a flowchart showing a procedure for realizing a two-dimensional smoothing process according to the second embodiment of the present invention. Parts having the same functions as those in FIG. 4 are denoted by the same reference numerals. The difference from FIG. 4 is that the smoothing circuits are cascaded. The smoothing circuit 37 is the same as the smoothing function of FIG. 4 and executes a smoothing process that is divided at boundary pixels along the main scanning direction. On the other hand, the delay element 39 is a delay element that outputs a signal of a pixel adjacent in the sub-scanning direction orthogonal to the main scanning direction, and the smoothing circuit 38 performs a smoothing process divided by boundary pixels along the sub-scanning direction. Execute. For example, by performing memory access in the scanning direction of FIG. 6A, a signal that has been smoothed from left to right and a signal that has been smoothed from top to bottom are generated in the memory. Further, by performing memory access in the scanning direction of FIG. 6C, a signal that has been smoothed from right to left and a signal that has been smoothed from bottom to top are generated in the memory. Therefore, in the flowchart of FIG. 12, the same smoothing process as that of the flowchart of FIG. 7 can be realized, and the number of memory accesses can be halved.

The smoothed color difference signals generated in this way are input to the subtracters 5 and 6, and the color difference signals BG and RG for the input image signal are subtracted from the smoothed color difference signals to generate a color difference error signal. In general, since the noise level in the color image signal is large in a dark portion and becomes substantially constant above a predetermined level, the noise level estimator 36 estimates the noise level from the G signal level of the input image, and the nonlinear processing circuits 7 and 8 are nonlinear. Control properties. The output of the non-linear processing circuit 7 is a chrominance signal with little influence on the B signal and the G signal by adding 1/2 to the B signal by the adder 9 and subtracting 1/2 from the G signal by the subtractor 37. Noise suppression of B-G signal components can be performed. The output of the non-linear processing circuit 8 is also input to the adder 10 and the subtractor 38, so that an R signal and a G signal in which noise of the color difference signal RG component is reduced can be output.

By using the thinning circuit 33, the number of vertical and horizontal pixels becomes 1 / n times and 1 / m times, respectively, and the size of the delay element and the memory required to generate the gradient signal DET and the region boundary signal BRDR is reduced to 1 /. The size of color difference memory and boundary memory is reduced to 1 / (n × m) times. Further, the calculation amount of the two-dimensional smoothing process is also reduced to 1 / (n × m) times. As a result, if the smoothing process is accommodated in the vertical blanking period, the memory for one screen as shown in FIG. Dimensional smoothing is possible, that is, the color difference signal value smoothed at the start of the thinning block is read out and used for color difference signal synthesis, the average value in the block of the color difference signal is written at the end of the thinning block, and the blanking period By executing the two-dimensional smoothing process and writing the smoothed color difference signal value, it is possible to reduce color noise with a small memory capacity and a small time delay.

FIG. 14 shows a block diagram of a still image color imaging apparatus according to the third embodiment of the present invention. The control unit 40 drives the solid-state image sensor 42 via the drive circuit 41. The solid-state image sensor 42 is provided with, for example, a primary color Bayer array color filter shown in FIG. 15, and three primary color signals R, G, and B encoded at pixel positions are output. The A / D converter 43 converts the output signal of the solid-state image sensor 42 into a digital value and outputs it to the processor 44. The sensitivity of the imaging device can be set by controlling the A / D conversion gain from the control unit 40. The processor 44 rearranges and stores the A / D converted signals in the same order as the pixel arrangement of the solid-state image sensor 42 in the memory 45 based on information from the control unit 40 so that adjacent pixel information can be easily extracted. . The processor 44 performs processing such as gamma correction, signal interpolation, and contour correction to generate and output a color image signal.

The relationship between the signal level of the color image signal output from the processor 44 and the noise level will be described with reference to FIG. FIG. 16A shows the relationship between the signal amount of the solid-state image sensor 42 and the amount of noise included therein. The background noise is a constant noise component even if the signal amount changes, and the optical shot noise is a noise component whose noise amount is proportional to the square root of the signal amount because the number of detected signal charges follows the Poisson distribution, and scene noise. Is a noise component which is generated due to sensitivity variation of each pixel and becomes a noise amount proportional to the signal amount. In recent solid-state imaging devices, since the charge detection sensitivity is improved and the pixels are built with high accuracy, the scene noise is at a negligible level.

The main components of the background noise are shot noise of dark current (current that flows even during light shielding due to leakage current in photoelectric conversion elements and charge transfer units), fixed pattern noise due to variations in dark current, and charge-voltage conversion amplifier and AD conversion It is amplifier noise of the instrument. FIG. 16B shows the relationship between the dark signal amount and the background noise amount. When the dark signal amount increases with long exposure, the background noise amount increases in proportion to the exposure time. FIG. 16C shows the gamma correction characteristic for the input signal of the processor 44 and its differential gain. The noise component superimposed on the input signal is multiplied by this differential gain and output.

FIG. 16D shows how the relationship between the output signal level of the processor 44 and the noise level superimposed thereon changes depending on the imaging conditions. Long exposure increases the amount of dark signal and consequently increases background noise. On the other hand, since the optical shot noise has the same input signal, the output noise level increases when the signal level is small, but becomes the same noise level as the standard imaging state as the signal level increases. When the sensitivity is increased, since the control unit 40 sets the A / D conversion gain to be large, both the background noise and the light shot noise increase, and the noise level increases over the entire signal level.

The output signal of the processor 44 subjected to gamma correction stored in the memory 45 is repeatedly input to the color noise reduction function 46. The control unit 40 gives the characteristics of the noise level estimation function 36 according to the imaging conditions. The color noise reduction function 46 generates a color difference error signal that is a difference signal between the output signal of the processor 44 and the smoothed color difference signal after generating a color difference signal that has been two-dimensionally smoothed by the above-described procedure. The noise level estimation function 36 calculates a noise level that matches the imaging conditions. When the amplitude of the color difference error signal is within the noise level, the non-linear processing functions 7 and 8 pass the color difference error signal and combine it with the output signal of the processor 44 to remove the noise component of the color difference signal. Further, when the amplitude of the color difference error signal determined to be a significant color difference signal is larger than the noise level, the nonlinear processing functions 7 and 8 reduce the amplitude of the color difference error signal and mainly synthesize the input color difference signal. A color difference signal is output. As a result, an image signal in which color difference signal noise is optimally reduced can be obtained.

As described above, according to the color noise reduction method of the present invention, the fixed pattern such as the color noise of one input image signal or the dark current unevenness of the solid-state image sensor without blurring the color at the image edge portion. Even color noise can be reduced. The color noise reduction method according to the present invention may be implemented by either hardware or software, and the color signal smoothing coefficient may be other than the exemplified values.

The functional block diagram which concerns on the color noise reduction method of this invention. The functional block diagram of inclination detection. The functional block diagram of a ridgeline detection. The functional block diagram of a smoothing function. The figure for demonstrating the memory access of a two-dimensional smoothing process. The figure for demonstrating the scanning direction. The flowchart which shows the implementation | achievement procedure of a two-dimensional smoothing process. The figure which shows the contribution rate to the smooth value of the smoothing process in the flat part of an image. The figure which shows the contribution rate to the smooth value of the smoothing process when there exists an area | region boundary. The block diagram which concerns on the 2nd Example of this invention. The block diagram of the smoothing circuit which concerns on the 2nd Example of this invention. The flowchart which shows the implementation | achievement procedure of the two-dimensional smoothing process which concerns on 2nd Example of this invention. The figure for demonstrating the memory access of the two-dimensional smoothing process which concerns on 2nd Example of this invention. FIG. 6 is a block diagram of a color imaging apparatus according to a third embodiment of the present invention. The figure which shows an example of the arrangement | sequence of the color filter used for the single-panel color camera concerning this invention The figure for demonstrating the relationship between a signal level and a noise level.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Inclination detection function 2 Ridge line detection function 3, 4 Two-dimensional smoothing function 5, 6, 12, 13, 34, 37, 38 Subtraction function 7, 8 Non-linear processing function 9, 10, 16, 23, 30, 35 Addition function Reference pixel 11 Reference pixel neighborhood 14, 15 Absolute value function 21 Target pixel and its neighboring pixels 22 Comparison function 24 Judgment function 25, 26, 45 Memory 27 Scan address generation function 28, 29 Delay function 31 Switch 32 OR function 33 Decimation function 36 Noise Level estimation function 37 Horizontal smoothing circuit 38 Vertical smoothing circuit 39 Delay element 40 Control unit 41 Drive circuit 42 Solid-state image sensor 43 A / D converter 44 Processor 46 Color noise reduction function

Claims (2)

The gradient signal is generated by taking the absolute value of the value obtained by first-order differentiation of the brightness signal included in the input color image signal, and the generated gradient signal value is larger than the gradient signal value of more than half of the adjacent pixels. Means for extracting as a region boundary pixel and storing the color difference signal image and the region boundary image; scanning the stored color difference signal image and the region boundary image; and the color difference signal value at the region boundary pixel and the next pixel position In other pixels, smoothing processing is performed in which the weighted average of the previous color difference signal value and the color difference signal value of the target pixel is updated as the color difference signal value, and the scanning direction is left, down, right, or up. By executing the smoothing process a plurality of times, a two-dimensional color difference smoothed signal is generated, and the input image is obtained from the smoothed color difference signal according to a difference signal between the smoothed color difference signal and the color difference signal of the input image. Color difference signal A color difference error signal is generated, and when the amplitude of the color difference error signal is small, a signal subjected to non-linear processing for reducing the amplitude when the amplitude of the color difference error signal exceeds a predetermined level is added to the color difference signal of the input image. A color noise reduction method characterized by combining color difference signals. An image pickup device that photoelectrically converts an optical image via an optical system and outputs an image pickup signal, a signal processing unit that generates a color image signal from the output signal of the image pickup device, and a color noise reduction method according to claim 1 A color imaging apparatus having at least means for reducing the color difference, wherein the predetermined level of the non-linear characteristic of the color difference error signal is changed according to an imaging condition.

Images