TWI528812B - System and method of reducing noise - Google Patents

Description

System and method for reducing noise

The present invention relates to the reduction of noise, and more particularly to a noise reduction system and method for an image sensor.

Images captured by digital cameras equipped with image sensors, such as complementary metal oxide semiconductor (CMOS) image sensors, inevitably contain various sources of noise. Common sources of noise are flicker noise, bad pixels, Gaussian noise, green imbalance, and the like. Therefore, it is generally necessary to reduce noise as much as possible.

The traditional method of reducing noise is usually only used to overcome a kind of noise, and most of them ignore the individual characteristics of the noise and the characteristics of the image. Therefore, the conventional method cannot achieve satisfactory results.

Therefore, it is urgent to propose a novel mechanism to reduce the noise of the image sensor.

In view of the above, one of the objects of the embodiments of the present invention is to provide a method for reducing noise of an image sensor, so that different types of noise can be reduced by an appropriate mechanism, and the characteristics of the image are considered to perform noise. The reduction.

According to an embodiment of the invention, a system for reducing noise includes an image sensor provided with a color filter array, a noise reduction device, and a color interpolation device. The color filter array is disposed on the image sensor for outputting the original data. The noise reduction device corrects the original data according to the distance between the original current pixel and the plurality of adjacent pixels of the same color in the processing mask, thereby generating a new current pixel for outputting the corrected original data. The color interpolation device receives the corrected raw data to produce full color data.

The first figure shows a block diagram of a noise reduction system 100 in accordance with an embodiment of the present invention. The system 100 of the present embodiment includes an image sensor 11 (eg, a complementary metal oxide semiconductor (CMOS) image sensor). The image sensor 11 is provided with a color filter array (CFA) or a color filter mosaic (CFM) 10, such as a Bayer color filter array, for outputting raw data. The second figure shows the pattern of a Bell color filter array, where 50% is a green filter, 25% is a blue filter, and 25% is a red filter. In other words, each 2x2 sub-array includes a two-green filter, a blue filter, and a red filter. These color filters filter the incident light according to different wavelength ranges, so that the output raw data provides intensity information for the red, green, and blue (R/G/B) wavelength regions. The color interpolation device 13 interpolates the original data to obtain full color data for display on the display device 14 (for example, a liquid crystal display). In the embodiment, before the interpolation processing of the original data, the noise reduction device 12 performs correction to generate the corrected original data, and the noise component can be greatly reduced. The noise reduction device 12 or/and the color interpolation device 13 may be part of an image signal processor (ISP) or implemented in hardware or/and software and controlled by an image signal processor.

Figure 3A illustrates a 5x5 process mask or window of green pixels that sequentially passes and processes the image formed by the original material, such as from top to bottom and from left to right. The middle column includes the current green pixel Cur in the center and the adjacent (green) color pixels P3 and P4; the two top columns containing adjacent (green) color pixels P0, P1 and P2 are processed; The two bottom columns of the (green) color pixels P5, P6, and P7 are unprocessed. The third B diagram illustrates a 5x5 processing mask of a non-green pixel, and the middle column includes a current non-green (eg, blue) pixel Cur located at the center and adjacent (non-green) color pixels P3 and P4; The two top columns of the green) color pixels P0, P1, and P2 are processed; and the two bottom columns including the adjacent (non-green) color pixels P5, P6, and P7 are unprocessed. In the present specification, "adjacent pixels of the same color" and "adjacent pixels" represent the same matter, and thus can be used interchangeably.

The fourth figure shows a flow chart of the method for reducing noise in the embodiment of the present invention. In step 41, the distance (or absolute difference) between the current pixel and the adjacent pixels is obtained respectively. In step 42, the distances are compared to a critical noise, respectively. In this embodiment, the critical noise can be defined by the user. If the distance is less than the critical noise, the associated neighboring pixels are determined to be similar neighboring pixels. In other words, the values of similar neighboring pixels are very close to the values of the current pixel, and the difference between the two is less than the critical noise. In step 43, the current pixel is classified into one of a plurality of types according to the count value of similar neighboring pixels. If there are no similar neighboring pixels (ie, type 0), it means that the current pixel may be a bad pixel. If there is a single similar neighboring pixel (ie, Type 1), it indicates that the current pixel may be a line endpoint. If there are two or more similar neighboring pixels, it is a type n (for a 5x5 processing mask exemplified for the third A/three B diagram, n=2 to 8). According to one of the features of the present embodiment, the (current) pixel is classified into one of different types according to different count values of similar adjacent pixels (step 43), and different types of pixels use different mechanisms (steps 44A, 44B). And 44C) to reduce noise.

If the current pixel is of type 0, in step 44A, interpolation is performed on adjacent pixels in the smoothest direction to generate a correction value as the new current pixel. Taking the third A picture or the third B picture as an example, if |P0-P7|<|P1-P6|<|P2-P5|<|P3-P4|, the P0-P7 direction is the smoothest direction, thus The correction value (for example, the interpolated value (P0+P7)/2) is substituted for the current pixel Cur.

If the current pixel is of type 1, in step 44B, a directional filter is used to perform interpolation on adjacent pixels and current pixels along a direction formed by similar adjacent pixels and current pixels to generate a correction value as a correction value. New current pixel. Taking the third A picture or the third B picture as an example, if P0 is a single similar neighboring pixel, the correction value in the P-Cur-P7 direction (for example, the interpolated value (P0+P7+Cur)/3) Replace the current pixel Cur.

If the current pixel is of the type n (n=2 to 8), in step 44C, a grouping-patch-based denoising mechanism is used to perform denoising on some similar neighboring pixels and the current pixel. To generate a correction value as the new current pixel. According to another feature of the embodiment, similar neighboring pixels and current pixels are first divided into two groups, namely, group 0 and group 1. The fifth A picture illustrates the current pixel of the pattern 8, and the fifth B picture sequentially displays the pixel values of the fifth A picture. Eight similar neighboring pixels and current pixels are divided into group 0 and group 1 with a maximum span. Thus, the pixel value of group 0 is smaller than the pixel value of group 1, and the maximum pixel value of group 0 to the smallest pixel value of group 1 has the largest span. Then, the group with more pixels counts becomes the selection group, and the other group is rounded off. If the number of pixels of the two groups is equal, the group including the current pixel becomes the selection group.

Next, a median or mean operation is performed according to the range of the selected group to obtain a correction value. In the present embodiment, the range is defined as the absolute difference between the largest pixel value and the minimum pixel value among the selected groups. If the range is greater than a preset value, a central number operation is used; otherwise, an average operation is used.

According to still another feature of the embodiment, in step 45, the new current pixel obtained for step 44A, 44B or 44C along with the original current pixel may also be adjusted according to the hue or saturation of the pixel. To produce full-color information. Chroma and saturation messages can be obtained from red/green/blue pixels using conventional methods. In the case of chrominance messages, if the chrominance of the original current pixel (within the processing mask) is significantly different from the chrominance of the new current pixel (for example, the absolute difference between the two is greater than a preset value), then The smaller intensity/weight is adjusted (eg, multiplied) by the new current pixel, and the greater intensity/weight is used to adjust the original current pixel, and vice versa. In other words, when a large chromaticity change is detected, less denoising is performed, and vice versa.

In the case of the saturation message, if the saturation (within the processing mask) is very low (for example, the average values of the red, green and blue pixels are close to each other, for example, the absolute difference between each other is less than a predetermined value), then The smaller intensity/weight is adjusted (eg, multiplied) by the new current pixel, and the greater intensity/weight is used to adjust the original current pixel, and vice versa. In other words, when a small change in saturation is detected, less denoising is performed, and vice versa.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention; all other equivalent changes or modifications which are not departing from the spirit of the invention should be included in the following Within the scope of the patent application.

100. . . Noise reduction system

10. . . Color filter array

11. . . Image sensor

12. . . Noise reduction device

13. . . Color interpolation device

14. . . Display device

Cur. . . Current pixel

P0~P7. . . Adjacent pixels of the same color

41~45. . . Steps to reduce noise

The first figure shows a block diagram of a noise reduction system in accordance with an embodiment of the present invention. The second figure shows the style of the Bell color filter array. The third A and third B diagrams illustrate a 5x5 processing mask. The fourth figure shows a flow chart of the method for reducing noise in the embodiment of the present invention. Figure 5A illustrates the current pixel of Type 8. The fifth B picture sequentially displays the pixel values of the fifth A picture.

100. . . Noise reduction system

10. . . Color filter array

11. . . Image sensor

12. . . Noise reduction device

13. . . Color interpolation device

14. . . Display device

Claims (16)

A system for reducing noise includes: an image sensor having a color filter array for outputting original data; and a noise reduction device for processing an original current pixel and a plurality of adjacent colors in the mask a distance between the pixels on the gray scale to correct the original data, thereby generating a new current pixel for outputting the corrected original data; and a color interpolation device receiving the corrected original data to generate a full color Data; wherein the distances are respectively compared with a critical noise, if the distance is less than the critical noise, the adjacent adjacent color pixels are similar adjacent color pixels; wherein the similar adjacent color pixels and the original Currently, pixels are divided into group 0 and group 1 according to the maximum span; groups with more pixels are selected groups; according to the range of the selected group, central or average operations are performed to obtain The new current pixel. The system for reducing noise according to claim 1, wherein the original current pixel is classified into one of a plurality of types according to count values of the similar adjacent color pixels. According to the system for reducing noise according to claim 2, if the count value of the similar adjacent color pixels is zero, the noise reduction device performs interpolation on the adjacent color pixels in the smoothest direction to generate The new current pixel. According to the system for reducing noise according to claim 2, if the count value of the similar adjacent color pixels is one, the noise reduction device forms a direction along the similar adjacent color pixels and the original current pixel. Interpolating the adjacent homochromatic pixels and the original current pixel to generate the new current pixel. According to the system for reducing noise according to claim 2, if the count values of the similar adjacent color pixels are two or more, the noise reduction device pairs some of the similar adjacent color pixels and the original current The pixel performs denoising to generate the new current pixel. The system for reducing noise according to claim 1, wherein the original current pixel and the new current pixel are mixed and adjusted according to chromaticity or/and saturation of pixels in the processing mask. According to the system for reducing noise according to claim 6 of the patent application, if the chrominance of the original current pixel is largely different from the chromaticity of the new current pixel, the smaller intensity/weight is used to adjust the new current pixel, and the A larger intensity/weight to adjust the original current pixel. According to the system for reducing noise according to claim 6 of the patent application, if the saturation is low, a smaller intensity/weight is used to adjust the new current pixel, and a larger intensity/weight is used to adjust the original current pixel. A method for reducing noise includes: outputting original data from an image sensor, wherein the image sensor is provided with a color filter array; according to processing between an original current pixel and a plurality of adjacent pixels of the same color a distance on the gray scale of the brightness to correct the original data, thereby generating a new current pixel for outputting the corrected original data; and color interpolating the corrected original data to generate full color data; wherein the distances are respectively Comparing a critical noise, if the distance is less than the critical noise, the adjacent adjacent color pixels are similar adjacent color pixels; wherein the similar adjacent color pixels and the original current pixel are divided according to the maximum span For group 0 and group 1; the group with more pixels is a selection group; according to the range of the selection group, a central number or average operation is performed to obtain the new current pixel. A method of reducing noise according to claim 9 wherein the original current pixel is classified into one of a plurality of types according to count values of the similar adjacent color pixels. According to the method for reducing noise according to claim 10, if the count value of the similar adjacent color pixels is zero, the adjacent color pixels are interpolated in the smoothest direction to generate the new current pixel. According to the method for reducing noise according to claim 10, if the count value of the similar adjacent color pixels is one, the phases are formed along the similar adjacent color pixels and the original current pixel. The adjacent color pixels and the original current pixel perform interpolation to generate the new current pixel. According to the method for reducing noise according to claim 10, if the count values of the similar adjacent color pixels are two or more, then some of the similar adjacent color pixels and the original current pixel are denoised. To generate the new current pixel. The method of reducing noise according to claim 9, wherein the original current pixel and the new current pixel are mixed and adjusted according to chromaticity or/and saturation of pixels in the processing mask. According to the method for reducing noise according to claim 14, if the original current pixel has a large chromaticity different from the chromaticity of the new current pixel, the smaller intensity/weight is used to adjust the new current pixel, and the A larger intensity/weight to adjust the original current pixel. According to the method for reducing noise according to claim 14 of the patent application, if the saturation is low, a smaller intensity/weight is used to adjust the new current pixel, and a larger intensity/weight is used to adjust the original current pixel.