KR20060019897A - Digital signal processing apparatus of image sensor capable of noise cancellation and method for noise cancellation in image sensor - Google Patents

Abstract

The present invention is to provide a digital signal processing device of the image sensor and a noise removing method of the image sensor that can effectively increase the degree of integration and reduce power consumption while effectively blocking noise due to interference and dispersion of light. An RGB kernel providing unit for receiving a R (Red) G (Green) B (Blue) digital signal to form and provide a plurality of RGB kernels; An edge detector for detecting edges using the plurality of RGB kernels; An interpolator configured to interpolate an RGB signal from the RGB kernel provider; A noise removing unit for removing a noise component from the interpolated RGB signal by referring to the noise component included in the information output from the edge detector; A color space conversion unit for converting the RGB signal having completed the interpolation and noise removal into a YCbCr signal; A YCbCr kernel providing unit for receiving the YCbCr signal and forming and providing a plurality of YCbCr kernels; And an edge emphasis unit for performing edge emphasis on a YCbCr signal in response to an output of the edge detection unit.

In addition, the present invention provides a method for removing noise of an image sensor.

Noise Reduction, Edge Value, Variable Amplifier, Edge Detection, False Color Suppression, Edge Enhancement, Image Sensor.

Description

DIGITAL SIGNAL PROCESSING APPARATUS OF IMAGE SENSOR CAPABLE OF NOISE CANCELLATION AND METHOD FOR NOISE CANCELLATION IN IMAGE SENSOR}             

1 is a diagram illustrating a Bayer pattern distribution of an image sensor.

Figure 2 is a block diagram showing a digital operation portion of the image sensor according to the prior art.

Figure 3 is a block diagram showing an image sensor according to an embodiment of the present invention.

4 is a flowchart illustrating a noise removal operation.

Explanation of symbols on the main parts of the drawings

30: 21: RGB kernel providing unit 31: interpolation unit

32: corner detection unit 33: noise removal unit

34: color space conversion unit 35: YCbCr kernel provider

36: image enhancement unit 300: first line memory unit

301: RGB kernel forming unit 350: first line memory unit

351: YCbCr kernel forming portion 360: corner emphasis

361: false color suppressor

The present invention relates to an image sensor, and more particularly, to a complementary metal oxide semiconductor (CMOS) image sensor, and more particularly, to a CMOS image sensor capable of removing noise while minimizing memory usage.

An image sensor refers to a device that reproduces an image by using a semiconductor's response to light. The image sensor detects brightness and wavelength of different light emitted from each subject and reads an electric value. It is the role of the image sensor to make the value a signal-processing level.

In other words, an image sensor is a semiconductor device that converts an optical image into an electrical signal. Among these, a charge coupled device (CCD) includes charge carriers having individual MOS capacitors in close proximity to each other. A CMOS image sensor is a device that is stored and transported in a capacitor. The CMOS image sensor uses a CMOS technology that uses a control circuit and a signal processing circuit as peripheral circuits to make MOS transistors as many as the number of pixels, and then uses them. It is a device that adopts a switching method that detects output, and because CMOS image sensor has a big advantage of low power consumption, it is very useful for personal portable systems such as mobile phones. Therefore, the image sensor can be used for various applications such as a PC (Personal Computer) camera, medical, toys.

1 is a diagram illustrating a Bayer pattern distribution of an image sensor.

In most image sensors, the colors of red (R), blue (B), and green (G) are not extracted from one pixel at a time. One color filter of RGB may be used for one pixel to have only one color. Among them, G reflects the brightness best, so it has twice the distribution of R and B. 1 shows such a Bayer pattern distribution.

Referring to FIG. 1, G in a horizontal line where R is located is indicated by Gr and G in a horizontal line where B is located by Gb. Gr and Gb have gone through the precise process of analog processing, and there is a slight difference due to dismissal gain and process loss. Also, even in the same Gr, the values of the even and odd numbers are different according to the analog structure. Therefore, noise due to the difference between the G channels exists between the pixels.

In addition, due to interference between RGB colors passing through the color filter from the light source for each wavelength and dispersion difference for each wavelength, even if Gr and Gb illuminate the same G, a difference occurs between the two. This has a larger effect as the size of the pixel becomes smaller. This can be seen as noise due to interference and dispersion of light. This phenomenon also occurs in the R and B pixels. The noise at this time has a difference of less than 1% of the statistical color value.

The noise does not significantly deteriorate the quality of the overall image color value, but the difference tends to be prominent when subsampling in accordance with the Bayer pattern.

To eliminate this difference, a filter that crushes the entire image is used, but this has the disadvantage of lowering the sharpness.

In addition, although a complicated filter or a separate memory is used, this also has a problem of power loss.

In designing CMOS image sensors, it is important to minimize the area rather than the speed. However, it is important to minimize the use of such a memory because the memory occupies a large area and a large power loss.

Hereinafter, a digital signal processing apparatus of an image sensor according to the prior art will be described.

Figure 2 is a block diagram showing a digital operation portion of the image sensor according to the prior art.

Referring to FIG. 2, a conventional image sensor inputs a first line memory unit 20 that receives a digitally converted RGB signal and stores the line-by-line unit, and an RGB digital signal provided from the first line memory unit 20. A kernel forming unit 21 which receives the RGB Bayer pattern and forms a kernel, an interpolation unit 22 which interpolates each pixel using the kernel provided from the kernel forming unit 21 with the corresponding RGB, and interpolates and outputs the kernel. A color space conversion unit 23 for converting the RGB signal into a format of luminance Y and color Cb and Cr, a second line memory unit 24 for receiving a color space converted signal in YCbCr and storing it in units of lines; YCbCr kernel forming unit 25 that forms a kernel using the YCbCr signal provided from the second line memory unit 24, and finds an edge within the kernel formed by the YCbCr kernel forming unit 25 Suppres to find false color and an image enhancing unit 26 for improving the image.

Here, the image enhancement unit 26 is a false color suppressor 261 that finds a false color and suppresses a false color, and an edge highlighter for highlighting an edge by increasing or decreasing a Y value in a portion corresponding to the corner. (262).

The RGB data signal input to the first line memory unit 20 includes an analog signal provided from a pixel array such as a decoder, an analog bus, a programmable gain amplifier, and an analog / The signal is digitized and output through an analog signal processor including an analog to digital converter.

Therefore, there is a need for a technology that can effectively increase noise and reduce power consumption while effectively blocking noise due to interference and dispersion of light during digital operation of an image sensor.

The present invention has been proposed to solve the above problems of the prior art, the digital signal processing device and image sensor of the image sensor that can effectively increase the degree of integration and reduce power consumption while effectively blocking noise due to interference and dispersion of light It is an object of the present invention to provide a method for removing noise.

In order to achieve the above object, the present invention, an RGB kernel providing unit for receiving a R (Red) G (Green) B (Blue) digital signal to form and provide a plurality of RGB kernel; An edge detector for detecting edges using the plurality of RGB kernels; An interpolator configured to interpolate an RGB signal from the RGB kernel provider; A noise removing unit for removing a noise component from the interpolated RGB signal by referring to the noise component included in the information output from the edge detector; A color space conversion unit for converting the RGB signal having completed the interpolation and noise removal into a YCbCr signal; A YCbCr kernel providing unit for receiving the YCbCr signal and forming and providing a plurality of YCbCr kernels; And an edge emphasis unit for performing edge emphasis on a YCbCr signal in response to an output of the edge detection unit.

In addition, in order to achieve the above object, the present invention provides a, b, c of any a, b, c having a relationship of a <b <c (a, b, c is a positive integer) in order to compare with the gain value Calculating a threshold value; Calculating a threshold of any x, y, z corresponding to a, b, c and having a relationship of x <y <z (x, y, z is a positive integer) to be compared with a corner value ; Extracting edge values for pixels using any kernel; Sequentially comparing the gain values of the variable amplifiers with the a, b, and c thresholds in descending order of magnitude; Sequentially comparing the corner values with the threshold values of x, y, and z in order of magnitude as the gain value of the variable amplifier is smaller than any one of the threshold values of a, b, and c; Determining that there is noise as the edge value is smaller than any one of threshold values of x, y, and z; And removing the noise by determining the RGB value as an average of all of the value of the previous pixel, the value of the current pixel, and the value of the rear pixel.

In the present invention, it was conceived that the noise appeared due to the difference of 1% in the green color affecting the overall luminance.

That is, in the conventional image sensor, the edge emphasis function is almost built in, and the edge value detected by the edge detection reflects all of these RGB values, and thus it can be seen that it includes the noise component. Therefore, the noise reduction function is implemented by using the edge value output from the edge detector without adding a separate circuit.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the technical idea of the present invention. do.

3 is a block diagram illustrating an image sensor according to an exemplary embodiment of the present invention.

Referring to FIG. 3, the image sensor of the present invention receives an RGB digital signal, and provides an RGB kernel providing unit 30 for forming and providing a plurality of RGB kernels, and an edge for detecting edges using the plurality of kernels. Noise component in the interpolated RGB signal by referring to the noise component included in the detector 32, the interpolator 31 interpolating the RGB signal from the RGB kernel providing unit, and the information output from the edge detector 32 A noise canceller 33 for removing a signal, a color space converter 34 for converting an RGB signal having completed interpolation and noise removal into a YCbCr signal, and a plurality of YCbCr kernels for receiving and supplying a YCbCr signal. A YCbCr kernel providing unit 35 and an image enhancement unit 36 for emphasizing edges in the YCbCr signal in response to the output of the edge detecting unit 32 and performing false color suppression. .

The false color detector and the edge detector 32 use color information provided by the RGB kernel provider 30 for forming a plurality of kernels using RGB signals.

The image enhancement unit 36 performs a false color suppression unit 361 that performs false color suppression on a pixel having a false color in response to the output of the false color detector, and an edge in response to the output of the corner detector 32. An edge highlighter 360 is provided to increase or decrease the Y value in the corresponding pixel portion corresponding to the edge highlighter 360.

Meanwhile, although not shown in the drawing, a false color detection unit for detecting a false color using a plurality of kernels provided from the RGB kernel providing unit 30 is further included, and the false color suppressor 361 includes a false color detection unit. In response to the output, false suppression is performed on the YCbCr signal.

The RGB kernel providing unit 30 receives an RGB digital signal and stores the line digital unit 300 to store the line unit, and receives the RGB signal from the first line memory unit 300 to form a plurality of RGB kernels. It consists of a kernel forming unit 301. Therefore, the interpolator 31 receives the RGB signal from the first line memory unit 300 and interpolates.                     

The YCbCr kernel providing unit 35 receives the YCbCr signal and stores the second line memory unit 350 to store the line unit, and receives the YCbCr signal from the second line memory unit 350 to form a plurality of YCbCr kernels. The YCbCr kernel forming unit 351 is formed.

The RGB data signal input to the first line memory unit 300 is a signal in which an analog signal provided from a pixel array unit (not shown) is output through an analog signal processing unit such as a decoder, an analog bus, a variable amplifier, and an analog / digital converter. to be.

During the kernel formation, both R and B of the pixel value are converted to G values and expressed as G. The kernel for detecting false colors and the kernel for edge detection are shared with each other. The kernel for false color detection consists of a 3 * 3 pixel array, and the kernel for edge detection consists of 1 * 5 or 5 * 1 pixel arrays. In addition, edge detection, the false color detection, and the interpolation operation are all performed in parallel.

Meanwhile, the second line memory unit 350 in the YCbCr kernel providing unit 35 may be omitted.

That is, since the first line memory unit is required before the interpolation unit, and the second line memory unit is required for color enhancement after edge enhancement and false color suppression, two line memory units are separately required. However, the line memory exists for color interpolation. In other words, since only the first line memory unit 300 needs to detect false colors and edges, a kernel for false color detection and edge detection is formed through the RGB kernel forming unit 301.

Laplacian or Sobel masks are usually used for edge detection, which requires a 3 * 4 kernel, and a 1 * 5 or 5 * 1 kernel to detect false colors. Since these two kernels have something in common, the hardware that forms them can also be shared. When we build this kernel we temporarily replace both R and B with G using the surrounding values.

In substituting G values, one takes the middle of four of the surrounding Gs, mostly centered on R and B, or uses the average of two G values in the vertical or horizontal direction. Using the kernel thus formed, edges are detected and false colors are detected.

Edge highlighting changes the luminance (Y) and false color suppression changes the saturation (Cb, Cr). Both functions perform YCbCr after color space conversion. Therefore, the edge detection part and the false color detection part should be performed in parallel with color interpolation, and then synchronized with false color suppression and edge emphasis at the rear end. Edge highlighting and false color suppression can be applied as is.

Look at the operation of the image sensor having the above configuration.

Most of the difference between the channel and light is within 1%, so when the edge value is less than that, it can be regarded as noise. Therefore, when the ADC has 11 bits, the largest edge value is 2048 in decimal, so 20%, which is 1% of the ADC, may be assumed to be noise. This is a small difference since it is about 2 codes in 8 bit operation.

The noise remover 33 is located at the rear end of the interpolator 31 to refer to the edge value of the edge detector 32 that is mostly present in the conventional image sensor.

If the edge value is less than "20", the RGB value averages the value of the previous pixel, the value of the current pixel, and the value of the rear pixel. However, image weight is minimized by giving twice the weight to the current pixel.

In general terms, when the N-bit RGB signal is used, the largest corner value is 2 ^{N in} decimal, and 1% thereof is 2 ^N / 100.

That is, when the corner value is "2 ^N / 100" or less, the average value may be expressed by Equation 1 below.

R '= (Rp + 2 x Rc + Rn) / 4

G '= (Gp + 2 × Gc + Gn) / 4

B '= (Bp + 2 × Bc + Bn) / 4

Here, R ', G', and B 'represent an average value, RpGpBp represents a corner value of a previous pixel, RcGcBc represents a corner value of a current pixel, and RnGnBn represents a corner value of a rear pixel. .

When the environment you are shooting in low light is more noise. This is because the pre-amplifier in the analog part, that is, the gain of the variable amplifier, increases and the noise is also amplified together. Therefore, the gain value of the preamplifier should be referred to and the degree to which the edge value turns out to be noise should be adjusted accordingly. When the gain of the preamplifier is high, the threshold value that turns out to be high is raised, and when it is low, the threshold value is decreased.

4 is a flowchart illustrating a noise removal operation, and looks at the noise removal process of the present invention with reference to the flowchart.

First, in order to compare with the gain value of the variable amplifier, a threshold value of any a, b, c having a relationship of a <b <c (a, b, c is a positive integer) is calculated, and a, b, c Calculate a threshold of any x, y, z corresponding respectively to and having a relationship of x &lt; y &lt; z (x, y, z are positive integers) to be compared with the edge values.

a, b, and c are values for comparing the gain values of the variable amplifiers and are used as a criterion for determining a low light environment. Therefore, the edge value is also compared with x, y, and z, and the smaller part is averaged, and the other part is determined as the edge and no action is performed.

Subsequently, an edge value of the pixel is extracted using an arbitrary kernel (S400). In this case, the edge value is mostly determined by referring to the up, down, left, right, and diagonal pixels with respect to the current pixel using a 3 × 3 kernel. After that, the edges of the edge values are applied differently in each of three steps with reference to the gain value of the variable amplifier.

The gain values of the variable amplifiers are sequentially compared with the a, b, and c thresholds in order of decreasing magnitude (S401 to S403).

As the gain value of the variable amplifier is smaller than any one of the threshold values of a, b, and c, the edge values are sequentially compared with the threshold values of x, y, and z in descending order (S404 to S406).

It is determined that there is noise as the edge value is smaller than any one of the threshold values of x, y, and z. As it is determined that there is noise, the RGB value is removed by averaging the value of the front pixel, the value of the current pixel, and the value of the rear pixel (S407).

In this case, when using the N-bit RGB signal, the corner value is the largest when the ^N is 2 ^{N in} decimal, and 1% of it is 2 ^N / 100, and there is noise when the corner is smaller than "2 ^N / 100". To judge. In the step of removing noise (S407), it is calculated by the above equation (1).

As the edge value is larger than any one of x, y, and z, it is determined that the edge has no noise, that is, no operation is performed (S408).

In addition, in the step of 'S403', as the gain value of the variable amplifier is greater than c, it is determined as a corner without noise.

According to the present invention made as described above, the noise value appears due to the difference of 1% even in the same G pixel that the G value affecting the overall luminance, the edge value output from the edge detector without adding a separate circuit Through the embodiment has been found that can be implemented using a noise canceling function.

Although the technical spirit of the present invention has been described in detail according to the above preferred embodiment, it should be noted that the above-described embodiment is for the purpose of description and not of limitation. In addition, those skilled in the art will understand that various embodiments are possible within the scope of the technical idea of the present invention.

The present invention as described above can remove noise without adding a separate circuit, thereby improving the performance and productivity of the image sensor.

Claims (14)

RGB kernel providing means for receiving a R (Red) G (Green) B (Blue) digital signal to form and provide a plurality of RGB kernels; Edge detecting means for detecting edges using the plurality of RGB kernels; Interpolation means for interpolating the RGB signal from the RGB kernel providing means; Noise removing means for removing the noise component from the interpolated RGB signal by referring to the noise component included in the information output from the edge detecting means; Color space conversion means for converting the RGB signal having completed interpolation and noise removal into a YCbCr signal; YCbCr kernel providing means for receiving the YCbCr signal to form and provide a plurality of YCbCr kernel; And Edge enhancement means for performing edge enhancement in a YCbCr signal in response to the output of the edge detection means; Digital signal processing device of the image sensor comprising a. The method of claim 1, The noise removing means, When using the N-bit RGB signal, the edge value is the largest when 2 ^N is decimal, and 1% of it is 2 ^N / 100. When the edge value is smaller than "2 ^N / 100", it is determined that there is noise. Digital signal processing device of the image sensor. The method of claim 2, The noise removing means, And the RGB value is an average of all of the value of the previous pixel, the value of the current pixel, and the value of the rear pixel as it is determined that there is noise. The method of claim 3, wherein The RGB value is a digital signal processing apparatus of the image sensor, characterized in that calculated by the following equation. R '= (Rp + 2 x Rc + Rn) / 4 G '= (Gp + 2 × Gc + Gn) / 4 B '= (Bp + 2 × Bc + Bn) / 4 (R ', G', B 'is the average value, RpGpBp is the corner value of the previous pixel, RcGcBc is the corner value of the current pixel, and RnGnBn is the corner value of the rear pixel) The method according to any one of claims 1 to 4, The noise removing means, A digital signal processing apparatus of an image sensor, characterized in that it refers to the gain of a variable amplifier and adjusts the degree to which noise is detected at a corner value according to the value of the variable amplifier. The method of claim 1, False color detecting means for detecting a false color using a part of the plurality of kernels provided by the RGB kernel providing unit, and false for a pixel having a false color in response to the output of the false color detecting means. And a false color suppression means for performing color suppression. The method of claim 6, The kernel for detecting the false color and the kernel for detecting the corner is a portion of the digital signal processing device of the image sensor, characterized in that shared with each other. The method of claim 6, The kernel for detecting the false color is a digital signal processing device of the image sensor, characterized in that for forming a pixel arrangement of 3 * 3. The method of claim 6, Kernel for detecting the corner is a digital signal processing device of the image sensor, characterized in that forming a pixel arrangement of 1 * 5 or 5 * 1. Calculating a threshold of any a, b, c having a relationship of a <b <c (where a, b, c is a positive integer) to compare with a gain value of the variable amplifier; Calculating a threshold of any x, y, z corresponding to a, b, c and having a relationship of x <y <z (x, y, z is a positive integer) to be compared with a corner value ; Extracting edge values for pixels using any kernel; Sequentially comparing the gain values of the variable amplifiers with the a, b, and c thresholds in descending order of magnitude; Sequentially comparing the edge values with the threshold values of x, y, and z in order of magnitude as the gain value of the variable amplifier is smaller than any one of the threshold values of a, b, and c; Determining that there is noise as the edge value is smaller than any one of threshold values of x, y, and z; And As the noise is determined, the RGB value is the average of the value of the previous pixel, the value of the current pixel, and the value of the rear pixel. Noise removal method of the image sensor comprising a. The method of claim 10, When using the N-bit RGB signal, the edge value is the largest when 2 ^N is decimal, and 1% of it is 2 ^N / 100. When the edge value is smaller than "2 ^N / 100", it is determined that there is noise. A method of removing noise from an image sensor. The method of claim 10, In the step of removing the noise, The RGB value is a noise removing method of the image sensor, characterized in that calculated by the following equation. R '= (Rp + 2 x Rc + Rn) / 4 G '= (Gp + 2 × Gc + Gn) / 4 B '= (Bp + 2 × Bc + Bn) / 4 (R ', G', B 'is the average value, RpGpBp is the corner value of the previous pixel, RcGcBc is the corner value of the current pixel, and RnGnBn is the corner value of the rear pixel) The method of claim 10, The noise reduction method of the image sensor, characterized in that determined as the edge is no noise as the edge value is greater than any one of the x, y, z. The method of claim 10, The noise reduction method of the image sensor, characterized in that determined as the edge of the noise-free as the gain value of the variable amplifier is greater than c.

Images