KR100575443B1 - Method and apparatus for interpolating image and removing noise - Google Patents

Abstract

The present invention relates to a method and apparatus for interpolating an image and removing noise. In the method according to the present invention, the interpolation and noise removing unit combined with (2n + 2) (where n = 1, 2, 3) line memory interpolates the image and removes the noise ( A first step of interpolating the image in a 2n + 3) × (2n + 3) mask; Generating a (2n + 1) × (2n + 1) mask by interpolating the image; And a third step of removing noise from the (2n + 1) × (2n + 1) mask. As a result, the present invention is cost effective because the same noise reduction filter effect is obtained by reducing the overall line memory.

Interpolation, noise, line memory, filter                                    

Description

Method and Apparatus for Interpolating and Removing Noise {Method and Apparatus for interpolating image and canceling an noise of image}

1A-1D illustrate an example of an RGB Bayer pattern of 5 × 5 mask size.

2 is a convex configuration diagram of an apparatus for interpolating a conventional image and removing noise.

Fig. 3A is an illustration of the case where G3 is the center pixel in the RG line of the RGB Bayer pattern of 3x3 mask size.

3B is an exemplary diagram when the R1 component is the center pixel in the RG line of the RGB Bayer pattern having the size of 3x3 mask.

4 is a block diagram of a noise removing filter for removing noise in a 3 × 3 mask.

5 is a block diagram illustrating an apparatus for interpolating an image and removing noise according to an exemplary embodiment of the present invention.

6A is an exemplary diagram when G5 is a center pixel value in a GB line having a 5 × 5 mask size.

The present invention relates to a method and apparatus for interpolating an image and removing noise, and more particularly, to a method and an apparatus for removing noise included in an image simultaneously with interpolation of the image using four line memories.

In the image sensor, several pixels are arranged in a two-dimensional structure, and each pixel converts it into an electrical signal according to the brightness of light. By measuring the electric signal, the amount of light entering each pixel can be known and converted into an electric signal to form an image of a pixel unit. The electric signal corresponding to the amount of light is expressed numerically. The value is called a pixel value. Here, the pixel value may be expressed as a value between 0 and 255.

Digital cameras generally use a single image sensor, which requires more information about each pixel in order to obtain a full color image. However, since the pixels of the image sensor extract pixel values of only one color from among the plurality of colors included in the image, information of the lost pixels using the color filter array (CFA) is used for the surrounding pixels. Inferred from information.

The RGB Bayer pattern is the most common color filter array, specifically as described with reference to FIGS. 1A-1D. 1A to 1D are exemplary views of RGB Bayer patterns having a 5 × 5 mask size. Referring to Fig. 1, the first column shows RGRGRG... The second column shows GBGB GB... The odd columns may have a repeating pattern of the first column, and the even columns may have a repeating pattern of the second column. The first pixel of the first column may be any one of R, G, and B pixels, and the R, G, and B pixels may have four types of patterns.

On the other hand, using the color filter array to restore the image is called color interpolation (Color Interpolation).

2 is a convex block diagram of an apparatus for interpolating a conventional image and removing noise. Referring to FIG. 2, the image sensor 201 generates an image of an RGB Bayer pattern by capturing an image.

Pixel values generated from the image sensor 201 are stored in the line memories 203 and 205. In order to interpolate the color image, a line memory is required because the pixel value of the previous line and the pixel value of the subsequent line are required in addition to the pixel value of the current line.

The interpolator 207 then interpolates the center pixel of the current line.

The method of interpolating the center pixel of the current line by the interpolator 207 is largely divided into non-adaptive interpolation methods (Adaptive Algorithms) and adaptive interpolation methods (Adaptive Algorithms). Non-adaptive interpolation is an algorithm that interpolates a fixed pattern for all pixels, which is easy to perform and has a small amount of calculation.

 The adaptive algorithm is an algorithm that estimates using the characteristics of the most effective neighboring pixels to find the missing pixel value. Although it has a large amount of calculation, it can obtain a better image than the non-adaptive interpolation method.

Non-adaptive interpolation methods are the nearest neighbor pixel interpolation (Nearest neighbor replication), bilinear interpolation (Bilinear interpolation), median interpolation (Median interpolation), and the like, gradually change in color interpolation, adaptive interpolation methods are pattern matching interpolation algorithm Pattern matching based interpolation algorithm, interpolation using a threshold-based variable number of gradients, and edge sensing interpolation.

The most representative bilinear interpolation among these interpolation methods will be described.

Bilinear interpolation is a method of multiplying the four nearest pixels by a weight and assigning a center pixel value. The weight is determined linearly, and each weight is directly proportional to the distance from each pixel.

Referring to Figures 3a and 3b will be described in more detail with respect to the bilinear interpolation method. 3A is an exemplary diagram in the case where G3 is the center pixel in the RG line of the RGB Bayer pattern having the size of 3x3 mask, and according to the bilinear interpolation method, R, G, and B components are determined as in Equation (1).

R_new = (R1 + R2) / 2

B_new = (B1 + B2) / 2

G_new = G3

On the other hand, in the case where G is a medium-sized pixel in the GB line, only the positions of R and B in FIG. 3A are interchanged, and thus the same method may be applied.

3B is an exemplary diagram in which the R1 component is the center pixel in the RG line of the RGB Bayer pattern having the size of 3 x 3 mask, and according to the bilinear interpolation method, the R, G, and B components are determined as in Equation (2).

R_new = R1

B_new = (B1 + B2 + B3 + B4) / 4

G_new = (G1 + G2 + G3 + G4) / 4

On the other hand, in the case where B is a small pixel in the GB line, only the positions of R and B in FIG. 3B are interchanged, and thus the same method may be applied.

Referring back to FIG. 2, noise is removed by the noise removing units 209, 211, and 213 through the interpolation unit 207.

The pixel value contains noise due to internal or external factors of the image sensor, and when reproduced as an image using the pixel value including this noise, it may be different from the actual image. The pixel value including is to be corrected by the noise removing unit.

The noise removing units 209, 211, and 213 generally remove noise for each of the R, G, and B components. The noise remover 209, 211, and 213 will be described in more detail with reference to FIG. 4. 4 is a block diagram of a noise removing filter for removing noise in a 3 × 3 mask.

The noise canceling filter is composed of two line memories 401 and 403 and a noise canceling filter 405. Since the noise removing filter 405 included in the noise removing unit needs the pixel value of the previous line and the pixel value of the subsequent line in addition to the pixel value of the current line like the interpolation unit to remove noise, the line memories 401 and 403. ) Is required.

The noise removing filter 405 removes noise using a median filter, a Gaussian filter, an arithmetic mean filter, and a midpoint filter. (Midpoint filter), alpha-trimmed mean filter. Since the above methods are obvious to those skilled in the art, detailed descriptions will be omitted.

However, in the conventional apparatus and method for interpolating and removing noise, the interpolation and the noise removal are separated because the interpolation unit can interpolate only the center pixel value at a 3 × 3 mask size. In the case of removing the noise of the B component, a total of six line memories, two for each component, are required, resulting in a large burden in terms of cost.

Accordingly, the present invention has been made to solve various problems of the prior art, and a method and apparatus capable of simultaneously performing interpolation and noise removal operations on R, G, and B components of an image acquired in an image sensor The purpose is to provide.

In addition, an object of the present invention is that the interpolation and noise can be performed at the same time, so that the number of line memory required to perform the operation can be reduced, the method of interpolating the image and can eliminate the noise that can achieve cost reduction compared to the prior art And providing an apparatus.

According to an exemplary embodiment of the present invention, an interpolation and noise removing unit combined with (2n + 2) (where n = 1, 2, and 3) line memories interpolate an image and removes noise. 1. A method of removing an image comprising: a first step of interpolating an image in a (2n + 3) × (2n + 3) mask having a Bayer pattern; Generating a (2n + 1) × (2n + 1) mask by interpolating the image; And a third step of removing noise from the (2n + 1) × (2n + 1) mask.

Here, the first step of the method may be performed by a bilinear interpolation method.

In addition, the third step of the method is a method using a median filter, a method using a Gaussian filter, a method using an arithmetic mean filter, and a midpoint filter. It may be made by at least one of the method used, the alpha-trimmed mean filter.

According to another embodiment of the present invention (2n + 2) (where n = 1, 2, 3) line memory for storing a pixel value; Interpolating the image in a (2n + 3) × (2n + 3) mask made of a Bayer pattern, generating a (2n + 1) × (2n + 1) mask, and including an interpolation and noise removing unit for removing noise An apparatus for interpolating an image and removing noise is provided.

The apparatus may further include an image sensor for capturing an image, generating a pixel value corresponding to the image, and storing the always-generated pixel value in a line memory.

Hereinafter, an apparatus and a method for interpolating an image and removing noise according to the present invention will be described with reference to the accompanying drawings.

5 is a block diagram illustrating an apparatus for interpolating an image and removing noise according to an exemplary embodiment of the present invention. Referring to FIG. 5, the image sensor 501 photographs an image to generate pixel values of an RGB Bayer pattern.

The pixel values generated from the image sensor 501 are stored in four line memories 503, 505. 507, 509. In order to remove noise while interpolating an image, a line memory is required because the pixel values of the previous two lines and the pixel values of two subsequent lines are required in addition to the pixel values of the current line. That is, a 5 × 5 mask is required because the R, G, and B pixel values must be interpolated to remove noise in the Bayer pattern having a 3 × 3 mask size.

The interpolation and noise removing unit 511 interpolates pixel values having a 3 × 3 mask size using pixels stored in four line memories 503, 505, 507, and 509. Thereafter, the interpolation and noise removing unit 511 removes the noise by the noise removing method described above at a 3 × 3 mask size. A method of interpolating and removing the noise by the interpolation and noise removing unit 511 will be described in more detail with reference to FIGS. 6A and 6B.

6A shows an exemplary view when G5 is a center pixel value in a GB line having a 5 × 5 mask size. The interpolation and noise removing unit 511 interpolates the G component in the G5 pixel and removes the noise as shown in Equation 3, respectively.

Gn = (G1 + G2 + G5 + G3) / 4 ------------------------- (1000a)

Gw = (G2 + G4 + G7 + G5) / 4 ------------------------- (1000b)

Gs = (G5 + G7 + G9 + G8) / 4 ------------------------- (1000c)

Ge = (G3 + G5 + G8 + G6) / 4 ------------------------- (1000d)

Gwn = G2 -------------------------------- (1000e)

Gws = G7 -------------------------------- (1000f)

Ges = G8 -------------------------------- (1000g)

Gen = G3 -------------------------------- (1000h)

Gc = G5 -------------------------------- (1000i)

Mg = noise_cancellation_filter ([Gn, Gw, Gs, Ge, Gwn, Gws, Ges, Gen, Gc]) -------------------------- -------------------- (1000j)

if G_FLAG == 1% bilinear interpolation and noise cancellation filter

Gnew = Mg

else% bilinear interpolation

Gnew = Gc

---------------------------------------------- (1000k)

end
(% Is a symbol representing a comment, and the same meaning hereinafter)

Specifically, the interpolation and noise removing unit interpolates the G components in the R2, B3, B4, and R5 pixels by bilinear interpolation as shown by reference numerals 1000a to 1000d. Thereafter, the interpolation and noise removing unit 511 removes the noise of the G component from the G5 pixels of the reference numerals 1000a to 1000i by using the noise removing filter as shown by reference numeral 1000j.

The interpolation and noise removing unit 511 may select whether to apply a noise removing filter, such as 1000k.

In addition, the interpolation and noise removing unit 511 interpolates the B component in the G5 pixel and removes the noise as shown in Equation 4, respectively.

Bwn = (B1 + B3) / 2 ------------------------- (2000a)

Bws = (B3 + B5) / 2 ------------------------- (2000b)

Bes = (B4 + B6) / 2 ------------------------- (2000c)

Ben = (B2 + B4) / 2 ------------------------- (2000d)

Bn = (B1 + B2 + B3 + B4) / 4 ------------------------- (2000e)

Bw = B3 ------------------------- (2000f)

Bs = (B3 + B4 + B5 + B6) / 4 ------------------------- (2000g)

Be = B4 ------------------------- (2000h)

Bc = (B3 + B4) / 2 ------------------------- (2000i)

Mb = noise_cancellation_filter ([Bn, Bw, Bs, Be, Bwn, Bws, Bes, Ben, Bc])

--------------------------------------------- (2000j)

if B_FLAG == 1% bilinear interpolation and noise cancellation filter

Bnew = Mb

else% bilinear interpolation

Bnew = Bc

--------------------------------------------- (2000k)

end

Specifically, the interpolation and noise removing unit interpolates the B component in the G2, G7, G8, G3, B3, and B4 G5 pixels by bilinear interpolation as shown by reference numerals 2000a to 2000e, 2000g, and 2000i. Thereafter, the interpolation and noise removing unit 511 removes the noise of the B component from the G5 pixels of the reference numerals 2000a to 2000i by using the noise removing filter as shown by reference numeral 2000j.

Meanwhile, the interpolation and noise removing unit 511 may select whether to apply a noise removing filter, such as 2000k.

In addition, the interpolation and noise removing unit 511 interpolates the R component in the G5 pixel and removes the noise as shown in Equation 5, respectively.

Rwn = (R1 + R2) / 2 ------------------------- (3000a)

Rws = (R4 + R5) / 2 ------------------------- (3000b)

Res = (R5 + R6) / 2 ------------------------- (3000c)

Ren = (R2 + R3) / 2 ------------------------- (3000d)

Rn = R2 ------------------------- (3000e)

Rw = (R1 + R2 + R4 + R5) / 4 ------------------------- (3000f)

Rs = R5 ------------------------- (3000g)

Re = (R2 + R3 + R5 + R6) / 4 ------------------------- (3000h)

Rc = (R2 + R5) / 2 ------------------------- (3000i)

Mr = noise_cancellation_filter ([Rn, Rw, Rs, Re, Rwn, Rws, Res, Ren, Rc])

--------------------------------------------- (3000j)

if R_FLAG == 1% bilinear interpolation and noise cancellation filter

Rnew = Mr

else% bilinear

Rnew = Rc

-------------------------------------------- (3000k)

end

Referring to Equation 5 in detail, the interpolation and noise removing unit as shown by reference numerals 3000a to 3000d, 3000f, 3000h, and 3000i may use the R component in the G2, G7, G8, G3, R2, R5, and G5 pixels by bilinear interpolation. Interpolate Thereafter, the interpolation and noise removing unit 511 removes the noise of the R component from the G5 pixels of the reference numerals 3000a to 3000i by using the noise removing filter as shown by reference numeral 3000j.

Meanwhile, the interpolation and noise removing unit 511 may select whether to apply a noise removing filter, such as 3000k.

6B shows an exemplary view when R5 is a central pixel value in an RG line having a 5 × 5 mask size. The interpolation and noise removing unit 511 interpolates the G component in the R5 pixel and removes noise as shown in Equation 6, respectively.

Gn = G4 ------------------------- (4000a)

Gw = G6 ------------------------- (4000b)

Gs = G9 ------------------------- (4000c)

Ge = G7 ------------------------- (4000d)

Gwn = (G1 + G3 + G6 + G4) / 4 ------------------------- (4000e)

Gws = (G6 + G8 + G11 + G9) / 4 ------------------------- (4000f)

Ges = (G7 + G9 + G12 + G10) / 4 ------------------------ (4000g)

Gen = (G2 + G4 + G7 + G5) / 4 ------------------------- (4000h)

Gc = (G4 + G6 + G9 + G7) / 4 ------------------------- (4000i)

Mg = noise_cancellation_filter ([Gn, Gw, Gs, Ge, Gwn, Gws, Ges, Gen, Gc])

--------------------------------------------- (4000j)

if G_FLAG == 1% bilinear interpolation and noise cancellation filter

Gnew = Mg

else% bilinear

Gnew = Gc

--------------------------------------------- (4000k)

end

Specifically, the interpolation and noise removing unit interpolates the G components in the B1, B3, B4, B2, and R5 pixels by bilinear interpolation as shown by reference numerals 4000e to 4000i. Thereafter, the interpolation and noise removing unit 511 removes the noise of the G component from the R5 pixels of reference numerals 4000a to 4000i by using a noise removing filter as shown by reference numeral 4000j.

The interpolation and noise removing unit 511 may select whether to apply a noise removing filter such as 4000k.

In addition, the interpolation and noise removing unit 511 interpolates the B component in the R5 pixel and removes the noise as shown in Equation 7, respectively.

Bwn = B1 ------------------------- (5000a)

Bws = B3 ------------------------- (5000b)

Bes = B4 ------------------------- (5000c)

Ben = B2 ------------------------- (5000d)

Bn = (B1 + B2) / 2 ------------------------- (5000e)

Bw = (B1 + B3) / 2 ------------------------- (5000f)

Bs = (B3 + B4) / 2 ------------------------- (5000g)

Be = (B2 + B4) / 2 ------------------------- (5000h)

Bc = (B1 + B2 + B3 + B4) / 4 ------------------------- (5000i)

Mb = noise_cancellation_filter ([Bn, Bw, Bs, Be, Bwn, Bws, Bes, Ben, Bc]);

--------------------------------------------- (5000j)

if B_FLAG == 1% bilinear interpolation and noise cancellation filter

Bnew = Mb

else% bilinear

Bnew = Bc

--------------------------------------------- (5000k)

end

Specifically, the interpolation and noise removing unit interpolates the B components in the pixels G4, G6, G9, G7, and R5 by bilinear interpolation as shown by reference numerals 5000e to 5000i. Thereafter, the interpolation and noise removing unit 511 removes the noise of the B component from the R5 pixels of the reference numerals 5000a to 5000i by using the noise removing filter as shown by reference numeral 5000j.

Meanwhile, the interpolation and noise removing unit 511 may select whether to apply a noise removing filter, such as 5000k.

In addition, the interpolation and noise removing unit 511 interpolates the R component in the R5 pixel and removes the noise as shown in Equation 8, respectively.

Rwn = (R1 + R2 + R4 + R5) / 2 ------------------------- (6000a)

Rws = (R4 + R5 + R7 + R8) / 2 ------------------------- (6000b)

Res = (R5 + R6 + R8 + R9) / 2 ------------------------- (6000c)

Ren = (R2 + R3 + R5 + R6) / 2 ------------------------- (6000d)

Rn = (R2 + R5) / 2 ------------------------- (6000e)

Rw = (R4 + R5) / 2 ------------------------- (6000f)

Rs = (R5 + R8) / 2 ------------------------- (6000g)

Re = (R5 + R6) / 2 ------------------------- (6000h)

Rc = R5 ------------------------- (6000i)

Mr = noise_cancellation_filter ([Rn, Rw, Rs, Re, Rwn, Rws, Res, Ren, Rc]);

-------------------------------------------------- -(6000j)

if R_FLAG == 1% bilinear interpolation and noise cancellation filter

Rnew = Mr

else% bilinear

Rnew = Rc

-------------------------------------------------- (6000k)

end

Specifically, the interpolation and noise removing unit interpolates the R components in the B1, B3, B4, B2, G4, G6, G9, and G7 pixels by bilinear interpolation as shown by reference numerals 6000a to 6000h. Thereafter, the interpolation and noise removing unit 511 removes noise of the R component from the R5 pixels of reference numerals 6000a to 6000i by using a noise removing filter as shown by reference numeral 6000j.

The interpolation and noise removing unit 511 may select whether to apply a noise removing filter such as 6000k.

The best embodiments have been disclosed in the drawings and specification above. Although specific terms have been used herein, they are used only for the purpose of describing the present invention and are not used to limit the scope of the present invention as defined in the meaning or claims. Therefore, those skilled in the art will understand that various modifications and equivalent other embodiments are possible from this.

For example, a method and apparatus for generating a 3 × 3 mask by removing an image from a 5 × 5 mask and removing noise from a 3 × 3 mask have been described as an example. By the way, the image is interpolated in a 7 × 7 mask to generate a 5 × 5 mask and the noise is removed from the 5 × 5 mask, or the image is interpolated in a 9 × 9 mask to generate a 7 × 7 mask, and the noise in the 7 × 7 mask. It is too obvious to those skilled in the art to eliminate the.

Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

An apparatus and method for interpolating an image and removing noise according to the present invention include a method and an apparatus capable of simultaneously performing interpolation and noise removing operations on components of R, G, and B, respectively, of an image obtained by an image sensor. Can be provided.
In addition, according to the present invention, since the interpolation and the noise can be performed simultaneously, the number of line memories required for performing the operation can be reduced, and thus a method for interpolating an image and removing noise, which can achieve cost reduction compared to the prior art. And an apparatus.

delete

Claims (5)

In the method of interpolation and noise removing unit combined with (2n + 2) (where n = 1, 2, 3) lines interpolate an image and remove noise, A first step of forming a (2n + 3) × (2n + 3) mask having a Bayer pattern including pixel values stored in the (2n + 2) line memories; (2n + 1) × (2n + 1) masks by interpolating the image corresponding to the (2n + 1) × (2n + 1) masks by the (2n + 3) × (2n + 3) masks Generating a second step; And A third step of removing noise for pixels corresponding to the (2n + 1) × (2n + 1) mask How to interpolate the image characterized in that the noise. The method of claim 1, The image interpolation in the second step is performed by a bilinear interpolation method using peripheral pixel values of pixels corresponding to the (2n + 1) × (2n + 1) mask. How to interpolate the image characterized in that the noise. The method of claim 1, The third step is a method using a median filter, a method using a Gaussian filter, a method using an Arithmetic mean filter, a method using a midpoint filter, alpha By at least one of alpha-trimmed mean filters How to interpolate the image characterized in that the noise. (2n + 2) (where n = 1, 2, 3) line memories for storing pixel values; Including the pixel values stored in the (2n + 2) line memory to form a (2n + 3) × (2n + 3) mask consisting of a Bayer pattern, to the (2n + 3) × (2n + 3) mask By interpolating the image corresponding to the (2n + 1) × (2n + 1) mask to generate a (2n + 1) × (2n + 1) mask, the (2n + 1) × (2n + 1) Interpolation and noise removing unit for removing noise with respect to pixels corresponding to a mask Apparatus for interpolating and removing noise, characterized in that it comprises a. The method of claim 4, wherein And an image sensor for capturing an image, generating a pixel value corresponding to the image, and storing the always-generated pixel value in a line memory. Apparatus for interpolating and removing noise, characterized in that the image.

Images