KR20070071633A - Color filter array and image interpolation method - Google Patents

Abstract

A color filter array and an image interpolation method are provided to combine images without using a frame memory and output a composite image of two images without decreasing an image output speed. A color filter array includes a first pixel region and a second pixel region. The first pixel region has pixel data according to a large quantity of light and the second pixel region has pixel data according to a small quantity of light. The first pixel region and the second pixel region are alternately arranged. Green color filters are arranged in a bayer pattern in the first pixel region and the second pixel region. A red color filter and a blue color filter are alternately arranged between neighboring green filters.

Description

Color filter array and image interpolation method

1 is an exemplary diagram of an RGB Bayer pattern.

2 and 3 are exemplary diagrams of a method of color interpolating a conventional image.

4 is an exemplary view of an image having a short exposure time, a long image, and a synthesized image thereof.

5 illustrates a pixel array of an image sensor according to an exemplary embodiment of the present invention.

6 shows a portion of a color filter array in accordance with one preferred embodiment of the present invention.

7 is a flowchart of a color interpolation method when applying a color filter array according to an exemplary embodiment of the present invention.

8 is an exemplary diagram of a color filter array for explaining a color interpolation method.

TECHNICAL FIELD The present invention relates to an image sensor, and more particularly, to a color filter array and a method of color interpolation for generating an efficient image signal for a wide dynamic range.

In the image sensor, a plurality of pixels are arranged in a two-dimensional structure, and each pixel converts them into electrical signals according to the brightness of incoming light. By measuring the electric signal, the amount of light flowing into each pixel can be known and a pixel-by-pixel image can be constructed using the same. As such, a value expressed as a numerical value of the electric signal corresponding to the amount of light is called a pixel value and is included in the pixel data. Here, the pixel value may be expressed as a value between 0 and 255.

Digital optical devices including image sensors, such as digital cameras, generally use one image sensor, which requires more information for each pixel to obtain full color. However, since the pixels of the image sensor extract pixel data of only one color from among the plurality of colors included in the image, the pixels lost from the information about the surrounding pixels using a color filter array (CFA). Infer information. The color filter array has a structure that is regularly arranged to pass only light representing one color per pixel of the pixel array. The color filter array has various forms depending on the structure in which the color elements are arranged, but the most widely used pattern is the RGB Bayer pattern. Here, R means red, G means green, and B means blue.

Half of the total number of pixels is green (G), and each quarter of the total number is allocated to red (R) and blue (B). In order to obtain color information, the color image pixels are formed in a repeating pattern with a red, green, or blue filter. For example, the Bayer pattern may be referred to as a 2 × 2 array. The Bayer pattern is based on the premise that the human eye derives most of the luminance data from the green content of the scene. Thus, by allowing more of the pixels to be green, a higher resolution image can be produced compared to an alternate RGB color filter in the same number of red, green and blue pixels.

1 is an illustration of an RGB Bayer pattern. Referring to FIG. 1, the first column includes RGRGRG. , The second row is GBGBGB... The odd columns have a repeating pattern of the first column and the even columns have a repeating pattern of the second column. The first pixel of the first column may be any one of R, G, and B pixels, and the R, G, and B pixels may have four types of patterns.

On the other hand, recovering the color of the image using such a color filter array is called color interpolation.

The interpolation method of the center pixel is largely divided into non-adaptive interpolation method (Nonadaptive Algorithms) and adaptive interpolation method (Adaptive Algorithms). Non-adaptive interpolation is an algorithm that interpolates a fixed pattern for all pixels, which is easy to perform and has a small amount of calculation. The adaptive algorithm is an algorithm that estimates using the characteristics of the most effective neighboring pixels to find the missing pixel value. Although it has a large amount of calculation, it can obtain a better image than the non-adaptive interpolation method.

Non-adaptive interpolation methods include nearest neighbor replication, bilinear interpolation, median interpolation, and progressive color change interpolation. Pattern matching based interpolation algorithm, interpolation using a threshold-based variable number of gradients, and edge sensing interpolation.

The most representative bilinear interpolation among these interpolation methods will be described.

Bilinear interpolation is a method of assigning a center pixel value by multiplying four nearest pixels by weight, and the weight is determined linearly, and each weight is directly proportional to the distance from each pixel.

2 and 3 are exemplary diagrams of a method of color interpolating a conventional image.

2A to 2C illustrate a case in which a red pixel R is a center pixel in an RGB Bayer pattern having a 3 × 3 mask size. The red pixel, which is the central pixel, acquires green and blue components using other pixels in the vicinity. As an example of the acquisition method, red, green, and blue components of the R position are determined by Equation 1 below.

<Equation 1>

Even when the center pixel is the blue pixel B, the red, green, and blue components of the B position may be determined by a method similar to the above Equation 1.

The case where the center pixel is the green pixel G is illustrated in FIGS. 3A to 3C.

The green pixel, which is the center pixel, acquires red and blue components using other pixels in the vicinity. As an example of the acquisition method, the red, green, and blue components of the R position are determined by Equation 2 below.

<Equation 2>

4 is an exemplary view of an image having a short exposure time, a long image, and a synthesized image thereof.

In the case of an image having a short exposure time, the image sensor can receive light in a short time, so that an image having a dark overall brightness is obtained (see FIG. 4A). In the case of an image having a long exposure time, an image having excessive brightness is obtained because the image sensor receives the light for more than a suitable time to receive the light (see FIG. 4B). Therefore, in both cases, there is a problem in that the dynamic range of the image sensor is narrow.

In this case, when a composite image is generated by combining an image having a short exposure time and an image having a long exposure time, the image sensor may generate an image having a wide dynamic range with suitable brightness as shown in (c) of FIG. 4. Can be obtained.

At this time, the pixel values of the red (r), green (g), and blue (b) components of each pixel of the image having a short exposure time, and the red (R), green (G), Pixel values for the blue (B) component are needed.

To this end, the image sensor having the color filter array of the RGB Bayer pattern described above additionally includes a frame memory to separately store pixel values of each pixel of an image having a short exposure time and pixel values of each pixel of an image having a long exposure time. There is a problem in that a value stored in the frame memory needs to be read when generating a composite image by a preset method.

Accordingly, the present invention provides a color filter array and a color interpolation method using the same, which eliminates the need for a frame memory for synthesizing an image in an image sensor.

The present invention also provides a color filter array and a color interpolation method using the same, in which a video output speed is not lowered in outputting a composite image of two images.

Also, for the wide dynamic range of the image sensor, the pixel values of the red (r), green (g), and blue (b) components of each pixel of the image due to the low amount of light and the red of each pixel of the image due to the large amount of light ( A color filter array capable of simultaneously obtaining pixel values for R, green, and blue components, and a color interpolation method using the same.

Other objects of the present invention will be readily understood through the following description.

In order to achieve the above objects, according to an aspect of the present invention, there is provided a semiconductor device comprising: a first pixel region having multi-light amount pixel data with a large amount of light; And a second pixel area having low light amount pixel data due to a small amount of light, wherein the first pixel area and the second pixel area are alternately configured, and the first pixel area and the second pixel area are green filters. May be arranged in the same manner as the Bayer pattern, and a color filter array may be provided in which a red filter and a blue filter are alternately arranged between the green filters.

Preferably, the green filter arrangement of the odd-numbered and even-numbered lines in the first pixel area is the same, and positions of the red filter and the blue filter may be interchanged. The second pixel area has the same green filter arrangement as that of the odd-numbered line and the even-numbered line, and positions of the red filter and the blue filter may be interchanged.

The first pixel area and the second pixel area may be divided into a row unit, a column unit, or a block unit.

In addition, the multi-light amount pixel data and the low-light amount pixel data may have a difference in light amount depending on the long and short exposure time, the light sensitivity of each pixel, or the size of each pixel.

In order to achieve the above objects, according to another aspect of the present invention, a first pixel region having a multi-light pixel data by a large amount of light, and a second pixel region having a low-light pixel data by a small amount of light, A method of interpolating colors in an image sensor having a color filter array configured by alternating first and second pixel regions, the method comprising: (a) determining a color interpolation target pixel for color interpolation; (b) pixels of the color interpolation target pixel among red, green, and blue pixel data from pixels on the first pixel region and pixels on the second pixel region among pixels adjacent to the position of the color interpolation target pixel; Generating interpolation data for at least three pixel data except for data; And (c) calculating final interpolation data of red, green, and blue colors of the color interpolation target pixel by using the interpolation data and pixel data of the color interpolation target pixel. .

Preferably, the method may further include repeating steps (a) to (c) until color interpolation is completed for the image of one frame.

In the step (b), the interpolation data may be the average of the multi-light pixel data and the low-light pixel data that are closest to each color based on the position of the color interpolation target pixel.

Hereinafter, exemplary embodiments of a color filter array and a color interpolation method using the same will be described in detail with reference to the accompanying drawings. In describing the present invention, when it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. Numbers (eg, first, second, etc.) used in the description of the present specification are merely identification symbols for sequentially distinguishing identical or similar entities.

5 is a diagram illustrating a pixel array of an image sensor according to an exemplary embodiment of the present invention.

In the present invention, in order to simultaneously obtain an image generated by a relatively large amount of light and an image generated by a relatively small amount of light without a frame memory on one frame, each pixel may be formed by varying the exposure time in units of rows as shown in FIG. Each pixel uses a different signal accumulation time or uses different pixels with different light intensity or size. Alternatively, the structure of the color expressed in the pixel array, that is, the color filters may be arranged differently. Hereinafter, a description will be given of a pixel array in which exposure times are arranged differently in rows and a color filter array applied thereto. However, the exposure time may be differently arranged in units of columns or blocks (such as N × N squares) in addition to rows. Of course it can.

In this embodiment, L is an overexposed pixel line with a long exposure time, and S is a low exposed pixel line with a short exposure time. The overexposed pixel lines L and the low-exposed pixel lines S are preferably alternately arranged so as not to be continuous for each row. This is because it is advantageous that the overexposed pixels and the low-exposed pixels are arranged at the same or similar ratio in adjacent positions for proper color interpolation for the red, green, and blue components in each pixel constituting the pixel array.

6 illustrates a portion of a color filter array in accordance with a preferred embodiment of the present invention. The color filter array is constructed by repeatedly applying a portion shown in FIG.

Referring to FIG. 6, the color filter array includes overexposed pixel lines L1 and L2 having pixel data of an image having a long exposure time, and low exposed pixel lines S1 and S2 having pixel data of an image having a short exposure time. It includes. The overexposed pixel lines L1 and L2 and the low-exposed pixel lines S1 and S2 alternately form L1, S1, L2, S2, ... to form a pixel array (see FIG. 5) of the image sensor. That is, the overexposed pixel lines L1 and L2 are arranged in odd-numbered rows such as the first and third rows, and the low-exposed pixel lines S1 and S2 are arranged in even-numbered rows such as the second and fourth rows. Alternatively, the low-exposed pixel lines S1 and S2 may be arranged in odd-numbered rows and the over-exposed pixel lines L1 and L2 may be arranged in even-numbered rows.

The green filter G in the overexposed pixel lines L1 and L2 and the low-exposed pixel lines S1 and S2 are arranged in the same manner as the RGB Bayer pattern shown in FIG. That is, the number of green, red, and blue filters is 2: 1: 1 based on the entire pixel array.

And the green filter arrangement in the first overexposed pixel line L1 and the second overexposed pixel line L2 is the same, and in the first low-exposed pixel line S1 and the second low-exposed pixel line S2. The green filter array is the same. The overexposed pixel lines L1 and L2 and the low-exposed pixel lines S1 and S2 are preferably arranged with green filters in different columns.

It is assumed that the second and fourth columns of the first overexposed pixel line L1 are occupied by a green filter, the first column is occupied by a red filter, and the third column is occupied by a blue filter. In this case, the second and fourth columns of the second overexposed pixel line L2 are occupied by the green filter in the same manner as the first overexposed pixel line L1, the first column is the blue filter, and the third column is the red filter. Occupies so that the arrangement of the red filter and the blue filter is different from the first overexposed pixel line L1. This is to ensure that a suitable color interpolation is performed on the red and blue components of the overexposed pixel data in each pixel of the first low-exposed pixel line S1.

It is assumed that the first and third columns of the first low exposure pixel line S1 are occupied by a green filter, the second column is occupied by a blue filter, and the fourth column is occupied by a red filter. In this case, the first and third columns of the second overexposed pixel line S2 are occupied by the green filter in the same manner as the first low-exposed pixel line S1, the second column is a red filter, and the fourth column is a blue filter. So that the arrangement of the red filter and the blue filter is different from the first low exposure pixel line S1. This is to ensure that a suitable color interpolation is performed on the red and blue components of the overexposed pixel data in each pixel of the first low-exposed pixel line S1.

In the present invention, the distinction between the first pixel region due to the multi-light amount and the second pixel region due to the low light amount instead of the over-exposed pixel line or the low-exposure pixel line may be based on the sensitivity of the pixel itself constituting the corresponding area, in addition to the long and short exposure time. The size of the pixel is also possible. When the first pixel region is composed of pixels having a relatively high light sensitivity, and the second pixel region is composed of pixels having relatively low light intensity, a difference in the amount of light received in each region occurs even if the exposure time is constant. In addition, when the pixel size of the first pixel region is increased and the pixel size of the second pixel region is reduced, a difference in the amount of light received in each region occurs even within the same exposure time.

7 is a flowchart of a color interpolation method when applying a color filter array according to an exemplary embodiment of the present invention, and FIG. 8 is an exemplary view of a color filter array for explaining a color interpolation method. Here, R, B, and G are defined as pixel data of a red component, a blue component, and a green component of the overexposed pixel data having a long exposure time, and r, b, and g are red components of the low exposed pixel data having a short exposure time, It is defined by pixel data of a blue component and a green component.

Referring to FIG. 7, a color interpolation target pixel for color interpolation is first determined in step S710.

In operation S720, interpolation data corresponding to red, green, and blue components is obtained from pixels adjacent to the position corresponding to the color interpolation target pixel. At this time, the multi-light interpolation data by a large amount of light and the low-light interpolation data by a small amount of light are separately acquired for red, green, and blue, respectively.

Therefore, r, G, B, which are interpolation data in the multi-light amount pixels in the first pixel region for red, green, and blue, and r, which is interpolation data in the low-light amount pixels in the second pixel region, for one pixel; It is possible to obtain g and b, and to realize a wide dynamic range of the image sensor.

In this case, since the own pixel data of the color interpolation target pixel can be used as it is, color interpolation is obtained by obtaining interpolation data for three or more components (preferably five) except for the own pixel data. Color interpolation using six data of R, G, B, r, g, and b is possible in various ways such as obtaining an average value of each value and a method of obtaining an intermediate value.

The closest multi-light red pixels, multi-light blue pixels, multi-light green pixels, low-light red pixels, low-light blue pixels, and low-light green pixels are searched based on the positions of the color interpolation target pixels. If there are a plurality of pixels having the same distance, the average value is calculated. These values are set as multi-light interpolation data and low-light interpolation data at the position of the color interpolation target pixel.

In step S730, final interpolation data may be obtained for each color component by performing color interpolation by obtaining an average value or a median value for each of red, green, and blue colors in the multi-light interpolation data and the low-light interpolation data.

In step S740, steps S710 to S730 are repeated for all pixels of one frame, and when color interpolation is completed for all pixels of one frame, the process proceeds to step S750 to perform color interpolation for the next frame.

Hereinafter, the color interpolation method will be described in detail with reference to Examples. For example, the color interpolation target pixel has a green component. Assuming that the color interpolation target pixel is G33 in FIG. 8, interpolation data in the multi-light pixel and interpolation data in the low-light pixel are obtained by Equation 3 below.

<Equation 3>

Here, R33, G33, B33 are multi-light quantity pixel data of the pixel corresponding to the 3rd column position of the 3rd row L2, and r33, g33, b33 are low-light quantity pixel data of the same position.

As another example, the color interpolation target pixel has a red component. Assuming that the color interpolation target pixel is R34 in FIG. 8, interpolation data in the multi-light pixel and interpolation data in the low-light pixel are obtained by Equation 4 below.

<Equation 4>

Here, R34, G34 and B34 are multi-light quantity pixel data of pixels corresponding to the fourth column position of the third row L2, and r34, g34 and b34 are low-light quantity pixel data at the same position.

In another embodiment, the case in which the color interpolation target pixel has a blue component is similar to that of the red component described above, and thus a detailed description thereof will be omitted.

In the present invention, the color interpolation method described above may be repeatedly applied to implement a wide dynamic range for all pixels corresponding to one frame.

In the present invention, the multi-light amount pixel data and the low-light amount pixel data are generated with the short and long exposure time, the sensitivity of the pixel itself constituting the corresponding area, or the size of the pixel. If the exposure time of the first pixel region is increased and the exposure time of the second pixel region is shortened, a difference in light amount may occur. In this case, since the pixel data of the image having a short exposure time can be generated by exposing a predetermined time and blocking other light while generating the pixel data of the image having a long exposure time, there is no damage in the generation time of the pixel data.

In addition, when the first pixel area is composed of pixels having a relatively high light sensitivity, and the second pixel area is composed of pixels having a relatively low light intensity, a difference in the amount of light received in each area occurs even if the exposure time is constant. . In addition, when the pixel size of the first pixel region is increased and the pixel size of the second pixel region is reduced, a difference in the amount of light received in each region occurs even within the same exposure time.

As described above, the color filter array and the color interpolation method using the same are advantageous in the chip area of the image sensor by eliminating the need for a frame memory for synthesizing the image in the image sensor.

In addition, in outputting a composite image of two images, the image output speed may be the same as the one image output speed without deterioration.

In addition, the pixel values for the red (r), green (g), and blue (b) components of each pixel of the low light amount image and the red (R), green (G), blue ( B) By simultaneously acquiring and synthesizing pixel values for the components, a wide dynamic range of the image sensor can be realized.

In addition, a wide dynamic range may be realized by using a pixel having a low light quantity characteristic and a multi light quantity characteristic using a pixel structure having a different signal accumulation time or having a different light sensitivity characteristic or size.

Claims (8)

A first pixel region having multi-light amount pixel data by a large amount of light; And A second pixel area having low light amount pixel data by a low light amount, The first pixel area and the second pixel area are alternately configured, and the first pixel area and the second pixel area have a green filter arranged in the same manner as a Bayer pattern, and a red filter between the green filters. And a color filter array in which blue filters are alternately arranged. The method of claim 1, And a plurality of green filter arrays in odd-numbered and even-numbered lines in the first pixel area, and positions of the red and blue filters are interchanged. The method of claim 1, The second pixel area is the same as the green filter arrangement of the odd-numbered line and the even-numbered line, the position of the red filter and the blue filter are interchanged with each other. The method of claim 1, The first pixel area and the second pixel area are divided in a row unit, a column unit, or a block unit. The method of claim 1, And the light quantity pixel data and the low light quantity pixel data have a difference in light quantity according to a long and short exposure time, a light sensitivity of each pixel, or a size of each pixel. A first pixel region having multi-light pixel data by a large amount of light and a second pixel region having low-light pixel data by a small amount of light, wherein the first pixel area and the second pixel area are alternately configured A method of interpolating colors in an image sensor having a color filter array, the method comprising: (a) determining a color interpolation target pixel for color interpolation; (b) pixels of the color interpolation target pixel among red, green, and blue pixel data from pixels on the first pixel region and pixels on the second pixel region among pixels adjacent to the position of the color interpolation target pixel; Generating interpolation data for at least three pixel data except for data; And and (c) calculating final interpolation data of red, green, and blue of the color interpolation target pixel by using the interpolation data and pixel data of the color interpolation target pixel. The method of claim 6, (d) repeating steps (a) to (c) until color interpolation is completed for one frame of image. The method according to claim 6 or 7, The step (b) is a color interpolation method using the average of multi-color pixel data and low-light pixel data closest to each color around the position of the color interpolation target pixel as the interpolation data.

Images