KR100738189B1 - Image sensor and down scaling method in image sensor - Google Patents

Abstract

An object of the present invention is to provide an image sensor and an image size reduction method of an image sensor that can efficiently reduce an image in terms of power consumption or memory use, and solve a problem in which a step phenomenon occurs in an edge region of an image. According to an aspect of the present invention, there is provided an image sensor which processes a pixel data output from a pixel array to generate a Bayer image, comprising: a first reduction unit configured to receive the Bayer image and filter the horizontally and vertically and subsample the Bayer image; An image sensor including a second reduction unit that reduces an image by using bilinear interpolation is provided.

Image Sensor, CMOS Image Sensor, Bayer Image, Reduction, Bilinear Interpolation

Description

How to reduce image size of image sensor and image sensor {IMAGE SENSOR AND DOWN SCALING METHOD IN IMAGE SENSOR}

1 is a block diagram showing the configuration of a general CMOS image sensor.

2 is a block diagram showing the configuration of an image sensor according to an embodiment of the present invention;

3 is a block diagram showing a configuration of an image reduction unit shown in FIG. 2;

4 is a conceptual diagram illustrating a Bayer image shown in FIG. 2.

FIG. 5 is a conceptual diagram illustrating bilinear interpolation applied to the post-processing reduction portion shown in FIG. 3. FIG.

<Explanation of symbols for the main parts of the drawings>

10: pixel array 11: row decoder

12: column decoder 13: analog line buffer

14: timing controller 15: PGA

16 ADC 17 gamma correction unit

18: image reduction unit 19: color interpolation unit

20: color space converter 21: ISP converter

22: image format switching unit 23: interface unit

24: AWB 25: AEC

30: bias generation unit

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to semiconductor design techniques, and more particularly, to a method for reducing image size of an image sensor.

Recently, the demand of digital cameras is exploding with the development of video communication using the Internet. Moreover, the demand for small camera modules increases as the popularity of mobile communication terminals such as PDAs equipped with cameras, International Mobile Telecommunications-2000 (IMT-2000), Code Division Multiple Access (CDMA) terminals, etc. increases. Doing.

The camera module basically includes an image sensor. In general, an image sensor refers to a device that converts an optical image into an electrical signal. As such an image sensor, a charge coupled device (hereinafter referred to as a CCD) and a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor are widely used.

CCD has a complicated driving method, high power consumption, complicated process due to the large number of mask processes in the manufacturing process, and it is difficult to realize a signal processing circuit in a chip, making it difficult to make one chip. There are disadvantages. In contrast, CMOS image sensors are receiving more attention recently because of the monolithic integration of control, drive, and signal processing circuitry on a single chip. In addition, CMOS image sensors offer potentially lower cost than conventional CCDs due to low voltage operation and low power consumption, compatibility with peripherals, and the availability of standard CMOS fabrication processes.

1 is a view illustrating a configuration diagram of a general CMOS image sensor.

As shown in FIG. 1, the CMOS image sensor includes a pixel array, a row decoder, an analog line buffer, a column decoder, a timing control, and an output. Consists of formatter, output gamma analog / digital converter (ADC), white balance, color correction, color interpolation, video graphics adapter, control unit, CCD Unlike the structure of the present invention, each pixel is randomly accessed using an XY address method.

In general, an image sensor is sized to fit a display device in order to apply an output image to a display device of various applications such as a liquid crystal display (LCD), a plasma display panel (PDP), and an electroluminescence display (ELD). An image reduction process must be performed to convert the image. If an image sensor is manufactured separately according to the size of various display devices, it is industrially expensive. Therefore, it is effective to output images of various sizes with one image sensor.

In many cases, image reduction according to the prior art performs a function using a frame buffer in a backend chip. In this case, a much larger image signal is transmitted to the back end chip than the final output image, which is inefficient in terms of power consumption and memory usage. In addition, since the image sensor has a reduction function, pixel data of a Bayer pattern is reduced in the RGB stage after color interpolation, which is also inefficient in terms of power consumption and memory usage. This is because, after color interpolation, data increases from one channel to three channels. In addition, even when the pixel data of the Bayer pattern is directly reduced, the filtering may not be sufficiently performed, resulting in a step phenomenon in the edge region.

Accordingly, the present invention has been proposed to solve the above problems of the prior art, and has the following objects.

First, an object of the present invention is to provide an image sensor capable of performing image reduction efficiently in terms of power consumption or memory usage.

Second, another object of the present invention is to provide an image sensor that can solve the problem that a step phenomenon occurs in the edge region of the image.

Third, another object of the present invention is to provide an image reduction method using the image sensor.

According to an aspect of the present invention, there is provided an image sensor which processes a pixel data output from a pixel array to generate a Bayer image. The Bayer image is received and filtered in a horizontal and vertical direction to perform subsampling. According to an aspect of the present invention, there is provided an image sensor including a first reduction unit and a second reduction unit that reduces the subsampled Bayer image by using bilinear interpolation.

In addition, the present invention according to another aspect for achieving the above object, in the image size reduction method of the image sensor for reducing the image output from the pixel array, horizontal and vertical direction of the Bayer image output from the pixel array And sub-sampling by filtering, and reducing the sub-sampled Bayer image using bilinear interpolation.

In the present invention, in order to reduce and convert pixel data having a Bayer pattern (hereinafter referred to as a Bayer image), the input image is filtered by the horizontal / vertical direction in the preprocessing step and then subsampling to sub-sample the input image to 1/4, 1/16. Reduce the image size by a fixed multiple reduction factor of 4, such as times (1/4 ⁿ magnification), and apply 1 to 1/4 times the reduced image by giving appropriate weights using the surrounding four pixel values in the post-processing step. Reduced by the reduction rate. This makes it possible to produce a Bayer image of any size without stepping in the edge region.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the technical idea of the present invention. In addition, parts denoted by the same reference numerals throughout the specification represent the same elements performing the same function.

Example

2 is a block diagram illustrating an image sensor according to an exemplary embodiment of the present invention.

Referring to FIG. 2, an image sensor according to an exemplary embodiment of the present invention includes an image reduction unit 18 for reducing a Bayer image before color interpolation is performed on the Bayer image.

Specifically, it is as follows.

An image sensor according to an exemplary embodiment of the present invention may include a pixel array 10, a row decoder 11, a column decoder 12, an analog line buffer 13, a timing controller 14, a programmable gain amplifier (PGA), or an AGC ( Auto Gain Control (15), Analog Digital Converter (ADC) (16), Gamma Correction (17), Image Reduction (18), Color Interpolation (19), Color Space Conversion A color space converter 20, an ISP (Image Signal Processing) converter 21, an image formatting switch 22, an interface 23, AWB (Auto White Balance) ( 24), AEC (Auto Exposure Control) 25.

The pixel array 10 arranges N pixels horizontally and M vertically (N, M is a natural number) so as to maximize the response to light, and detects information about an image coming from the outside and outputs the analog signal. .

The row decoder 11 selects a row line of the pixel array 10 according to a row address input in response to a control signal of the timing controller 14.

The analog line buffer 13 receives, buffers, and stores pixel data of the pixel array unit 10 selected by the row decoder 11 for each row line, and sequentially variable amplifies in response to the control signal of the column decoder 12. Output to the denial PGA 15. That is, the analog line buffer 13 senses and stores the pixel data for each row line selected by the row decoder 11. The analog line buffer 13 is composed of several lines for use in color interpolation and image signal processing to be used at a later stage. The analog pixel data stored in the analog line buffer 13 responds to the control signal of the column decoder 12. The pixel data for each column line is output to the PGA 15 through an analog bus.

The PGA 15 amplifies and outputs a predetermined gain value when the pixel data sequentially output from the analog line buffer unit 13 for each column line is less than or equal to a threshold. For example, under low light conditions, the gain value is set higher than that of the normal environment.

The ADC 16 converts the pixel data output from the PGA 15 into a digital signal and outputs it. At this time, the ADC 16 outputs a Bayer image.

The gamma correction unit 17 performs gamma correction so that the gain of the Bayer image output from the ADC 16 has linearity. The gamma correction unit 17 may be installed after the color space converter 20 depending on the design.

The image reduction unit 18 reduces the Bayer image output from the gamma correction unit 17 to an arbitrary size to meet the needs of various users. Detailed description thereof will be described later.

The color interpolator 19 performs interpolation so that each pixel has an RGB value in the Bayer signal, and then outputs an RGB signal that is interpolated for each pixel.

The color space converter 20 converts the RGB signal into a YCbCr signal.

The AWB 24 automatically adjusts the saturation of the color according to the light source using the pixel average luminance of the frame.

The AEC 25 adjusts the degree of exposure of the pixel with the automatic exposure function by using the pixel average luminance of the frame.

The ISP converter 21 processes the YCbCr signal output from the color space converter 20.

The image format switching unit 22 converts the YCbCr signal into a format such as YUV so as to be suitable for display.

The interface unit 23 performs an interface between the timing controller 14, the controller (not shown), and the external system interface unit.

The timing controller 14, the controller, and the external system interface unit control the overall operation of the image sensor by using a finite state machine (FSM). In addition, it has a batch register, which allows programming of various internal operations and controls the operation of the entire chip according to the programmed information.

On the other hand, the color interpolation 19, the color correction unit (not shown), the gamma correction unit 17, the AWB 24, the AEC 25, and the like use the output values of the pixels stored in all the pixel line memory units to perform each operation. Perform.

In addition, the image sensor generates fixed pattern noise (FPN) due to an offset voltage due to a slight difference in the manufacturing process, and at each pixel of the pixel array 10 to compensate for the fixed pattern noise. It uses a CDS (Corelated Double Sampling) method that reads the reset signal, reads the data signal and outputs the difference.

Hereinafter, a Bayer image reduction method using the image reduction unit 18 shown in FIG. 2 will be described. FIG. 3 is a block diagram showing the configuration of the image reduction unit 18 shown in FIG.

Referring to FIG. 3, the image reduction unit 18 includes a preprocessing reduction unit 181 and a post-processing reduction unit 182.

The preprocessing reduction unit 181 reduces the input Bayer image at a fixed magnification such as 1/4, 1/16, and 1/64. At this time, the Bayer image is configured as color information as shown in FIG.

The preprocessing reduction unit 181 uses a low band filter having a sufficient tap according to the reduction ratio in the horizontal direction, and uses a low band filter having two taps in the vertical direction. Here, the tap refers to the number of adjacent pixels for obtaining an average value. As shown in the Bayer image shown in FIG. 4, since the odd lines and the even lines have different color structures, when filtering (reducing) in the vertical direction, lines having the same color structure are filtered. In order to reduce the memory usage, in order to reduce the memory usage, the vertical low band filtering requires one line information. In this case, a small amount of data is stored in the memory, and vertical filtering is performed using the same.

The post-processing reduction unit 182 reduces the input of the Bayer image reduced through the pre-processing reduction unit 181 as an input. Unlike the preprocessing reduction unit 181, the post-processing reduction unit 182 has an arbitrary reduction ratio of 1 to 1/4 to reduce the image to an arbitrary size. To provide an arbitrary reduction rate, the post-processing reduction unit 182 uses bilinear interpolation as shown in FIG. 5. This is an effective method in consideration of image quality and hardware cost by obtaining pixel values at positions to be obtained by appropriate weighting using the four color component values of the surroundings.

For example, when post-processing reduction is performed by applying a bilinear interpolation method to a sampling point g _s as shown in FIG. 5, the calculation method is as follows.

g _s = (A1, G1 + A2, G2 + A3, G3 + A4, G4) / sum,

Where A1 = w2 h2, A2 = w1 h2, A3 w2 h1, A4 w1 h1,

sum = A1 + A2 + A3 + A4.

Finally, the total reduction ratio SFtotal is determined as a product of the reduction ratio SFpre of the preprocessing reduction unit 181 and the reduction ratio SFpost of the post-processing reduction unit 182 as shown in Equation 2 below.

SFtotal = SFpre × SFpost

In the above, the present invention has been described in detail with reference to the exemplary embodiments, but for the purpose of description, the present invention may be variously applied according to application fields and hardware requirements. For example, the pretreatment reduction section can be expanded to use sufficient filter tabs in the horizontal direction, and in the vertical direction, the filter tabs can be extended to greater than two if additional memory is also available. In addition, it is also possible to improve the image quality of the output image after predicting the RGB value at each pixel position of the Bayer image in the post-processing reduction unit. In this case, as a prediction method, a method widely used for interpolation of a known image sensor may be applied.

As described above, although the technical idea of the present invention has been described in detail in the preferred embodiments, it should be noted that the above-described embodiments are for the purpose of description and not of limitation. In addition, those skilled in the art will understand that various embodiments are possible within the scope of the technical idea of the present invention.

As described above, according to the present invention, after performing low-band filtering in the horizontal / vertical direction through the pre-processing reduction unit, and performing bilinear interpolation in the post-processing reduction unit, sufficient filtering is performed to perform the boundary of the image, that is, the edge portion. It is possible to effectively reduce the step while preventing the phenomenon of to increase the application range of the image sensor.

Claims (10)

An image sensor which processes a pixel data output from a pixel array to generate a Bayer image, A first reduction unit which receives the Bayer image and filters the sub-samples in horizontal and vertical directions; And A second reduction unit configured to reduce the sub-sampled Bayer image by using bilinear interpolation; The first reduction unit reduces the Bayer image at a fixed magnification. delete The method of claim 1, The first reduction unit reduces the Bayer image by one of a magnification ratio of 1/4 ⁿ (n is an integer). The method of claim 3, wherein And the first reduction unit first filters the Bayer image in a horizontal direction, and then filters the Bayer image in a vertical direction by using a reduced pixel value to reduce memory usage. The method according to any one of claims 1, 3 and 4, The second reduction unit reduces the Bayer image reduced through the first reduction unit by using any one of magnifications of 1 to 1/4 magnification. In the image size reduction method of the image sensor for reducing the image output from the pixel array, Subsampling the Bayer image output from the pixel array in a horizontal and vertical direction; And Reducing the sub-sampled Bayer image by using bilinear interpolation; The subsampling may include reducing the Bayer image at a fixed magnification. delete The method of claim 6, The subsampling may include reducing the Bayer image to a magnification of any one of 1/4 ⁿ magnifications (where n is an integer). The method of claim 8, The subsampling may include filtering the Bayer image first in a horizontal direction and then filtering the Bayer image in a vertical direction using a reduced pixel value to reduce memory usage of the image sensor. The method according to any one of claims 6, 8 and 9, The reducing of the bilinear interpolation may include reducing the reduced Bayer image by using any one of magnifications between 1 and 1/4 magnifications.

Images