KR100738185B1 - Image sensor and image contour detection method in image sensor - Google Patents

Abstract

The present invention is to provide an image sensor that can more accurately represent the detection of the contour while reducing the hardware burden, the present invention for this purpose is to process the image data output from the pixel array to the digitized pixel data output image sensor A color interpolation unit comprising: a first kernel forming unit forming a first kernel using the digitized pixel data, a color interpolating unit performing color interpolation using a first kernel formed from the first kernel forming unit, and the color interpolating unit A luminance detector which detects a luminance signal from the pixel data outputted from the apparatus, a pattern detector that classifies the first kernel formed from the first kernel forming unit for each pattern, and a first kernel for each pattern detected by the pattern detector A second kernel forming unit to form a second kernel, and the second kernel forming unit It provides an image sensor including a contour detection unit for detecting the contour by applying a contour detection mask corresponding to the second kernel formed from the second kernel, and a contour correction unit for correcting the luminance signal by using the contour detected by the contour detection unit. .

 Image sensor, CMOS image sensor, contour, kernel

Description

Image sensor and image contour detection method of image sensor {IMAGE SENSOR AND IMAGE CONTOUR DETECTION METHOD IN IMAGE SENSOR}

1 is a block diagram showing the configuration of a general CMOS image sensor.

FIG. 2 is a diagram illustrating a moaic pattern of a general pixel. FIG.

3 is a block diagram showing the configuration of an image sensor according to an embodiment of the present invention;

4 shows a 5 × 5 kernel mosaic pattern in which green (G) is located at the center of the kernel.

FIG. 5 is a diagram illustrating a 5 × 5 kernel mosaic pattern in which red (R) is located at the center of the kernel; FIG.

FIG. 6 shows a 5 × 5 kernel mosaic pattern in which blue (B) is located in the center of the kernel. FIG.

<Explanation of symbols for the main parts of the drawings>

10: pixel array 11: row decoder

12: column decoder 13: analog line buffer

14: timing controller 15: PGA

16: ADC 17: first kernel forming unit

18 pattern detecting unit 19 second kernel forming unit

20: contour detection unit 21: color interpolation unit

22: luminance signal detection unit 23: contour correction unit

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to semiconductor design techniques, and more particularly, to an image signal processing method of an image sensor, and more particularly, to an image contour detection method of an image sensor.

Recently, the demand of digital cameras is exploding with the development of video communication using the Internet. Moreover, the demand for small camera modules increases as the popularity of mobile communication terminals such as PDAs equipped with cameras, International Mobile Telecommunications-2000 (IMT-2000), Code Division Multiple Access (CDMA) terminals, etc. increases. Doing.

The camera module basically includes an image sensor. In general, an image sensor refers to a device that converts an optical image into an electrical signal. As such an image sensor, a charge coupled device (hereinafter referred to as a CCD) and a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor are widely used.

CCD has a complicated driving method, high power consumption, complicated process due to the large number of mask processes in the manufacturing process, and it is difficult to realize a signal processing circuit in a chip, making it difficult to make one chip. There are disadvantages. In contrast, CMOS image sensors are receiving more attention recently because of the monolithic integration of control, drive, and signal processing circuitry on a single chip. In addition, CMOS image sensors offer potentially lower cost than conventional CCDs due to low voltage operation and low power consumption, compatibility with peripherals, and the availability of standard CMOS fabrication processes.

1 is a view illustrating a configuration diagram of a general CMOS image sensor.

As shown in FIG. 1, the CMOS image sensor includes a pixel array, a row decoder, an analog line buffer, a column decoder, a timing control, and an output. Consists of formatter, output gamma analog / digital converter (ADC), white balance, color correction, color interpolation, video graphics adapter, control unit, CCD Unlike the structure of the present invention, each pixel is randomly accessed using an XY address method.

There are many kinds of such image sensors, but as shown in FIG. 2, most of the image sensors have a moaic pattern having only one color filter in one pixel. An image sensor having such a pixel structure must interpolate to generate another color that it does not have with reference to a pixel at a later stage.

The contour of the image sensor having the lower pattern, that is, the Bayer pattern, is mostly detected by applying a mask by extracting only the luminance signal after performing the color interpolation process. In this case, not only the color interpolation process but also a memory for applying a mask, the burden is placed on hardware.

This problem was solved by detecting the contour using Bayer images of the green filter before color interpolation. As described above, in the case of detecting the contour using a Bayer image, a mask for detecting the contour has been linearly applied. However, in this case, the contour is determined depending on whether or not the green (G) pixel is present in the center of the current kernel. Information may or may not appear, resulting in outlines appearing in a grid.

Accordingly, an object of the present invention is to provide an image sensor capable of more accurately expressing contour detection while reducing hardware burden.

In addition, another object of the present invention is to provide a method for detecting an outline of an image sensor, which can detect contour information more efficiently by sharing information of an adaptive interpolation system.

According to an aspect of the present invention, there is provided an image sensor for processing analog pixel data output from a pixel array into digitized pixel data, and outputting the first kernel using the digitized pixel data. A color interpolation unit for color interpolation using a first kernel forming unit to be formed, a first kernel formed from the first kernel forming unit, a luminance detector for detecting a luminance signal from pixel data output from the color interpolation unit, A pattern detecting unit for classifying the first kernel formed from the first kernel forming unit for each pattern, a second kernel forming unit for forming a second kernel using the pattern-specific first kernel detected from the pattern detecting unit, and A contour for detecting the contour by applying an contour detection mask corresponding to the second kernel formed from the second kernel formation unit Using the detected contour through the line detection, and the contour line detector provides an image sensor including the contour correction portion for correcting said luminance signal.

According to another aspect of the present invention, there is provided a method for detecting an image contour of an image sensor that processes and outputs analog pixel data output from a pixel array into digitized pixel data. Forming a first kernel, classifying the first kernel by a pattern, forming a second kernel by using the first kernel by pattern, and a contour corresponding to the second kernel The present invention provides a method for detecting an image outline of an image sensor including applying a detection mask to detect an outline.

The present invention classifies the pattern of the current kernel into a smoothing region, a vertical contour region, a horizontal contour region, and other regions, and applies a mask for detecting contour information adaptively according to the pattern so that the contour information appears in a grid form. To solve this problem.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the technical idea of the present invention. In addition, parts denoted by the same reference numerals throughout the specification represent the same elements performing the same function.

Example

3 is a block diagram illustrating an image sensor according to an exemplary embodiment of the present invention.

Referring to FIG. 3, the image sensor according to the embodiment of the present invention further includes a first kernel forming unit 17, a pattern detecting unit 18, a second kernel forming unit 19, and a contour detecting unit 20.

The first kernel forming unit 17 receives the digital pixel data output from the ADC 16 and has a Bayer pattern having a 5 × 5 kernel as shown in FIG. 4 for color interpolation and contour detection (hereinafter, referred to as a first kernel). To form.

The pattern detector 18 receives kernel information of a Bayer pattern output from the first kernel forming unit 17 and generates gradient information in the same manner as in Equations 1 and 2 below. Equation 1 below corresponds to the case where the green G exists in the center as shown in FIG. 4, and Equation 2 corresponds to the case where the red R exists in the center as shown in FIG. 5. , Equation 3 corresponds to the case where blue (B) is present in the center as shown in FIG.

Vedge = │G31-G33│ <Th & │G35-G33│ <Th & │B32-B34│ <Th

Hedge = │G13-G33│ <Th & │G53-G33│ <Th & │R23-R43│ <Th

Cedge = │G22-G33│ <Th & │G44-G33│ <Th & │G24-G33│ <Th & │G42-G33│ <Th

Vedge = │R31-R33│ <Th & │R35-R33│ <Th & │G32-G34│ <Th

Hedge = │R13-R33│ <Th & │R53-R33│ <Th & │G23-G43│ <Th

Cedge = │B22-B44│ <Th & │B24-B42│ <Th

Vedge = │B31-B33│ <Th & │B35-B33│ <Th & │G32-G34│ <Th

Hedge = │B13-B33│ <Th & │B53-B33│ <Th & │G23-G43│ <Th

Cedge = │R22-R44│ <Th & │R24-R42│ <Th

In Equations 1 to 3, 'Vedge' denotes a vertical edge, 'Hedge' denotes a horizontal edge, 'Cedge' denotes a cross edge, and 'Th' denotes a set threshold. .

If the 'Vedge', 'Hedge', and 'Cedge' obtained through Equations 1 to 3 are all true (1), the kernel generated through the first kernel forming unit 17 is classified into a smoothed region pattern. Whereas 'Vedge' is false (0), while 'Hedge' is true, it is classified as a horizontal contour area pattern. Likewise, if 'Hedge' is false but 'Vedge' is true, it is classified as a vertical contour area pattern. The other patterns are classified into unknown patterns. In the above, true means '1' and false means '0'. That is, if | R31-R33 | <Th is true, it is '1', and if it is false, it is '0'.

Meanwhile, the pattern information classified in the above manner is input to the second kernel forming unit 19.

The second kernel forming unit 19 forms a second kernel in a manner as shown in equations (4) and (5). Here, Equation 4 corresponds to the case where the green G exists in the center, and Equation 5 corresponds to the case where the red R or blue B exists in the center.

As shown in Equations 4 and 5, since the contour detection is not performed in the flattened pattern, it is possible to prevent the detection of the wrong contour due to random noise, and to determine the unknown pattern. In this case, the data of the first kernel may be filtered using a low filter to detect pure contour information from which noise is removed.

The mask for contour detection may amplify random noise like a high band filter.

However, the image sensor according to the embodiment of the present invention not only removes the influence of random noise in an unknown pattern, such as a flattened pattern, but also can see the kernel formation process in the unknown pattern. As can be seen, the contour can be detected after suppressing the variation of the Gr value and the Gb by filtering using a low pass filter by adjusting the ratio of the Gr value of the G / R line and the Gb of the G / B line to 1: 1. have.

When the second kernel is formed in the same manner as in Equations 4 and 5, the contour detection unit 20 detects the contour information by selecting a suitable mask according to the pattern of the second kernel in the same manner as in Equation 6. .

The contour correction unit 23 increases the sharpness by correcting the contour of the image output from the luminance signal detector 22 using the contour information detected by the contour detector 20.

Meanwhile, the components not described in FIG. 3 will be described in detail.

The pixel array 10 arranges N pixels horizontally and M vertically (N, M is a natural number) so as to maximize the response to light, and detects information about an image coming from the outside and outputs the analog signal. .

The row decoder 11 selects a row line of the pixel array 10 according to a row address input in response to a control signal of the timing controller 14.

The analog line buffer 13 receives, buffers, and stores pixel data of the pixel array unit 10 selected by the row decoder 11 for each row line, and sequentially variable amplifies in response to the control signal of the column decoder 12. Output to the denial PGA 15. That is, the analog line buffer 13 senses and stores the pixel data for each row line selected by the row decoder 11. The analog line buffer 13 is composed of several lines for use in color interpolation and image signal processing to be used at a later stage. The analog pixel data stored in the analog line buffer 13 responds to the control signal of the column decoder 12. The pixel data for each column line is output to the PGA 15 through an analog bus.

The PGA 15 amplifies and outputs a predetermined gain value when the pixel data sequentially output from the analog line buffer unit 13 for each column line is less than or equal to a threshold. For example, under low light conditions, the gain value is set higher than that of the normal environment.

The ADC 16 converts the pixel data output from the PGA 15 into a digital signal and outputs it.

The color interpolator 21 performs color interpolation using the first kernel generated from the pattern detector 18.

The luminance signal detection unit 22 detects luminance from the RGB data output from the color interpolation unit 21.

In addition, although not shown, the apparatus further includes AWB (Auto White Balance), AEC (Auto Exposure Control), an image formatter, and an interface unit.

AWB automatically adjusts the saturation of color according to the light source by using the pixel average luminance of the frame. The AEC adjusts the exposure level of the pixel by using an automatic exposure function by using the pixel average luminance of the frame. The image formatter converts the YCbCr signal into a format such as YUV to be suitable for display. In addition, the interface unit performs an interface between the timing controller 14, the controller (not shown), and the external system interface unit. Meanwhile, the color interpolator 21, the color corrector (not shown), the gamma corrector (not shown), and the AWB AEC perform each operation using output values of pixels stored in all pixel line memory units.

The timing controller 14, the controller, and the external system interface unit control the overall operation of the image sensor by using a finite state machine (FSM).

In addition, the image sensor generates fixed pattern noise (FPN) due to an offset voltage due to a slight difference in the manufacturing process, and at each pixel of the pixel array 10 to compensate for the fixed pattern noise. It uses a CDS (Corelated Double Sampling) method that reads the reset signal, reads the data signal and outputs the difference.

As described above, although the technical idea of the present invention has been described in detail in the preferred embodiments, it should be noted that the above-described embodiments are only for explanation and not for limitation. In addition, those skilled in the art will understand that various embodiments are possible within the scope of the technical idea of the present invention.

As described above, according to the present invention, the following effects can be obtained.

First, the present invention does not detect contours for kernels classified as smoothing regions, and thus it is possible to make contour corrections resistant to random noise.

Further, the present invention can enable contour correction resistant to random noise by filtering the original signal using a low band filter when forming the second kernel in an unknown pattern.

In addition, the present invention can prevent the detection of wrong contour information from the variation of Gr / Gb by adjusting the ratio of Gr and Gb to 1: 1 when forming the second kernel in an unknown pattern.

Claims (24)

An image sensor for processing and outputting analog pixel data output from a pixel array as digitized pixel data, A first kernel forming unit forming a first kernel using the digitized pixel data; A color interpolation unit which color interpolates using a first kernel formed from the first kernel formation unit; A luminance detector for detecting a luminance signal from the pixel data output from the color interpolator; A pattern detection unit classifying the first kernel formed from the first kernel forming unit for each pattern; A second kernel forming unit which forms a second kernel using the first kernel for each pattern detected by the pattern detecting unit; An outline detection unit detecting an outline by applying an outline detection mask corresponding to the second kernel formed from the second kernel forming unit; And Contour correction unit for correcting the luminance signal by using the contour detected by the contour detection unit Image sensor comprising a. The method of claim 1, The first kernel is a 5 × 5 Bayer pattern. The method of claim 2, The pattern comprises a smoothing pattern, a vertical pattern and a horizontal pattern. The method of claim 3, wherein The pattern detecting unit classifies the pattern in the same manner as in Equation 1 when green G is present in the middle of the first kernel. [Equation 1] Vedge = │G31-G33│ <Th & │G35-G33│ <Th & │B32-B34│ <Th Hedge = │G13-G33│ <Th & │G53-G33│ <Th & │R23-R43│ <Th Cedge = │G22-G33│ <Th & │G44-G33│ <Th & │G24-G33│ <Th & │G42-G33│ <Th Here, 'Vedge' represents a vertical edge, 'Hedge' represents a horizontal edge, 'Cedge' represents a cross edge, and 'Th' represents a set threshold. The method of claim 3, wherein The pattern detecting unit classifies the pattern in the same manner as in Equation 2 when red (R) exists in the middle of the first kernel. [Equation 2] Vedge = │R31-R33│ <Th & │R35-R33│ <Th & │G32-G34│ <Th Hedge = │R13-R33│ <Th & │R53-R33│ <Th & │G23-G43│ <Th Cedge = │B22-B44│ <Th & │B24-B42│ <Th Here, 'Vedge' denotes a vertical edge, 'Hedge' denotes a horizontal edge, 'Cedge' denotes a cross edge, and 'Th' denotes a set threshold. The method of claim 3, wherein The pattern detecting unit classifies the pattern in the same manner as in Equation 3 when blue (B) exists in the middle of the first kernel. [Equation 3] Vedge = │B31-B33│ <Th & │B35-B33│ <Th & │G32-G34│ <Th Hedge = │B13-B33│ <Th & │B53-B33│ <Th & │G23-G43│ <Th Cedge = │R22-R44│ <Th & │R24-R42│ <Th Here, 'Vedge' denotes a vertical edge, 'Hedge' denotes a horizontal edge, 'Cedge' denotes a cross edge, and 'Th' denotes a set threshold. The method according to any one of claims 4 to 6, When the 'Vedge', 'Hedge' and 'Cedge' are all true, the first kernel is classified as a smoothing region pattern, and the 'Vedge' is false, but when the 'Hedge' is true, it is classified as a horizontal contour region pattern. When the 'Hedge' is false but 'Vedge' is true, it is classified as a vertical contour area pattern. An image sensor classified into an unknown pattern except for the smoothing area pattern, the horizontal contour area pattern, and the vertical contour area pattern. The method of claim 7, wherein And the second kernel forming unit does not form a second kernel with respect to the smoothing pattern. The method of claim 8, The second kernel forming unit filters the unknown pattern by using a low-band filter to match the ratio of the Gr value of the G / R line and the Gb of the G / B line by 1: 1 to form the second kernel. sensor. The method of claim 9, And the second kernel forming unit forms the second kernel in the same manner as in Equation 4 when green G is present in the middle of the first kernel. [Equation 4]

The method of claim 10, The second kernel forming unit forms the second kernel in the same manner as in Equation 5 when red (R) or blue (B) is present in the middle of the first kernel. [Equation 5]

The method of claim 11, The contour detection unit is an image sensor for detecting the contour in the same manner as in Equation 6. [Equation 6]

In the image contour detection method of the image sensor for processing and outputting the analog pixel data output from the pixel array to the digitized pixel data, Forming a first kernel using the digitized pixel data; Classifying the first kernel by a pattern; Forming a second kernel by using the first kernel for each pattern; And Detecting an outline by applying an outline detection mask corresponding to the second kernel Image contour detection method of the image sensor comprising a. The method of claim 13, And the first kernel is a 5 × 5 Bayer pattern. The method of claim 14, And the pattern comprises a smoothing pattern, a vertical pattern and a horizontal pattern. The method of claim 15, The detecting of the pattern may include classifying the pattern in the same manner as in Equation 1 when green G is present in the middle of the first kernel. [Equation 1] Vedge = │G31-G33│ <Th & │G35-G33│ <Th & │B32-B34│ <Th Hedge = │G13-G33│ <Th & │G53-G33│ <Th & │R23-R43│ <Th Cedge = │G22-G33│ <Th & │G44-G33│ <Th & │G24-G33│ <Th & │G42-G33│ <Th Here, 'Vedge' denotes a vertical edge, 'Hedge' denotes a horizontal edge, 'Cedge' denotes a cross edge, and 'Th' denotes a set threshold. The method of claim 15, The detecting of the pattern may include classifying the pattern in the same manner as in Equation 2 when red (R) exists in the middle of the first kernel. [Equation 2] Vedge = │R31-R33│ <Th & │R35-R33│ <Th & │G32-G34│ <Th Hedge = │R13-R33│ <Th & │R53-R33│ <Th & │G23-G43│ <Th Cedge = │B22-B44│ <Th & │B24-B42│ <Th Here, 'Vedge' denotes a vertical edge, 'Hedge' denotes a horizontal edge, 'Cedge' denotes a cross edge, and 'Th' denotes a set threshold. The method of claim 15, The detecting of the pattern may include classifying the pattern in the same manner as in Equation 3 when blue B is present in the middle of the first kernel. [Equation 3] Vedge = │B31-B33│ <Th & │B35-B33│ <Th & │G32-G34│ <Th Hedge = │B13-B33│ <Th & │B53-B33│ <Th & │G23-G43│ <Th Cedge = │R22-R44│ <Th & │R24-R42│ <Th Here, 'Vedge' denotes a vertical edge, 'Hedge' denotes a horizontal edge, 'Cedge' denotes a cross edge, and 'Th' denotes a set threshold. The method according to any one of claims 16 to 18, When the 'Vedge', 'Hedge' and 'Cedge' are all true, the first kernel is classified as a smoothing region pattern, and the 'Vedge' is false, but when the 'Hedge' is true, it is classified as a horizontal contour region pattern. When the 'Hedge' is false but 'Vedge' is true, it is classified as a vertical contour area pattern. And a pattern other than the smoothing area pattern, the horizontal contour area pattern, and the vertical contour area pattern, are classified into unknown patterns. The method of claim 19, The forming of the second kernel does not form a second kernel with respect to the smoothing pattern. The method of claim 20, The forming of the second kernel may include filtering the second kernel by using a low band filter by adjusting the ratio of the Gr value of the G / R line and the Gb of the G / B line to 1: 1 for the unknown pattern. Image contour detection method of the image sensor to be formed. The method of claim 21, The forming of the second kernel is an image contour detection method of an image sensor which forms the second kernel in the same manner as in Equation 4 when green G is present in the middle of the first kernel. [Equation 4]

The method of claim 22, The forming of the second kernel may include detecting an image contour of an image sensor that forms the second kernel in the same manner as in Equation 5 when red (R) or blue (B) is present in the middle of the first kernel. . [Equation 5]

The method of claim 23, The detecting of the contour may include detecting an contour in an image sensor according to Equation 6 below. [Equation 6]

Images