KR20110079315A - Image processing pipeline - Google Patents

Abstract

PURPOSE: An image processing pipeline is provided to reduce the digital image processing area of a chip. CONSTITUTION: A bayer processing unit(110) performs a noise cancelation process. The bayer processing unit comprises an edge calculating part which extracts an edge value of a bayer image through an edge calculator. A demosaicing unit(120) amplifies the pixel value of the bayer image by performing demosaicing process. An RGB processing unit(130) comprising a color correction unit, a gamma correction unit, and an edge controller performs color correction function.

Description

Image processing pipeline

Embodiments relate to an image processing pipeline.

In the digital image processing pipeline for the conventional CMOS image sensor, the blocks responsible for noise reduction and edge enhancement functions have a specific size in proportion to the size of the input image to be processed due to the characteristics of the function. Each block of memory is required.

This requires an independent memory because the location of the noise reduction and sharpness enhancement blocks is not the same in the pipeline, and both blocks refer to neighboring pixels based on the pixels to be processed.

The required memory size increases in proportion to the local kernel support referenced by each block and the size of the overall input image.

The biggest drawback of this image processing structure is that the blocks for noise reduction and sharpness improvement each take a certain amount of individual memory, thereby increasing the area of the digital image processing area occupied by the chip, thereby lowering productivity as well as miniaturization. It is not suitable for market demands.

The embodiment provides an image processing pipeline that can reduce the area of the digital image processing region by reducing the number of block memories and improve the edge extraction structure to suppress the generation and amplification of false colors during image processing.

According to an embodiment, an image processing pipeline may include: a Bayer processing unit including an edge calcuating unit configured to perform noise reduction processing and extract an edge value of a Bayer image through an edge calculating unit; A demosaicing unit which amplifies pixel values of the Bayer image by performing demosaicing processing; And an RGB processing unit performing a color correction function, including a color correction unit, a gamma correction unit, and an edge controller unit.

According to the embodiment, the following effects are obtained.

First, since the blocks that perform the noise reduction and the sharpness improvement do not need to have all the block memories, the area of the digital image processing area occupied by the chip can be reduced.

Second, by introducing a new edge operator and improving the edge extraction structure, it is possible to suppress the generation and amplification of false color during image processing, and to freely adjust the sharpness free from the influence of the block to which the nonlinear function is applied.

Third, it is possible to minimize the size of the image processor chip and secure a competitive edge in the market by providing high quality images.

According to the embodiment with reference to the accompanying drawings

Hereinafter, in describing the embodiments, detailed descriptions of related well-known functions or configurations are deemed to unnecessarily obscure the subject matter of the present invention, and thus only the essential components directly related to the technical spirit of the present invention will be referred to. .

1 is a block diagram showing the structure of an image processing pipeline 100 according to an embodiment.

When the light signal is processed by the image sensor 10 as an analog signal, the ADC 20 converts the light signal into a digital signal and transmits the converted signal to the image processing pipeline 100.

Hereinafter, each component of the image processing pipeline 100 will be described.

(Bayer processing part)

The Bayer processing unit 110 is a component that performs a large noise reduction process, and includes a digital gain processing unit, a lens shading correction unit, a black level compensation unit, and a noise reduction unit. ) Edge, and edge calcuating unit.

At this time, data necessary for the noise canceling process is stored in the block memory 112.

In addition, the edge extracting portion includes an edge operator.

In general, the edge value extracted by the RGB processor 130 or the YUV processor 150 is obtained by applying a two-dimensional Laplacian operator of a G channel or a Y channel representing brightness of an image. In both cases, the value is obtained from a single channel (G or Y), whereas the Bayer image array does not provide a complete single channel. That is, since the color channel changes in units of pixels due to the characteristics of the Bayer array, when the operator of the same type is applied regardless of the position of the pixel, the extracted edge value is different. Thus, regular pattern noise occurs near the boundary.

2 is a view schematically illustrating an operation method of an edge operator according to an embodiment.

Referring to FIG. 2, the edge operator calculates an edge value by the following calculation method.

First, as shown in FIG. 2A, when the center pixel of the Bayer array is green, the next four green pixels arranged vertically and horizontally with respect to the center pixel are extracted, and "{4 * center pixel numerical value- ( The sum of the extracted pixel values)} ÷ 4 "is calculated.

Second, as shown in (b) of FIG. 2, when the center pixel of the Bayer array is not green, the first group of four green pixels which are diagonally abutted is extracted, and the fourth green pixels which are diagonally abutted with the extracted four green pixels. Eight green pixels of two groups are extracted. Next, the formula " (sum of green pixels of first group ÷ 4)-(sum of green pixels of second group ÷ 8) ”is calculated.

Such an edge operator can be applied to a single channel and is optimized to extract edge values symmetrically from Bayer array as well as free from pattern noise.

(Demosaicing department)

The edge value extracted by the edge operator is not applied to the demosaicing unit 120.

When the edge value is applied before the demosaicing unit 120, demosaicing is performed using the Bayer image with the edges highlighted, so that false color is generated near the boundary of the subject, and the RGB processor There is a problem that the false color is amplified through the color correction unit and the gamma correction unit of 130.

(RGB processing section)

The RGB processor 130 includes a color correction unit, a gamma correction unit, and an edge controller unit.

For reference, since the edge extracting unit is included in the Bayer processing unit 110, the RGB processing unit 130 does not require a block memory.

In the first case, the edge value extracted by the edge operator is not applied to the color correction.

3 is a view schematically illustrating the operation of the edge control unit.

In this case, as shown in FIG. 3, the edge control unit considers the processing time taken from the image processing pipeline 100 from the time point c at which the edge value is extracted to the time point d at which the edge value is actually applied. The register R is used to align the actual pixel value and the corresponding edge value at the same position.

In the second case, the extracted edge value is not applied to the gamma correction unit.

In this case, in consideration of the degree to which the value of the pixel transferred on the image processing pipeline 100 is converted by the gamma correction unit, a corresponding conversion is also required for the edge value.

An ideal way to improve the sharpness of a subject in an image is to apply an approximation of the effect of the gamma curve to that region, assuming that there is an area where the user can identify the object better by emphasizing a certain part of the subject.

The part defined as the constant area of the above-mentioned subject becomes an area having an edge value smaller than any suitable value rather than an area having a large edge value. This assumes that parts of the subject with large edge values are in themselves discernible, so it does not matter if the effects of the gamma curve described below are somewhat distorted, if the effects are qualitatively approximated only for the specific areas mentioned above. Is based on.

4 is a graph measuring the degree of conversion of the edge value to the gamma curve.

In the embodiment, a method of approximating the influence on the edge value of the gamma curve may be described as shown in FIG. 4.

In gamma correction, if the value of any input pixel is converted to a new output pixel by a predefined conversion curve, the edge value itself represents a relative difference with respect to the surrounding pixel with respect to the input pixel. It can be represented as a specific band centered.

Therefore, the influence of the gamma curve may be regarded as that the edge value represented by a specific band around the input pixel is converted to the new output band by the conversion curve.

In Fig. 4, "A" represents an input pixel value and "B" represents a result value (output pixel value) by gamma correction. Also, the output edge value corresponding to the output pixel value for the input edge value defined by "e1" in the graph is equal to the size of "e2".

The color space conversion unit 140 converts RGB data into YUV data.

The YUV processor 150 processes color noise filtering and scales the filtered image.

The output formatter adjusts and outputs the scaled image according to the format of an external display device.

The present invention has been described above with reference to the preferred embodiments, which are merely examples and are not intended to limit the present invention, and those skilled in the art to which the present invention pertains do not depart from the essential characteristics of the present invention. It will be appreciated that various modifications and applications not illustrated above are possible. For example, each component specifically shown in the embodiments of the present invention can be modified and implemented. And differences relating to such modifications and applications will have to be construed as being included in the scope of the invention defined in the appended claims.

1 is a block diagram showing the structure of an image processing pipeline 100 according to an embodiment.

2 is a view schematically illustrating a calculation method of an edge operator according to an embodiment.

3 is a view schematically illustrating the operation of the edge control unit.

Figure 4 is a graph measuring the degree of conversion of the edge value to the gamma curve.

Claims (15)

A Bayer processing unit for performing a noise removing process and including an edge calcuating unit for extracting an edge value of the Bayer image through an edge calculating unit; A demosaicing unit which amplifies pixel values of the Bayer image by performing demosaicing processing; And An image processing pipeline including an RGB processing unit for performing a color correction function, including a color correction unit, a gamma correction unit, and an edge controller unit. The method of claim 1, wherein the Bayer processing unit A digital gain processor; A lens shading correction unit; Black level compensation unit; And An image processing pipeline further comprising a noise reduction unit. The method of claim 1, wherein the Bayer processing unit An image processing pipeline, which stores data necessary for noise reduction processing in a block memory. The method of claim 1, wherein the edge operator When the center pixel of the array of Bayer images is green, An image characterized by extracting the next four green pixels arranged vertically and horizontally with respect to the center pixel, and calculating an expression of "{4 x center pixel value-(sum of extracted pixel values)} ÷ 4" Processing pipeline. The method of claim 1, wherein the edge operator In the arrangement of the Bayer images, when the center pixel is not green, the first group of four green pixels that are diagonally abutted is extracted, and the second group of eight green pixels that are diagonally abutted with the extracted four green pixels. Extracting and calculating a formula of "(sum of green pixels of first group ÷ 4)-(sum of green pixels of second group ÷ 8)". The method of claim 1, wherein the RGB processing unit And a block memory for processing the edge value. The method of claim 1, wherein the edge control unit And the extracted edge value is controlled to not be applied to image processing of the demosaicing unit. The method of claim 1, wherein the edge control unit And controlling the extracted edge value not to be applied to the color compensator. The method of claim 1, wherein the edge control unit In consideration of the time taken from the time point at which the edge value is extracted to the time point at which the edge value is applied, the edge value is aligned by using a register so that the actual pixel value and the corresponding edge value can be positioned at the same position. Image processing pipeline, characterized in that not applied to the government. The method of claim 1, wherein the edge control unit And the extracted edge value is not applied to the gamma correction unit. The method of claim 10, wherein the edge control unit While the extracted edge value is not applied to the gamma correction unit, And converting the edge value to approximate a degree (gamma correction curve) at which the pixel value of the Bayer image is converted by the gamma correction unit. The method of claim 1, When the light signal is processed into an analog signal in the image sensor, the ADC converts it into a digital signal, and the Bayer processing unit converts the digital signal into a Bayer image. The method of claim 1, And a color space conversion unit which receives the processed RGB data from the RGB processing unit and converts the RGB data into YUV data. The method of claim 13, And a YUV processor configured to process color noise filtering on the YUV data and to perform scaling on the filtered image. The method of claim 14, And an output formatter for adjusting and outputting the scaled image in accordance with a format of an external display device.

Images