KR20110138583A - Apparatus and method of compensating saturation pixel using by neighbor pixel - Google Patents

Abstract

PURPOSE: A device for compensating a saturation pixel by using neighboring pixel is provided to solve the different brightness of each line of an image signal due to an error which is generated by the maximum expression range of an ADC through the compensation of a saturation pixel. CONSTITUTION: A saturation pixel determining unit(210) determines a saturation pixel in an image signal. A saturation value prediction unit(220) uses a neighboring pixel about the distinguished saturation pixel. The saturation value estimator predicts a saturation value. A saturation pixel compensation unit(230) processes compensation about the saturation pixel based on a predicted saturation value.

Description

Apparatus and method for compensating saturation pixels using surrounding pixels {APPARATUS AND METHOD OF COMPENSATING SATURATION PIXEL USING BY NEIGHBOR PIXEL}

In order to solve the problem of the line step, the present invention determines a saturated pixel, predicts a saturation value using pixels around the determined saturated pixel, and compensates for the saturated pixel based on the predicted saturation value. Saturation pixel compensation processing apparatus and method for processing.

Background Art [0002] In recent years, imaging devices have been used to acquire images using an electrical light receiving device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) instead of a film.

In the case of a CCD using an interpolation scanning complementary color filter, most of the CCD follows a signal specification of adding up and down pixel values.

Such a signal standard is a method of transmitting the pixels of a CCD configured by crossing complementary pixel values of CMYG (Cy, Mg, Ye, G, Color Filter Array) by adding up and down two rows of values.

This method can also be expected to increase the correlation (correlation) that is lacking in the interlaced scanning method.

However, since the method of summing two pixels, an error may accumulate due to the maximum expression range of the ADC when the ADC (Analog to Digital Convertor) has a limitation of conversion.

Accumulation of this error is very likely to cause a problem that the brightness value is different for each line when the combination of the luminance and the chrominance component in the ISP (Image Signal Processor).

Conventionally, the method of compensating the line step, which is a problem in which the brightness value is different for each line, compensates the step by compensating the data of the side not limited by the maximum value when it is determined that the value of the corresponding line is limited by the maximum value. Was used.

This method has the advantage that the value of the final output video signal keeps the same bit as the value of the input signal by processing the unsaturated data, but makes the value that has already been saturated become non-saturated. For this reason, there is a problem that the brightness of the saturated portion is lowered as a whole and the image is unnatural.

In accordance with another aspect of the present invention, a saturation pixel compensation processing apparatus includes a saturation pixel determination unit that determines a saturation pixel from an image signal, a saturation value prediction unit that predicts a saturation value using pixels around the determined saturation pixel, and the On the basis of the predicted saturation value, it may include a saturated pixel compensation unit for processing the compensation for the saturated pixel.

In the saturated pixel compensation processing method according to an embodiment of the present invention, the method of determining saturated pixels in an image signal photographed through a charge coupled device (CCD) using CMYG CFA (Cy, Mg, Ye, G, Color Filter Array) And predicting a saturation value using the surrounding pixels with respect to the determined saturated pixel, and processing compensation for the saturated pixel based on the predicted saturation value.

According to an embodiment of the present invention, the compensation of the saturated pixels may solve the problem of different brightness values for each line of the image signal due to an error caused by the maximum representation range of the ADC.

According to one embodiment of the present invention, by solving the line step problem caused by the saturation of the video signal, it is possible to provide an improved image signal.

1 is a diagram illustrating a digital photographing apparatus to which a saturated pixel compensation processing apparatus is applied according to an exemplary embodiment.
2 is a block diagram illustrating a saturation pixel compensation processing apparatus according to an exemplary embodiment of the present invention.
3 is a diagram illustrating an image signal photographed using a CCD of a CMYG CFA according to an embodiment of the present invention.
4 is a diagram illustrating an embodiment of determining a saturated pixel according to an embodiment of the present invention.
5 is a flowchart illustrating a saturation pixel compensation processing method according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In describing the present invention, when it is determined that detailed descriptions of related known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. Terminology used herein is a term used to properly express a preferred embodiment of the present invention, which may vary according to a user, an operator's intention, or a custom in the field to which the present invention belongs. Therefore, the definitions of the terms should be made based on the contents throughout the specification. Like reference numerals in the drawings denote like elements.

1 is a diagram illustrating a digital photographing apparatus 100 to which a saturated pixel compensation processing apparatus 140 is applied according to an embodiment of the present invention.

The digital photographing apparatus 100 according to an exemplary embodiment of the present invention may include a lens 110, a light receiving element 120, an analog to digital converter (ADC) 130, and a saturation pixel compensation processing device 140. .

The digital photographing apparatus 100 may acquire an image signal incident to the lens 110 through the light receiving element 120, and convert the obtained analog image signal into a digital form using the ADC 130.

The image signal converted into the digital form may be saturated according to the characteristics of the ADC 130, and the brightness value may be different for each line by specific pixels of the saturated image signal.

The saturated specific pixel of the image signal may be processed by the saturated pixel compensation processing apparatus 140 to be processed into an image signal that is close to the original image because it is compensated by using pixels in adjacent neighboring regions.

As a result, the digital photographing apparatus 100 according to an exemplary embodiment of the present invention may solve the problem of different brightness values for each line of the image signal due to an error caused by the maximum expression range of the ADC by compensating for the saturated pixel. By solving the line step problem caused by the saturation of the image signal, it is possible to provide an image signal of improved quality.

2 is a block diagram illustrating a saturation pixel compensation processing apparatus 200 according to an embodiment of the present invention.

The saturation pixel compensation processing apparatus 200 may include a saturation pixel discriminator 210, a saturation value predictor 220, and a saturation pixel compensator 230.

The saturated pixel determination unit 210 may determine a saturated pixel from an image signal input through the ADC.

Specifically, the saturated pixel determination unit 210 determines whether the current pixel is a saturated pixel by being limited to the maximum value of the ADC.

Looking at the pixel value of the image signal input to the light receiving element, for example, the light receiving element of the CMYG CFA, each filter component may be composed of a mixture of three primary colors (R, G, B) as shown in Equation 1 above. have.

[Equation 1]

Ye = R + G, Mg = B + R, Cy = G + B

In Equation 1, Ye stands for yellow, R stands for red, G stands for green, B stands for blue, Mg stands for magenta, and Cy stands for cyani. Can be.

In general, a photographing apparatus such as a digital camera may use a special filter to produce an image signal including three primary colors. Among them, a filter using a complementary color scheme of CMYG CFA may be used. Such a complementary filter may be Cy, Mg, Ye, G.

3 is a diagram illustrating an image signal photographed using a CCD of a CMYG CFA according to an embodiment of the present invention.

Basically, in case of CCD of CMYG CFA, each filter component is composed of a mixture of three primary colors (R, G, B) as shown in [Equation 1].

Therefore, the values of the S1 and S2 components of A1 and A2 in FIG. 3 are made of the same components as shown below.

[Equation 2]

S1A1 = G + Cy = G + (G + B)

S2A2 = Ye + Mg = (R + G) + (B + R)

S1A2 = Cy + Mg = (G + B) + (B + R)

S2A2 = Ye + G = (R + G) + G

As shown in Equation 2, in the pixel component of the CCD filter of the CMYG CFA, the value of R may be concentrated in one component of the S1 pixel, and the value of B may be concentrated in the S2 pixel.

Therefore, when the value of R or B is high, the probability that the output signal of the CCD saturates with respect to R for S1 and for B for S2 increases.

In general, the sum of Ye and Mg, which is the red component of the A1 line, is higher than the sum of Cy and Mg, which is the B component of the A2 line.

In other words, a CCD using CMYG CFA is likely to be susceptible to line stepping in R color. In addition, since the R value is concentrated in the S2 component of A1, the deviation of the generated value may be relatively low in the S1 value of A1. Therefore, the saturated pixel discriminating unit can find the next characteristic pattern at present by looking at the S1 and S2 pixel values of the A1 line and the S1 and S2 pixel values of the A2 line.

Referring to the image signal 310 of FIG. 3, the image signal 310 includes red pixels having R, G, and B components of 150, 50, and 50, respectively.

The image signal 320 is obtained by converting each component of the image signal 310 to CMYG, and the first row has pixel values of 100, 200, 200, and 50.

When the respective pixels are summed and displayed as S1S2 signals, as shown by reference numeral 330, the signals of S1 have values of 250 and 300, and the signals of S2 have values of 400 and 150. However, when the image signal 310 is digitally converted through an 8-bit ADC, each pixel value has a limit value of 255 as shown by reference numeral 340, and thus may face a line step problem in which brightness values differ from line to line. .

In other words, some pixels of reference numeral 340 have a value of 255, which is a limit of 8-bit ADC, instead of the pixel value that should be substantially changed, so that the brightness between the original image signal and the line is changed.

Referring to FIG. 2, when the specific pixel has a pixel having a higher value than the pixel located at the left and right, the saturated pixel determination unit 210 may determine that the specific pixel is saturated, and the saturated pixel Can be determined that the R component is saturated.

In addition, the saturated pixel determination unit 210 may determine that the specific pixel is saturated when the pixels located above and below the pixels positioned to the left and right of the specific pixel have a higher value. In other words, since the saturated pixel discrimination unit 210 is composed of 2G + R components in the case of G + Ye pixels in the upper and lower lines, it is determined that the specific pixel is saturated when it is larger than the value of the left and right pixels composed of 2G + B components. Can be.

In order to determine the saturation, the specific pixel value must be equal to or greater than the reference pixel value in order to determine the specific pixel as the saturated pixel.

A detailed embodiment of discriminating saturated pixels will be described in detail with reference to FIG. 4.

4 is a diagram illustrating an embodiment of determining a saturated pixel according to an embodiment of the present invention.

As shown by reference numeral 410, the current image signal 410 may be classified for each pixel.

When determining whether the pixel X is a saturated pixel in the center pixel of the image signal 420, that is, the image signal 410, it may be checked whether the pixel X has a predetermined pixel value or more.

If the pixel X has a predetermined pixel value or more, the saturated pixel determination unit 210 may determine whether the pixel X has a pixel having a higher value than the pixels (pixels D and E) positioned on the left and right sides.

Next, the saturated pixel determination unit 210 may check whether the pixels (pixels B and G) positioned above and below the pixels (pixels D and E) positioned to the left and right of the pixel X have a higher value. .

The saturated pixel discriminating unit 210 has a pixel having a higher value than the pixels (pixels D and E) in which the pixel X is located at the left and right, and the pixels (pixels B and the pixels positioned above and below the pixels (pixels D and E). If G) has a higher value, it can be determined that the pixel X is saturated.

In other words, as shown in FIG. 4, when the current video signal has a high R value, when the pixels of the peripheral A1 and A2 are viewed, all of the pixels except for S1 of the A1 line have R components.

Therefore, if the current pixel is the S2 pixel of the saturated A1 line, all the pixels of the upper and lower lines except for the sides of the self (pixel X) have a high value like the image signal 420.

Here, in the case of the A2 line, the S2 pixel and the S1 pixel have similar reactivity with respect to R, and therefore, there is no big problem even when comparing only the S2 pixel.

Therefore, the condition for determining the saturated pixel may be determined by the discriminant of Equation (3).

&Quot; (3) "

neighbor_ condition = ((X> E) &(X> D)) &((B> D) &(B> E)) &((G> D) &(G> E))

Here, X, B, D, E, and G mean pixel X, pixel B, pixel D, pixel E, and pixel G.

In addition, since the pixel for determining saturation (pixel X) should be limited to the maximum value, the input value should be near the maximum value of the ADC used. Therefore, in the final saturated pixel, the current input pixel value has a predetermined value or more as shown in [Equation 4], and the relationship with the neighboring pixels must satisfy the condition as shown in [Equation 3]. As a result, the saturation pixel determination unit 210 finds the pixel value at which the Saturation_pixel value of Equation 4 is 1.

&Quot; (4) "

Saturation_ condition = (X> reference pixel value) & neighbor _ condition

Here, 'neighbor_condition' may be determined by Equation 3.

Referring to FIG. 2, the saturation value predictor 220 may predict the saturation value using surrounding pixels with respect to the determined saturated pixel.

In other words, the saturation value predictor 220 may generate a saturation value prediction value for the saturation pixel by linearly interpolating pixel values of a plurality of pixels adjacent to the saturation pixel.

When the saturated pixel determination unit 210 determines whether the pixel value is saturated, the saturation value predictor 220 needs to predict the original pixel value for the saturated pixel.

The value of the A1 line S2, that is, the value of the saturated pixel (pixel X) is composed of the value of Ye + Mg, which can be interpolated and predicted from the values of the surrounding pixels.

The value of the neighboring pixel for the saturated pixel (pixel X) is composed of (G + Ye), (Mg + Cy), (G + Cy), and the prediction of the corresponding value is shown in [Equation 5]. Can be calculated.

[Equation 5]

Ye + Mg = (G + Ye) + (Mg + Cy)? (G + Cy)

As can be seen from [Equation 5], the Ye + Mg value can be predicted by the difference of the (G + Cy) value of the left and right pixels from the value of (G + Ye) + (Mg + Cy) of the upper and lower lines. Thus, in order to interpolate up, down, left, and right, by applying the linear interpolation method can be obtained [Equation 6].

&Quot; (6) "

X '= [{B + (A + C) / 2} {G + (F + H) / 2}-(D + E)] / 2

X ′ calculated through Equation 6 may be interpreted as a saturation value prediction value for the saturated pixel by linearly interpolating pixel values of a plurality of neighboring pixels adjacent to the saturated pixel.

The saturation value predicted value X ′ calculated by the saturation value predictor 220 may be used to compensate the saturated pixel (pixel X).

The saturated pixel compensator 230 may process compensation for the saturated pixel based on the predicted saturation value.

In other words, compensation may be performed on the saturated pixel by using the pixel value of the saturated pixel and the generated saturation prediction value X ′.

The predicted saturation value may be replaced by the saturation pixel compensation unit 230 according to the current input value and a predetermined compensation ratio as shown in Equation 7 below.

[Equation 7]

X "= α × X + (1? Α) × X '

As shown in [Equation 7], by the value of a, the final output pixel value X " value can be calculated by mixing the saturated pixel value X of the saturated pixel with the predicted saturation value X 'of the predicted pixel, The closer a is to 1, the closer to the first input image.

As a result, according to an embodiment of the present invention, by compensating for the saturated pixel of the image signal, it is possible to solve the problem that the brightness value is different for each line of the image signal due to an error caused by the maximum expression range of the ADC, and the saturation of the image signal By solving the line step problem according to the present invention, it is possible to provide an improved image signal.

5 is a flowchart illustrating a saturation pixel compensation processing method according to an embodiment of the present invention.

In the saturation pixel compensation processing method according to an embodiment of the present invention, an analog to digital (AD) conversion may be performed by receiving an image signal (step 510).

The image signal according to an embodiment of the present invention may be interpreted as an image signal photographed through a charge coupled device (CCD) using CMYG CFA (Cy, Mg, Ye, G, Color Filter Array).

In the saturated pixel compensation processing method according to an embodiment of the present invention, the saturated pixel may be determined from the image signal (step 520).

In order to determine the saturation pixel, the specific pixel to determine the saturation has a higher value of the pixel than the pixels located on the left and right, and the pixels located above and below the pixel located on the left and right of the specific pixel have a higher value. When the value of a specific pixel is equal to or greater than a predetermined reference pixel value, it may be determined that the specific pixel is a saturated saturated pixel.

In the saturation pixel compensation processing method according to an embodiment of the present invention, the saturation value may be predicted using the surrounding pixels with respect to the determined saturation pixel (step 530).

In order to predict the saturation value, the saturation pixel compensation processing method according to an embodiment of the present invention may generate a saturation value prediction value for the saturation pixel by linearly interpolating pixel values of a plurality of pixels adjacent to the saturation pixel. .

The saturated pixel compensation processing method according to an embodiment of the present invention may process the compensation for the saturated pixel based on the predicted saturation value (step 540).

In other words, the saturated pixel compensation processing method according to an exemplary embodiment may process the saturated pixel by using the pixel value of the saturated pixel and the generated saturation value prediction value.

The saturation pixel compensation processing method according to an embodiment of the present invention may be implemented in the form of program instructions that may be executed by various computer means and may be recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

200: saturated pixel compensation processing device 210: saturated pixel discriminating unit
220: saturation value prediction unit 230: saturation pixel compensation unit

Claims (12)

A saturation pixel determination unit that determines a saturation pixel from an image signal;
A saturation value predictor for predicting a saturation value with respect to the determined saturated pixel by using neighboring pixels; And
A saturation pixel compensator for processing compensation for the saturation pixel based on the predicted saturation value
Saturated pixel compensation processing device comprising a. The method of claim 1,
The image signal is saturated pixel compensation processing apparatus characterized in that the image is taken through a charge coupled device using a CMYG CFA (Cy, Mg, Ye, G, Color Filter Array). The method of claim 2,
The saturated pixel determination unit,
And when the specific pixel has a pixel having a higher value than the pixel located at the left and right, determining that the specific pixel is saturated. The method of claim 3,
The saturated pixel determination unit,
And the specific pixel determines that an R (Red) component is saturated. The method of claim 3,
The saturated pixel determination unit,
And determining that the specific pixel is saturated when the pixels located above and below the pixels positioned to the left and right of the specific pixel have a higher value. The method of claim 5,
The saturated pixel determination unit,
And determining that the specific pixel is saturated when the value of the specific pixel is equal to or larger than a predetermined reference pixel value. The method of claim 1,
The saturation value prediction unit,
And linearly interpolating pixel values of a plurality of pixels adjacent to the saturated pixel to generate a saturation value prediction value for the saturated pixel. The method of claim 7, wherein
The saturated pixel compensator,
And processing the compensation for the saturated pixel by using the pixel value for the saturated pixel and the generated saturation prediction value. Determining a saturated pixel from an image signal photographed through a charge coupled device using a CMYG CFA (Cy, Mg, Ye, G, Color Filter Array);
Predicting a saturation value with respect to the determined saturated pixel using surrounding pixels; And
Processing compensation for the saturated pixel based on the predicted saturation value
Saturated pixel compensation processing method comprising a. 10. The method of claim 9,
The determining of the saturated pixel may include:
A specific pixel has a higher value pixel than a pixel positioned left and right, a pixel positioned above and below a pixel positioned left and right of the specific pixel has a higher value, and the value of the specific pixel is equal to or greater than a predetermined reference pixel value. If it is determined that the specific pixel is saturated. 10. The method of claim 9,
Predicting the saturation value,
Linearly interpolating pixel values of a plurality of pixels adjacent to the saturated pixel to generate a saturation prediction value for the saturated pixel,
Processing the compensation for the saturated pixel,
Processing compensation for the saturated pixel using the pixel value for the saturated pixel and the generated saturation prediction value
Saturated pixel compensation processing method comprising a. A computer-readable recording medium having recorded thereon a program for performing the method of any one of claims 9 to 11.

Images