KR101241739B1 - Method of processing video, computer readable recording medium and device of processing video therefor - Google Patents

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method and a computer-readable storage medium for the same, comprising: performing color interpolation on a plurality of pixels configured as a Bayer array and outputting RGB color data, and performing the color interpolation It is determined whether the error candidate point is one pixel among the pixels, the error boundary direction is determined based on the error candidate point, and filtering is performed using the determined error boundary direction. In simple ways, the negative color of the Bayer array filter can be reduced.

Description

FIELD OF PROCESSING VIDEO, COMPUTER READABLE RECORDING MEDIUM AND DEVICE OF PROCESSING VIDEO THEREFOR

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method, a computer readable storage medium and an image processing apparatus for the same, and in particular, an image processing for correcting a false color frequently occurring at an edge of a subject in a simple and rapid manner. Method and the like.

Bayer color filter array is a name after inventor Bryce Bayer as an array of RGB color filters arranged on a square grid of a photo sensor. The Bayer filter is composed of an array of mosaic shapes, and the RGB color filters are distributed on the array in a ratio of 25%, 50%, and 25%, so that RGB colors can be detected pixel by pixel. In addition, since one pixel can determine only one color of RGB according to a filter, its own color can be obtained by color interpolation combining RGB values of the pixel and other pixels around the pixel. . As a color interpolation technique, a demosaicing algorithm or the like may be used.

On the other hand, in the case where the color change suddenly occurs in a specific direction of the Bayer filter, even if the color is interpolated, the color of the pixel itself may not be properly obtained, for example, a false color may occur. In addition, the negative color may be caused by a sudden change in the color of the original image, a lack of resources of the image sensor, and an inaccuracy of the post-processing technique.

Negative colors can be reduced by processing such as complementing color interpolation processing or adding post-processing, but this requires more resources and time.

The present invention is to provide an image processing method and the like that can reduce the negative color through a simple post-processing of the image after color interpolation to solve the above problems.

An image processing method according to an embodiment of the present invention includes performing color interpolation on a plurality of pixels configured as a Bayer array to output RGB color data; Determining whether the pixel is an error candidate point for one of the plurality of color interpolated pixels; Determining an error boundary direction based on the error candidate point; And performing filtering using the determined error boundary direction.

In a computer-readable storage medium including an instruction code for processing an image by RGB color data output from a plurality of pixels according to an embodiment of the present invention, the instruction code is a processor, the processor array is configured Color interpolation is performed on a plurality of pixels outputting RGB color data, and it is determined whether one of the plurality of pixels having color interpolation is an error candidate point, and an error is performed based on the error candidate point. And determining a boundary direction and performing filtering by using the determined error boundary direction.

An image processing apparatus according to an embodiment of the present invention includes a color interpolation unit configured to perform a color interpolation on a plurality of pixels configured as a Bayer array to output RGB color data; An error point determination unit that determines whether an error candidate point is included in one pixel of the plurality of color interpolated pixels; A boundary direction determination unit for determining an error boundary direction based on the error candidate point; And a filtering unit which performs filtering using the determined error boundary direction.

According to the present invention, the image quality of an image can be easily improved with a relatively small resource.

1 shows an example of a Bayer filter array.
2 (a) shows an example of the actual image of the subject, and FIG. 2 (b) shows an example of the color interpolated image.
3 is a flowchart of an image processing method according to an embodiment of the present invention.
4 is a block diagram of a configuration of an image processing apparatus according to an embodiment of the present invention.

In the description of the embodiments, it is to be understood that each layer (film), area, pattern or structure may be referred to as being "on" or "under" the substrate, each layer Quot; on "and" under "are intended to include both" directly "or" indirectly " do. Also, the criteria for top, bottom, or bottom of each layer will be described with reference to the drawings.

The thickness and size of each layer in the drawings are exaggerated, omitted, or schematically shown for convenience and clarity of explanation. In addition, the size of each component does not necessarily reflect the actual size.

In addition, the image processing apparatus according to the present invention may be implemented as a camera, a digital camera, a network camera, a computer, a portable mobile terminal, a laptop, or a smartphone capable of recognizing and storing an image, or may be configured and mounted on the device. It can include any device that can be recognized.

1 shows an example of a Bayer filter array. Referring to FIG. 1, the Bayer filter array 100 may be configured of a plurality of pixel arrays including 50% green filters, 25% red and blue filters on a square array. Each pixel may include one of red, green, and blue filters to output only corresponding color data. For example, R11, R13, R15, R31, R33, R35, R51, R53, and R55 pixels output red color data, and G12, G14, G21, G23, G25, G32, G34, G41, G43, G45, G52, G54 outputs green color data, while B22, B24, B42 and B44 output blue color data. Color data output from each pixel may be processed into full color data by color interpolation and reproduced as an image. While 25 pixels are illustrated in FIG. 1, the Bayer filter array can be further expanded.

In the color interpolation process, a large number of irregular colors are generated when a pixel outputting red color data, for example, when interpolating a blue color B33 at R33, or a pixel outputting blue color data, such as a red color R22 at B22. It may be the case of interpolation. In view of the advantages, the present invention proposes a method of improving image quality by performing false color suppression on a pixel which is original color data output from a pixel, which is red color data or blue color data.

2 (a) shows an example of the actual image of the subject, and FIG. 2 (b) shows an example of the color interpolated image. As shown in FIG. 2 (b), as the color value between the railing 10 and the background remains different, a negative color may occur in the color of the railing, resulting in a distorted color than the original color. In the present invention, in consideration of occurrence of distorted colors in the same direction as that of the handrail 10, the present invention considers this direction as an edge direction and reduces noise around the error boundary direction.

Hereinafter, a case in which negative color suppression is performed when interpolating a blue color B22 in the R33 pixel of FIG. 1 will be described as an example. In performing negative color suppression, color interpolation must be assumed for a plurality of pixels outputting RGB color data of the Bayer array 100. As the color interpolation technique, nearest-netgbor interpolation, bilinear interpolation, bicubic interpolation, spline interpolation and renczos resampling can be used.

First, it is determined whether the R33 pixel is an error candidate point among the plurality of color interpolated pixels. In the present specification, the error candidate point may be a pixel in which the generated full color image is determined to be noisy when color full interpolation is performed on the corresponding pixel.

In determining the error candidate point, original green color data arranged around R33 is used as in Equation 1.

fcd = ｜ G32 + G34-G23-G43 ｜,

if fcd> T, R33 = false color candidate, -------- (Equation 1)

Referring to Equation 1, the difference value between the green color data G23 and G43 arranged in the vertical direction around the R33 pixel and the green color data G32 and G24 arranged in the horizontal direction is used. When the absolute value of the difference value is larger than the set reference value T, the pixel R33 is determined as an error candidate point. The processing in the case where the pixel R33 is determined as an error candidate point will be described below.

Next, an error boundary direction is determined based on the pixel R33 which is the error candidate point. The error boundary direction described in this specification indicates a direction in which the color data is greatly different around the pixel, and in the Bayer array, the error boundary direction is changed by using a point in which the color data is significantly different in the vertical or horizontal direction due to its characteristics. Determining whether it is vertical or horizontal.

First, the difference value dv in the vertical direction and the difference value dh in the horizontal direction are calculated for the original color data of the pixel arranged around the R33 pixel, which is an error candidate point.

dv and dh can be obtained as in Equations 2 to 9 below.

dv2 = | G12-G32 | + 2 * | B22-B42 | + | G32-G52 |

dv3 = | R13-R33 | + 2 * | G23-G43 | + | R33-R53 |

dv4 = | G14-G34 | + 2 * | B24-B44 | + | G34-G54 | --- Equation 4

dv = dv2 + 2 * dv3 + dv4 ----------------------------------- Equation 5

dh2 = | G21-G23 | + 2 * | B22-B24 | + | G23-G25 |

dh3 = | R31-R33 | + 2 * | G32-G34 | + | R33-R35 |

dh4 = | G41-G43 | + 2 * | B42-B44 | + | G43-G45 |

dh = dh2 + 2 * dh3 + dh4 ------------------------------------ Equation 9

At this time, the difference value dv in the vertical direction and the difference value dh in the horizontal direction are compared, and if dh is larger than dv, the error boundary direction becomes a vertical direction, otherwise the error boundary direction becomes a horizontal direction. That is, the larger the difference value in a specific direction, the larger the difference between both color data in a specific direction centering on the error candidate point, and an error boundary direction as shown in FIG. 2 (b) may occur along the orthogonal direction of the specific direction. Because it means.

Once the error boundary direction is determined, it can be used to filter the interpolation color for the corresponding error candidate point. For example, filtering of the color B33 color interpolated in the pixel R33 may be performed by a median filter. For example, when the error boundary direction is determined in the vertical direction, the interpolation color B33 may be filtered as in Equation 10. Referring to Equation 10, interpolation color B33 and value1 and value2 may be input to the entry of the median filter function median () as correction data for the error boundary direction. The result of the median filter function is output as corrected interpolation color data. Correction data value1 and value2 are correction data for the vertical direction about error candidate point R33.

median (B33, value1, value2)

value1 = {-G12-G32 + 2 * B22 + G23 + 2 * B24-G14-G34} / 4

value2 = {-G32-G52 + 2 * B42 + G43 + 2 * B44-G34-G54} / 4

Equation 10

On the other hand, when the error boundary direction is the horizontal direction, the interpolation color B33 may be filtered and corrected as in Equation 11.

median (B33, value3, value4)

value3 = {-G21-G23 + 2 * B22 + G32 + 2 * B42-G41-G43} / 4

value2 = {-G23-G25 + 2 * B24 + G34 + 2 * B44-G43-G45} / 4

Equation 11

In Equation 11, value3 and value4 are correction data for the horizontal direction.

In the present specification, three entry values are input as the entry of the median filter. However, the present invention is not limited thereto, and other data values that can be input as correction data may be input, and the number of entries is not limited.

Meanwhile, in FIG. 1, negative color suppression is performed mainly on a pixel to which red color data is output, but negative color suppression may be similarly performed to a pixel to which blue data is output.

In addition, although the above-described correction of the interpolation color by filtering of the interpolation color is performed only in the pixel R33, the correction of the interpolation color must also be performed on all the pixels included in the image sensor, so that pixels other than R33, for example, B22, Also for B24, B42, B44, R11, R13, R15, ..., the entire image can be corrected by performing the above-described determination of the error candidate point, determination of the error boundary direction, and filtering.

3 is a flowchart of an image processing method according to an embodiment of the present invention.

In step S11, the pixel to be corrected is selected. As described above, since the color correction is performed in whole or in part with respect to the plurality of pixels, the image processing method according to the present invention may be performed sequentially or simultaneously with respect to each pixel, and the pixel to be corrected is performed every time. This can be selected by any method.

In step S12, it is determined whether the corresponding pixel is an error candidate point. The determination of the error candidate point may use the value of the original color data arranged around the pixel as described above.

In step S13, when the pixel is an error candidate point, an error boundary direction is determined based on the error candidate point. The error boundary direction can be, for example, a vertical direction or a horizontal direction.

In step S14, filtering on the interpolation color of the error candidate point is performed using the error boundary direction. In the filtering, a median filter that employs interpolation color data of the error candidate point and at least one correction data as an entry may be used. The filtering result is a corrected value of the interpolation color.

In step S15, when filtering of one pixel is completed, it is determined whether there are more error candidate points to be corrected. When the determination of the error candidate point is all completed, the color correction is completed. If the determination of the error candidate point is not completed, the process may be repeated by returning to step S11 and selecting the target pixel again.

4 is a block diagram of a configuration of an image processing apparatus according to an embodiment of the present invention. Referring to FIG. 4, the image processing apparatus 200 according to the present invention includes a color interpolation unit 201 configured to perform a color interpolation on a plurality of pixels configured as a Bayer array to output RGB color data. An error point determination unit 202 for determining whether an error candidate point is one pixel among the plurality of color interpolated pixels, a boundary direction determination unit 203 for determining an error boundary direction based on the error candidate point, and the determined It may include a filtering unit 204 for performing filtering using the error boundary direction.

In addition, the image processing apparatus 200 may determine whether the determination of the error candidate point for the plurality of pixels is completed, select another pixel among the plurality of pixels, and determine the error candidate point for the selected pixel. The control unit may further include a control unit 205 which controls to repeat the determination and filtering of the error boundary direction.

The color interpolator 201 may receive an RGB color signal recognized by a plurality of pixels and predict the full color of each pixel using each color signal, and includes a color filter array (CFA) for this purpose. can do.

The error point determination unit 202 performs a series of operations performed in step S12 of FIG. 3, and the boundary direction determination unit 203 performs a series of operations performed in step S13 of FIG. 3. The filtering unit 204 may perform a series of operations performed in step S14.

Before the operation of the error point determining unit 202, the control unit 205 selects a pixel that is a determination target of an error candidate point, and when filtering of the pixel is completed or is determined to be not an error candidate point, A series of actions can be performed to determine whether to perform the judgment. In order to perform such an operation, the controller 205 may be configured as a system on chip processor.

Meanwhile, the image processing apparatus 200 may, in addition to the components illustrated in FIG. 4, have an image sensor (CCD / CMOS) for recognizing an image, an optical filter such as an RGB filter mounted at each pixel, and a network for connecting to a network. It may further include a module, a decoder or a memory.

In addition, although the color interpolation unit 201, the error point determination unit 202, the boundary direction determination unit 203, and the filtering unit are configured independently of the control unit 205 in FIG. 4, these are included in the control unit 205. The above-described operations may be performed.

As described above, according to an embodiment of the present invention, it is possible to provide an image processing method capable of reducing an irregular color by a simple method.

The features, structures, effects and the like described in the embodiments are included in at least one embodiment and are not necessarily limited to only one embodiment. Furthermore, the features, structures, effects, and the like illustrated in each embodiment may be combined or modified with respect to other embodiments by those skilled in the art to which the embodiments belong. Accordingly, the contents of such combinations and modifications should be construed as being included in the scope of the embodiments.

In addition, the above description has been made with reference to the embodiments, which are merely examples and are not intended to limit the embodiments, and those skilled in the art to which the embodiments belong may not be exemplified above without departing from the essential characteristics of the embodiments. It will be understood that various modifications and applications are possible. For example, each component specifically shown in the embodiments can be modified and implemented. And differences relating to such modifications and applications will have to be construed as being included in the scope of the embodiments set forth in the appended claims.

Claims (17)

delete Performing color interpolation on a plurality of pixels configured as a Bayer array to output RGB color data;
Determining whether the pixel is an error candidate point for one of the plurality of color interpolated pixels;
Determining an error boundary direction based on the error candidate point; And
Performing filtering using the determined error boundary direction,
The filtering may include correcting interpolated color data of a pixel for the error candidate point, and using the interpolated color data of the pixel, at least one correction data for the error boundary direction as an entry. An image processing method for correcting interpolated color data by performing a median filter. Performing color interpolation on a plurality of pixels configured as a Bayer array to output RGB color data;
Determining whether the pixel is an error candidate point for one of the plurality of color interpolated pixels;
Determining an error boundary direction based on the error candidate point; And
Performing filtering using the determined error boundary direction,
The error candidate point is a pixel of which original color data to be output is R (red) or B (blue), and is determined using original G (green) color data disposed around the one pixel. The method of claim 3,
And the error candidate point is determined using a difference between at least one original G color data disposed in a first direction around the one pixel and at least one original G color data disposed in a second direction. Performing color interpolation on a plurality of pixels configured as a Bayer array to output RGB color data;
Determining whether the pixel is an error candidate point for one of the plurality of color interpolated pixels;
Determining an error boundary direction based on the error candidate point; And
Performing filtering using the determined error boundary direction,
The determining of the error boundary direction may include obtaining a difference value in a first direction and a difference value in a second direction with respect to original RGB data of a pixel disposed around the error candidate point;
Comparing the difference in the first direction and the difference in the second direction; and
And determining an error boundary direction as one of the first direction and the second direction according to the comparison result. Performing color interpolation on a plurality of pixels configured as a Bayer array to output RGB color data;
Determining whether the pixel is an error candidate point for one of the plurality of color interpolated pixels;
Determining an error boundary direction based on the error candidate point; And
Performing filtering using the determined error boundary direction,
When the original color data of the one pixel is R33 and the original G color data arranged in the first direction around the one pixel is G32 and G34 and the original G color data arranged in the second direction is G23 and G43,
And judging the one pixel as an error candidate point when the value of | G32 + G34-G23-G43 | is larger than a preset reference value. The method according to any one of claims 2 to 6,
And the error boundary direction is determined by using difference values between original color data arranged in a first direction around the error candidate point and original color data arranged in a second direction. The method according to any one of claims 2 to 6,
Determining whether determination of an error candidate point for the plurality of pixels is completed;
Selecting another pixel of the plurality of pixels; and
And repeating the determining of the error candidate point, determining the error boundary direction, and performing filtering on the selected pixel. A computer-readable storage medium including instruction code for processing an image by RGB color data output from a plurality of pixels,
The instruction code, the processor,
Color interpolation is performed on a plurality of pixels configured as a Bayer array to output RGB color data,
An error candidate point using original G (green) color data disposed around the one pixel with respect to one pixel among the plurality of color interpolated pixels, wherein the original color data is R (red) or B (blue). Judge
Determine an error boundary direction based on the error candidate point;
And code for causing filtering using the determined error boundary direction. delete A color interpolation unit configured as a Bayer array to perform color interpolation on a plurality of pixels outputting RGB color data;
An error point determination unit that determines whether an error candidate point is included in one pixel of the plurality of color interpolated pixels;
A boundary direction determination unit for determining an error boundary direction based on the error candidate point; And
And a filtering unit which performs filtering using the determined error boundary direction.
The filtering unit corrects the interpolated color data of the pixel with respect to the error candidate point, and medians the median filter by using the interpolated color data of the pixel and at least one correction data of the error boundary direction as an entry. image processing apparatus for correcting interpolated color data by performing a filter). A color interpolation unit configured as a Bayer array to perform color interpolation on a plurality of pixels outputting RGB color data;
An error point determination unit that determines whether an error candidate point is included in one pixel of the plurality of color interpolated pixels;
A boundary direction determination unit for determining an error boundary direction based on the error candidate point; And
And a filtering unit which performs filtering using the determined error boundary direction.
The error point determination unit is a pixel in which the original color data to be output is R (red) or B (blue), and determines an error candidate point using original G (green) color data arranged around the one pixel. Image processing device. The method of claim 12,
And the error candidate point is determined using a difference between at least one original G color data disposed in a first direction around the one pixel and at least one original G color data disposed in a second direction. A color interpolation unit configured as a Bayer array to perform color interpolation on a plurality of pixels outputting RGB color data;
An error point determination unit that determines whether an error candidate point is included in one pixel of the plurality of color interpolated pixels;
A boundary direction determination unit for determining an error boundary direction based on the error candidate point; And
And a filtering unit which performs filtering using the determined error boundary direction.
The boundary direction determination unit,
Obtaining a difference value in a first direction and a difference value in a second direction with respect to original RGB data of a pixel disposed around the error candidate point,
Comparing the difference in the first direction with the difference in the second direction,
And determining the boundary direction as one of the first direction and the second direction based on the comparison result. A color interpolation unit configured as a Bayer array to perform color interpolation on a plurality of pixels outputting RGB color data;
An error point determination unit that determines whether an error candidate point is included in one pixel of the plurality of color interpolated pixels;
A boundary direction determination unit for determining an error boundary direction based on the error candidate point; And
And a filtering unit which performs filtering using the determined error boundary direction.
The error point determination unit,
When the original color data of the one pixel is R33 and the original G color data arranged in the first direction around the one pixel is G32 and G34 and the original G color data arranged in the second direction is G23 and G43,
G32 + G34-G23-G43 | Image processing apparatus for determining the one pixel as an error candidate point when the value of the | The method according to any one of claims 11 to 15,
And the boundary direction determiner determines the boundary direction using difference values between original color data arranged in a first direction around the error candidate point and original color data arranged in a second direction. The method according to any one of claims 11 to 15,
Determine whether the determination of the error candidate point for the plurality of pixels is completed;
Selecting another pixel of the plurality of pixels,
And a controller configured to repeat determination of the error candidate point, determination of the error boundary direction, and filtering with respect to the selected pixel.

Images