KR101068652B1 - Apparatus and method for interpolating color, and the recording media storing the program performing the said method - Google Patents

Abstract

The present invention relates to a color interpolation apparatus for color interpolating an image by selectively / fusionly applying color interpolation expressions according to a local area, and a method and a recording medium on which a program for implementing the method is recorded. According to an exemplary embodiment of the present invention, a first edge direction in which a direction of a first edge is determined in an input image in consideration of a first edge direction determination criterion derived from a difference value between identical colors interpolated by each of predetermined color interpolation equations is disclosed. Determination unit; A second edge that determines the direction of the second edge in an area not including the first edge in consideration of the second edge direction determination criterion derived from the variance of the colors between the at least three images interpolated by each of the color interpolation equations Direction determination unit; And an interpolation value determiner that determines a color interpolation value for color interpolation of the input image in consideration of the direction of the first edge or the second edge. According to the present invention, even fine and complex edges can be accurately determined, and high-quality color images can be obtained through excellent color interpolation performance.

Description

Apparatus and method for interpolating color, and the recording media storing the program performing the said method}

The present invention relates to a color interpolation apparatus, a method thereof and a recording medium having recorded thereon a program for implementing the method. More specifically, the present invention relates to a color interpolation apparatus for color interpolating an image by selectively / fusionly applying color interpolation expressions according to a local area, and a method and a recording medium on which a program for implementing the method is recorded.

The image sensor is used to acquire an image signal and is widely used in digital cameras, camcorders, and the like. The image sensor stores brightness values of each pixel, and includes a charge coupled device (CCD) and a complementary metal oxide semiconductor (CMOS).

CCDs are classified into single CCDs and multiple CCDs according to the number of colors formed in the pixel. Although multiple CCDs have better performance than single CCDs, they must be equipped with a sensor three times or more, resulting in increased manufacturing costs and increased device size. Thus, single CCDs are commonly used.

A single CCD uses a single sensor to obtain multiple color information by covering a color filter on the sensor. However, a single CCD may store only one color information in one pixel. Thus, in order to obtain complete image information, unavoidable color information is estimated (interpolated) by using neighboring pixel information. This process is called color interpolation. The quality of the acquired image depends on the performance of the color interpolation algorithm. Therefore, many color interpolation algorithms have been proposed to obtain high quality images.

Most conventional color interpolation algorithms perform color interpolation in consideration of correlations between R, G, and B channels. Of these inter-channel correlations, the assumption of constant cross channel color difference is mainly used. This is because the hardware is easy to implement and the algorithm is simple.

The color difference assumption between fixed channels assumes that the edges of the R, G, and B channels coincide in the local region. The initial algorithm using the color difference assumption between the fixed channels interpolates the lost color component values such that the difference between two channels (G-R, G-B) between the current pixel and the neighboring pixel is constant.

However, since the initial algorithm using the color difference assumption between fixed channels does not consider the edge direction in the local region, false color error or moire effect occurs in the high frequency region. In the above description, an incorrect color error refers to a phenomenon in which an image is broken by interpolating a color completely different from the original color in some pixels. In addition, the wave pattern phenomenon refers to a phenomenon in which the color of the checkered clothes or the checkered tie included in the image is broken colorfully like a rainbow. Moreover, the image quality is worse because these things occur mainly at the edges that contain the main information in the image.

On the other hand, there is also an edge directional color interpolation algorithm to compensate for the shortcomings of the initial color interpolation algorithm that appears mainly in the edge region. The edge directional color interpolation algorithm is a method of interpolating according to the edge direction while considering the direction of the edge during color interpolation. However, even when using the edge directional color interpolation algorithm, if the edge direction is not properly considered, an incorrect color error or wave pattern may occur, resulting in poor image quality. In addition, when the pattern area is complicated, the edge criterion is not properly matched, and more false color errors occur in the image.

The present invention has been made to solve the above-described problem, by determining the edge direction in the image by selectively / fusionally applying the color interpolation equations according to the local region, the color to interpolate the image based on the determined edge direction It is an object of the present invention to provide a recording medium on which an interpolation apparatus, a method thereof, and a program for implementing the method are recorded.

The present invention has been made to achieve the above object, the first edge in the input image in consideration of the first edge direction determination criteria derived from the difference value between the same colors interpolated by each of the predetermined color interpolation equations ( a first edge direction determination unit determining the direction of the edge; At least three images interpolated by each of the color interpolation equations determine a direction of a second edge in an area not including the first edge in consideration of a second edge direction determination criterion derived from a dispersion value of colors; 2 edge direction determination unit; And an interpolation value determiner configured to determine a color interpolation value for color interpolation of the input image in consideration of a direction of the first edge or the second edge.

Preferably, the first edge direction determination unit is a first difference value between the same colors interpolated in the first direction with a difference value between the same colors and the first colors between the same colors interpolated in the second direction. 2 Use the difference. More preferably, the first edge direction determination unit considers a gradient in the same colors when obtaining the first edge direction determination criterion. Even more preferably, the sum of the gradient in the same colors interpolated in the first direction and the first difference value when the first direction is the horizontal direction is a horizontal edge determination reference value, and the second direction. In the vertical direction, the sum of the gradient between the second difference value and the same colors interpolated in the second direction is a vertical edge determination reference value, and the first edge direction determination unit includes a range of 0 to 1; A first multiplication value of a variable and the vertical edge determination reference value is compared with the horizontal edge determination reference value, or a reference value comparing a second multiplication value of the variable and the horizontal edge determination reference value with the vertical edge determination reference value part; And when the first multiplication value is greater than the horizontal edge determination reference value, the edge direction is determined as a horizontal edge, and when the second multiplication value is greater than the vertical edge determination reference value, an edge direction to determine the edge direction as a vertical edge. It includes a determination unit.

Preferably, the second edge direction determination unit is a difference value between the first colors interpolated in the first direction and the difference value between the second colors interpolated in the first direction as a variance value between the different colors. The second dispersion value derived from the difference value between the first variance value derived from and the first colors interpolated in the second direction and the difference value between the second colors interpolated in the second direction are used. More preferably, the first dispersion value is a horizontal edge determination reference value when the first direction is made horizontal, and the second dispersion value is a vertical edge determination reference value when the second direction is made vertical. The second edge direction determination unit may compare the first multiplication value of a variable in the range of 0 to 1 with the vertical edge determination reference value with the horizontal edge determination reference value, or compare the variable and the horizontal edge determination reference value. A reference value comparison unit for comparing a second multiplication value with the vertical edge determination reference value; And when the first multiplication value is greater than the horizontal edge determination reference value, the edge direction is determined as a horizontal edge, and when the second multiplication value is greater than the vertical edge determination reference value, an edge direction to determine the edge direction as a vertical edge. It includes a determination unit.

Advantageously, the color interpolation apparatus predicts said color interpolations as a taylor series, predicts a first color interpolation formula considering at least two adjacent colors, and at least a predetermined distance from said adjacent colors. It further includes an interpolation predictor for predicting the second color interpolation formula in consideration of one color. More preferably, the interpolation predictor uses a color adjacent to the adjacent color as at least one color spaced apart from the predetermined distance when predicting the second color interpolation equation.

Preferably, the interpolation value determiner uses two color interpolations as a color interpolation equation to determine the color interpolation value, wherein the two color interpolations are at least adjacent to the first color and the interpolation value of the first color. Consider the weight resulting from the relationship between the true values of one color. More preferably, said relationship value comprises a difference value between said interpolation value and said main value, said weight being inversely proportional to said relationship value.

Preferably, the color interpolation apparatus further includes a non-edge region extracting unit which extracts a non-edge region not including the first edge and the second edge from the input image, and the interpolation value determiner interpolates a selected color. A color interpolation value of the non-edge region is determined by considering a weight derived from a difference between a value and a main value of at least one color neighboring the selected color.

Preferably, the color interpolation device is mounted on an image sensor or an image signal acquisition device.

In addition, the present invention (a) determines the direction of the first edge (edge) in the input image in consideration of the first edge direction determination criteria derived from the difference value between the same colors interpolated by each of the predetermined color interpolation equations Making; (b) the direction of the second edge in the region not including the first edge in consideration of the second edge direction determination criterion derived from the dispersion value of the colors between the at least three images interpolated by each of the color interpolation equations; Determining; And (c) determining a color interpolation value for color interpolation of the input image in consideration of the direction of the first edge or the second edge.

Preferably, the step (a) may include a first difference value between the same colors interpolated in the first direction and a second difference between the same colors interpolated in the second direction with a difference value between the same colors. The difference value is used, and the gradient in the same colors is taken into consideration when obtaining the first edge direction determination criterion.

Preferably, the step (b) is a difference value between the first colors interpolated in the first direction and the difference value between the second colors interpolated in the first direction as the variance value of the different colors. The second dispersion value derived from the difference between the first variance values derived and the first colors interpolated in the second direction and the difference values between the second colors interpolated in the second direction is used.

Preferably, prior to step (a), (a ') predicting the color interpolations as a Taylor series, predicting a first color interpolation taking into account at least two adjacent colors, and determining the adjacent colors And predicting a second color interpolation equation in consideration of at least one color spaced apart from and a predetermined distance away, wherein step (a ') comprises at least one spaced apart from the predetermined distance when predicting the second color interpolation equation. As the color, a color adjacent to the adjacent color is used.

Preferably, step (c) uses two color interpolations to determine the color interpolation value, wherein the two color interpolations are adjacent to the first color and the interpolation value of the first color. Consider a weight resulting from the difference between the true values of at least one color and the weight is inversely proportional to the difference value.

According to the present invention, the following effects can be obtained. First, by determining the edge direction in an image by selectively / fusionly applying color interpolation equations according to local regions, it is possible to accurately determine not only thick edges but also detailed and complex edges. In addition, it is possible to accurately interpolate the color values over the entire area of the image.

Second, by interpolating the image by selectively / converging at least two color interpolation equations, loss of resolution can be reduced and high quality color images can be obtained. In addition, the performance of the post-processing algorithm attached to the back of the color interpolation algorithm can be improved.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. First, in adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even if displayed on different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, the following will describe a preferred embodiment of the present invention, but the technical idea of the present invention is not limited thereto and may be variously modified and modified by those skilled in the art.

1 is a block diagram schematically showing a color interpolation apparatus according to a preferred embodiment of the present invention. The following description refers to FIG. 1.

The color interpolation apparatus 100 performs color interpolation using correlations between R, G, and B channels. In detail, the color interpolation apparatus 100 estimates an edge direction for each pixel position in a Bayer image and performs interpolation according to the estimated edge direction. Through this process, the color interpolation apparatus 100 may reduce the loss of resolution due to interpolation and obtain a high resolution image.

The color interpolation apparatus 100 obtains a high-order color interpolation equation by using a local pixel approximation method by Taylor series. The color interpolation apparatus 100 adaptively fuses the color interpolation equations acquired when interpolating colors for each predetermined region to restore color components lost in the image to nature.

The color interpolation apparatus 100 obtains information on the correlation between colors using the obtained color interpolation equations. In addition, the color interpolation apparatus 100 determines an edge direction determination criterion from the obtained correlation information in consideration of characteristics of the color interpolation equations appearing in the edge region and the region crossing the edge. In addition, the color interpolation apparatus 100 distinguishes areas of areas to which colors are to be interpolated according to the determined edge direction determination criteria.

The color interpolation apparatus 100 includes an edge direction determiner (not shown), an interpolation value determiner 150, and a color interpolator 160. The edge direction determiner divides the input image into an edge region, a pattern region, a flat region, and the like according to region determination criteria. The edge direction determination unit includes a concept including an interpolation predictor 110, a temporary interpolator 120, a first edge direction determiner 130, and a second edge direction determiner 140. do. The interpolation value determiner 150 generates a final interpolation value by appropriately fusing the G channel value temporarily interpolated in the vertical direction and the G channel value interpolated in the horizontal direction according to the divided region. Then, the color interpolator 160 interpolates the color of the input image using the final interpolation value. In the above description, the input image is a Bayer image having a Bayer pattern. In particular, the input image is a Bayer image by RGB CFA (Color Filter Array).

In more detail, the color interpolation apparatus 100 may include an interpolation predictor 110, a temporary interpolator 120, a first edge direction determiner 130, a second edge direction determiner 140, and an interpolation value. The decision unit 150 and the color interpolation unit 160 are included. The interpolation predictor 110 predicts a first- or second-order interpolation equation to be applied to interpolation of the input image by using a Taylor series. The temporary interpolator 120 obtains the R, G, B values temporarily interpolated in the vertical direction or the horizontal direction from the input image by using the first and second interpolation equations. The temporarily interpolated R, G, and B values are then used when extracting the edge region, generating final interpolation values, and the like. The first edge direction determiner 130 extracts an edge region from the input image by using a difference between the temporarily interpolated G values and determines a thick edge direction in the input image. At this time, the remaining region not extracted as the edge region is regarded as a pattern region. The second edge direction determiner 140 extracts the edge area from the pattern area by using the difference between the temporarily interpolated KR (GR) values or the difference between the KB (GB) values, and extracts the detailed and complicated edge direction in the input image. To judge. At this time, the remaining region which is not extracted as the edge region is regarded as a flat region. The interpolation value determiner 150 appropriately selects or fuses values interpolated by the first interpolation or the second interpolation according to an area to obtain a final interpolation value.

The interpolation predictor 110 performs a function of predicting the color interpolations using a Taylor series. In detail, the interpolation predictor 110 predicts the first color interpolation equation in consideration of at least two adjacent colors, and calculates a second color interpolation equation in consideration of at least one color away from the adjacent colors by a predetermined distance. Predictive functions. The temporary interpolator 120 performs a function of temporarily interpolating color values of pixels included in the input image by using a first color interpolation formula or a second color interpolation formula.

Meanwhile, the interpolation predictor 110 uses a color neighboring an adjacent color as at least one color separated from a predetermined distance when predicting the second color interpolation equation.

In the present embodiment, the color interpolation apparatus 100 is a device for driving a color interpolation algorithm. The color interpolation apparatus 100 generates an interpolation value using an interpolation equation according to an edge direction classified through edge direction determination criteria. At this time, as the interpolation formula well reflects the characteristics of the local region, the interpolation value using the interpolation formula is similar to the color value actually lost. So, in this embodiment, J.S. The interpolation is determined using the Taylor series proposed by Jimmy Li and Sharmil Randhawa in "High order extrapolation using taylor series for color filter array demosacing, LNCS, vol. 3656, pp. 703-711, Oct, 2005." The Taylor series is

In the above, x and a each represent a coordinate position, and g (·) represents an arbitrary function that returns a G value.

By the way, Equation 1 can approximately display the G value at the x position as the value at the a position. It is assumed that the entire Bayer pattern is the same as in FIG. 2A, and the 1D Bayer pattern is extracted from it as shown in FIG. The G value at the x position along the arrow direction with respect to the extracted 1D Bayer pattern is expressed as a G value at the x-1 position as follows.

In the above, g ⁽ⁿ⁾ (*) represents the nth derivative of g (*).

Using Equation 2, g (x) can be obtained from the derivative of g (·) at the x-1 position. However, it is not easy to find the derivative of g (·) because a given G channel value is down sampled. Thus, the color difference between fixed channels is assumed. According to the color difference assumption between the fixed channels, it may be assumed that the G function change rate and the R function change rate are similar in the local region as shown in FIG. 3. Therefore, by using the color difference assumption between fixed channels, a differential value of g (·) can be obtained at each position. For example, the g '(x-1) value is as follows.

는 다운 샘플링 되기 전의 G 값을 나타낸다.In the above,

Is the G value before downsampling.

In this way, after assuming that the first few terms of the differential value of g (·) exist based on the assumption of the color difference between fixed channels, the g (x) value is derived as follows.

는 G_x의 임시 보간된 색상값을 나타낸다. 또한, (a)는 n이 2일 때 g(·)의 미분값이 0인 조건에서 얻어낸 1차 보간식을 의미한다. 또한, (b)는 n이 3 이상일 때 g(·)의 미분값이 0인 조건에서 얻어낸 2차 보간식을 의미한다. 본 실시예에서 (a)와 (b)는 영역 구분 과정과 보간값 생성 과정에서 이용된다.In the above,

_Denotes a temporary interpolated color value of G _x . Further, (a) means a first-order interpolation equation obtained under the condition that the derivative value of g (·) is 0 when n is 2. In addition, (b) means the second interpolation formula obtained on the condition that the derivative value of g (*) is 0 when n is three or more. In the present embodiment, (a) and (b) are used in the area classification process and the interpolation value generation process.

Both the first and second interpolation equations perform interpolation using neighboring same or different colors. The difference between these two interpolations is that the second interpolation uses more colors than the first interpolation. Therefore, the second interpolation equation also uses pixels farther than the pixel positions to be interpolated than the first interpolation equation.

The first edge direction determiner 130 determines the direction of the first edge in the input image in consideration of the first edge direction determination criterion derived from the difference between the same colors interpolated by each of the predetermined color interpolation equations. To determine the function. As described above, the predetermined color interpolations include a first color interpolation (ie, a first interpolation) and a second color interpolation (ie, a second interpolation).

The first edge direction determiner 130 may determine a first difference value between the same colors interpolated in the first direction and a second difference value between the same colors interpolated in the second direction as a difference value between the same colors. I use it. Preferably, the first edge direction determination unit 130 considers gradients in the same colors when obtaining the first edge direction determination criteria.

When the first direction is the horizontal direction, the sum of the gradient between the first difference value and the same colors interpolated in the first direction is defined as the horizontal edge determination reference value. In addition, when the second direction is vertical, the sum of the gradients in the same colors interpolated in the second direction and the second difference value is defined as a vertical edge determination reference value. In this case, the first edge direction determiner 130 includes a first reference value comparator (not shown) and a first edge direction determiner (not shown). The first reference value comparator compares the first multiplication value of the variable in the range of 0 to 1 and the vertical edge determination reference value with the horizontal edge determination reference value. In addition, the first reference value comparison unit performs a function of comparing the second multiplication value of the variable and the horizontal edge determination reference value with the vertical edge determination reference value. The first edge direction determination unit performs a function of determining the edge direction as the horizontal edge when the first multiplication value is larger than the horizontal direction edge determination reference value. In addition, the first edge direction determination unit performs a function of determining the edge direction as a vertical edge when the second multiplication value is larger than the vertical direction determination criterion value.

In the present embodiment, the color interpolation equation obtained above is used as a method for finding the edge direction while reflecting the characteristic of the local region of the image. The accuracy of the interpolation equation obtained using the Taylor series depends on the degree of hypothesis of color difference between channels in each region. That is, the accuracy of the obtained interpolation equation is determined depending on whether the derivative value in the G channel is replaced with the derivative value in the R channel or the B channel in the process of obtaining the derivative value of g (·). In general, the more consistent with the edge direction, the better the color difference assumption between channels is made, and the more the direction across the edge is less well established. When interpolation is performed along the edge direction, the result of the interpolation expression is small. Therefore, in the present embodiment, the edge direction, the edge region, the pattern region, and the flat / pattern region are set in the edge direction by using the interpolated values temporarily using the first interpolation and the second interpolation. ), Etc. First, in the first step, a thick edge is determined by dividing it into an edge region and a pattern region. Then, in the second step, the pattern region is divided into an edge region and a flat region to determine a fine and complicated edge.

First, the criterion for distinguishing the edge region from the pattern region is obtained by using the interpolated image temporarily using the first interpolation and the second interpolation. To determine the edge direction criterion, consider the difference between the first-order interpolation and the second-order interpolation. In the taylor series, the first and second interpolation equations were determined and obtained according to the differential value of g (·). At this time, in order to obtain a higher-order differential value of g (·), a pixel value far from the current pixel should be used. Unlike the gray image, in the Bayer pattern image, since the G sample value does not exist in every pixel, it is difficult to obtain a derivative of the higher order g (·). Therefore, as shown above, the R sample value or the B sample value is used to obtain a higher order differential value of g (·). However, in order to use the R sample value or the B sample value, the inter-channel correlation of the current pixel must be kept the same even in the distant pixel. Here, a criterion for dividing the edge region and the pattern region can be obtained. For example, in the edge direction, it is not a big problem to use the R sample value or the B sample value when obtaining a higher-order g (·) differential value. However, in the direction crossing the edge, if the derivative of the higher order g (·) is obtained using the R sample value or the B sample value, the obtained derivative value is a large difference from the derivative value of the actual higher order g (·). Will generate incorrect interpolation values. If this is summarized as an edge direction criterion, the difference is smaller when the difference between the values interpolated in the vertical direction or the horizontal direction using the first interpolation and the second interpolation is smaller.

On the other hand, the higher order g (·) derivative is reflected in the form of adding a high frequency component to the interpolation value. Unlike the first interpolation equation, the high frequency components added in the second interpolation equation are shown in FIG. 4. In FIG. 4, ⓐ means R _x -R _x-2 of Equation 4 (b), and ⓑ means G _x-1 -G _x-3 of Equation 4 (b). Ⓒ means R _x -R _{x -2} -G _{x -1} -G _{x -3} in the expression (4). Under the assumption that the inter-channel correlation is established, the result of the quadratic interpolation is generally closer to the actual value lost than the result of the first-order interpolation due to the added high frequency component value. However, in the region where the inter-channel correlation does not hold, the second derivative generates more error than the first derivative due to the added high frequency component. Therefore, it can be seen that it is effective to use the interpolated G value when determining the edge direction by obtaining a gradient within the G value interpolated temporarily in the vertical direction or the horizontal direction. The two edge determination criteria are expressed as follows.

(a) Horizontal Edge Determination Criteria = Diff (Gh1, Gh2) + ∇Gh2

(b) Vertical Edge Determination Criteria = Diff (Gv1, Gv2) + ∇Gv2

In the above, Diff (A, B) represents the absolute value of the difference between A and B. Gh1 means the G value interpolated in the horizontal direction using the first interpolation formula, and Gh2 means the G value interpolated in the vertical direction using the second interpolation formula. Gv1 denotes a G value interpolated in the vertical direction using the first interpolation formula, and Gv2 denotes a G value interpolated in the vertical direction using the second interpolation formula. ∇Gh2 means a gradient in Gh2, ∇Gv2 means a gradient in Gv2.

The determination of the direction of the edge from the two edge determination criteria follows the following equation.

(a) Horizontal Edge Judgment Criteria <Variable × Vertical Edge Judgment Criteria

    → horizontal edge

(b) Vertical Edge Judgment Criteria <Variable × Horizontal Edge Judgment Criteria

    → vertical edge

(c) otherwise

    → pattern edge

In the above, the variable has a value between 0 and 1. For example, when the variable is 0.8, the horizontal edge is determined as the horizontal edge when the horizontal edge criterion is smaller than the product of the variable and the vertical edge criterion. In other words, it can be seen that the smaller the value of the variable is, the more reliable the edge direction is determined.

The second edge direction determination unit 140 may be configured to determine whether the at least three images interpolated by the color interpolation equations do not include the first edge in consideration of a second edge direction determination criterion derived from a dispersion value of colors. A function of determining the direction of the second edge is performed. Preferably, the second edge direction determination unit 140 is a difference between the first colors interpolated in the first direction and the difference between the second colors interpolated in the first direction as a dispersion value between the different colors. The first dispersion value derived from the value and the second dispersion value derived from the difference value between the first colors interpolated in the second direction and the second colors interpolated in the second direction are used.

When making a 1st direction into a horizontal direction, a 1st dispersion value is defined as a horizontal edge determination reference value. Moreover, when making a 2nd direction into a vertical direction, a 2nd dispersion value is defined as a vertical edge determination reference value. In this case, the second edge direction determination unit 140 includes the second reference value comparison unit 141 and the second edge direction determination unit 142, similarly to the case of the first edge direction determination unit 130. The second reference value comparator 141 compares the first multiplication value of the variable in the range of 0 to 1 with the vertical edge determination reference value with the horizontal edge determination reference value. In addition, the second reference value comparator 141 performs a function of comparing the second multiplication value between the variable and the horizontal edge determination reference value with the vertical edge determination reference value. The second edge direction determining unit 142 determines the edge direction as the horizontal edge when the first multiplication value is larger than the horizontal edge determination reference value. In addition, the second edge direction determination unit 142 performs a function of determining the edge direction as a vertical edge when the second multiplication value is larger than the vertical edge determination reference value.

Hereinafter, a method of distinguishing the edge region and the flat region in the pattern region will be described in detail.

The pattern region is a detailed and complicated edge region, and it is an area where it is not easy to determine the edge direction in any one direction. Since it is difficult to accurately determine the edge direction using only the G channel information, the edge direction is determined using the R channel information and the B channel information. R, G, and B values temporarily interpolated in the vertical and horizontal directions using interpolation equations may be used to determine the reliability of the inter-channel colors. When interpolated in the edge direction, the color difference values appear smooth. However, when interpolated in the direction crossing the edge, the color difference value is discontinuous. In the present exemplary embodiment, when the color values are temporarily interpolated using the first interpolation equation and the second interpolation equation, the edge direction is determined in a direction in which the correlation between channels is well established.

Since the R or B channel is sampled less than the G channel due to the characteristic of the Bayer pattern, the color difference value is calculated according to the number of R or B channels. When the first and second interpolation equations are used, the color difference values are obtained, respectively, and then the absolute value of the difference between them is obtained. Since the color difference value is constant when interpolated in the edge direction, the variance of the absolute value of the difference is obtained, and the edge direction is determined as the variance value is smaller. If this is expressed as an equation, it is as follows.

(a) Horizontal edge judgment criterion = Var {Diff (KRh1, KRh2)}

(b) vertical edge judgment criterion = Var {Diff (KRv1, KRv2)}

In the above, Var {·} represents a variance. KRh1 refers to the difference between the G and R values interpolated in the horizontal direction using the first-order interpolation equation, and KRh2 refers to the difference between the G and R values interpolated in the vertical direction by the second interpolation equation. do. KRv1 denotes the difference between the G value and the R value interpolated in the vertical direction using the first-order interpolation equation, and KRv2 denotes the difference between the G value and R value interpolated in the vertical direction by the second interpolation equation. do.

The determination of the direction of the edge from the two edge determination criteria follows the following equation.

(a) Horizontal Edge Judgment Criteria <Variable × Vertical Edge Judgment Criteria

    → horizontal edge

(b) Vertical Edge Judgment Criteria <Variable × Horizontal Edge Judgment Criteria

    → vertical edge

(c) otherwise

    → flat / pattern edge

In the above, the variable has a value between 0 and 1.

The interpolation value determiner 150 determines a color interpolation value for color interpolation of the input image in consideration of the direction of the first edge or the second edge. Preferably, the interpolation value determiner 150 uses two color interpolations as a color interpolation equation to determine a color interpolation value, and the two color interpolations are at least adjacent to the first color and the interpolation value of the first color. Consider the weight resulting from the relationship between the true values of one color. More preferably, the relationship value includes a difference value between the interpolation value and the main value, and the weight is inversely proportional to the relationship value.

In the present embodiment, the interpolation value generation process obtains the final interpolation value by using the color interpolation equation according to the edge direction determined in the region division process. However, since the degree of correlation between the channels is different according to the local region, there is a problem in that the optimal interpolation formula should be selected from the first interpolation formula or the second interpolation formula. Therefore, there is a need for a process of properly fusion of the first interpolation or the second interpolation according to the local area. Since the process of generating interpolation value in the edge area is to perform the 1D interpolation on the direction determined as the edge direction to obtain the final interpolation value, the values interpolated using the first interpolation or the second interpolation according to the edge direction are peripheral. You can choose linear or quadratic interpolation to see if it exists continuously with the original value of. In this embodiment, only one of the two interpolation equations is selected, and the final interpolation values are determined by fusion of the two values as appropriate. This is expressed as the following equation.

는 최종 보간값을 의미하고,

는 1차 보간식으로 보간한 값을 의미하며,

는 2차 보간식으로 보간한 값을 의미한다. w₁은 1차 보간식에 대한 가중치를 의미하며, w₂는 2차 보간식에 대한 가중치를 의미한다.

Means the final interpolation value,

Is the value interpolated by the first-order interpolation formula,

Is the value interpolated by the quadratic interpolation formula. w ₁ means the weight for the first-order interpolation formula, w ₂ means the weight for the second-order interpolation formula.

The two weights can be obtained from the difference between the interpolated value and the surrounding original value. The higher the interpolated value is to the surrounding original value, the larger the weight, and the smaller the weight. This is expressed as an equation.

In the above, N denotes the originally existing G value position around the interpolated pixel position.

Similarity with the original value can be obtained by measuring the difference between the interpolated value and the original value. The smaller the difference between the two values, the higher the similarity; the larger the difference, the lower the similarity. In this embodiment, the difference between the two values is used as a similarity measuring tool, and it is also possible to extend the variation (dispersion) between the interpolated values and the original values to the similarity measuring tool.

The color interpolation apparatus 100 may further include a non-edge area extractor (not shown) that extracts a non-edge area not including the first edge and the second edge from the input image. The interpolation value determiner 150 is interlocked with the non-edge region extracting unit in consideration of the weight resulting from the difference between the interpolation value of the selected color and the main value of at least one color neighboring the selected color. To determine the function.

The flat / pattern edge region is an area in which the edge direction is difficult to be determined. The final interpolation value is determined by fusion of the interpolated value in the vertical direction and the interpolated value in the horizontal direction. This is expressed as an equation.

는 수평 방향으로 보간된 값을 의미하며,

는 수직 방향으로 보간된 값을 의미한다. 앞서 설명한 방법을 토대로 1차 보간식 및 2차 보간식으로 보간한 값을 적절히 융합하여 얻는다. 또한, w_h는 수평 방향으로 보간된 값에 대한 가중치를 의미하며, w_v는 수직 방향으로 보간된 값에 대한 가중치를 의미한다. 이 두 가중치 역시 앞서 설명한 두 가중치와 유사하게 가중치 값을 결정한다. 즉, 수직 방향으로 보간된 값 또는 수평 방향으로 보간된 값이 주변 원본값과 차이가 적을수록 큰 가중치를 주고, 그렇지 않은 경우는 작은 가중치를 준다. 이를 수학식으로 표현하면 다음과 같다.In the above,

Is the value interpolated in the horizontal direction.

Is the value interpolated in the vertical direction. Based on the method described above, the values interpolated by the first interpolation and the second interpolation are properly fused. In addition, w _h means a weight for the value interpolated in the horizontal direction, w _v means a weight for the value interpolated in the vertical direction. These two weights also determine weight values similar to the two weights described above. That is, as the value interpolated in the vertical direction or the value interpolated in the horizontal direction is smaller from the surrounding original value, the larger weight is given, otherwise the weight is smaller. This is expressed as an equation.

In this embodiment, as described above, the interpolation value obtained through the interpolation equation is appropriately fused according to the local area, and the aim is to reduce the resolution loss due to interpolation.

The color interpolator 160 performs a function of color interpolating the input image based on the final interpolation value. Preferably, the color interpolator 160 first interpolates the G channel and then interpolates the R and B channels based on the above description.

R channel interpolation and B channel interpolation are obtained using the G channel interpolation result. 4 illustrates a case of interpolating an R channel value at a B position. The KR value is calculated at the peripheral R position of the B position to establish the color difference assumption between the channels, and the final interpolation value R is obtained through the linear combination between them. When the R channel value interpolation expression at the B position is expressed by the following equation.

In the above, (a) is for the R channel value interpolation formula at the B position, and (b) is for the R channel value interpolation formula at the G position. In (a), w represents the R position located around the B position, and in (b) w represents the R position located around the G position.

On the other hand, B channel interpolation is the same as the R channel interpolation process.

The power supply unit 170 performs a function of supplying power to each component constituting the color interpolation device 100.

The controller 180 controls the overall operation of each component constituting the color interpolation device 100.

Next, the color interpolation method of the color interpolation apparatus 100 is demonstrated. 5 is a flowchart illustrating a color interpolation method according to a preferred embodiment of the present invention. The following description refers to FIG. 5.

First, the first edge direction determining unit 130 considers the first edge direction determination criterion derived from the difference value between the same colors interpolated by each of the predetermined color interpolation equations. Determine the direction of (S510).

When the first edge direction determiner 130 determines the first edge direction, the first edge direction determiner 130 interpolates the first color difference between the same colors interpolated in the first direction and the same color interpolated in the second direction. Using the second difference value between the two, the gradient in the same colors can be considered when obtaining the first edge direction determination criteria.

Subsequently, the second edge direction determination unit 140 does not include the first edge in consideration of the second edge direction determination criterion derived from the variance value of the colors between the at least three images interpolated by each of the color interpolation equations. In S520, the direction of the second edge (fine and complex edge) is determined.

The second edge direction determination unit 140 determines the second edge direction, and the difference between the first colors interpolated in the first direction and the second color interpolated in the first direction as the dispersion values of the different colors. The first dispersion value resulting from the difference between the fields and the second dispersion value resulting from the difference between the first colors interpolated in the second direction and the second colors interpolated in the second direction can be used. have.

Thereafter, the interpolation value determiner 150 determines a color interpolation value for color interpolation of the input image in consideration of the direction of the first edge or the second edge (S530).

The interpolation value determiner 150 uses two color interpolations as a color interpolation equation to determine a color interpolation value, and the two color interpolation equations of the first color and at least one color adjacent to the first color are used. Consider the weight resulting from the difference between these values. The weight is inversely proportional to the difference.

Prior to step S510, the interpolation predictor 110 predicts the color interpolations using the Taylor series, and the first color interpolation and the second colors considering the same colors as the color interpolations. An interpolation equation can be predicted (S500). Preferably, when the interpolation predictor 110 predicts the first color interpolation equation, adjacent pixels adjacent to both sides of a specific pixel are the same colors, or when the second color interpolation equation is predicted, the adjacent pixels are different colors. And at least one pixel that is different in color from the adjacent pixels.

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. The computer-readable recording medium may be a magnetic storage medium (for example, a ROM, a floppy disk, a hard disk, a magnetic tape, etc.), an optical reading medium (for example, a CD-ROM, a DVD, an optical data storage device, etc.). And storage media such as carrier waves (eg, transmission over the Internet).

The above description is merely illustrative of the technical idea of the present invention, and various modifications, changes, and substitutions may be made by those skilled in the art without departing from the essential characteristics of the present invention. It will be possible. Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical spirit of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by the embodiments and the accompanying drawings. . The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

Optical imaging devices such as digital cameras and mobile phone cameras use a method of obtaining color images by applying a color interpolation algorithm from an image obtained by applying a color filter array on a single image sensor (CCD, CMOS) to reduce hardware size and cost. Therefore, the quality of the image obtained through the optical imaging apparatus is determined according to the performance of the color interpolation algorithm.

The present invention is a color interpolation algorithm using a local pixel approximation method. This algorithm can be largely divided into the area classification step and the interpolation value generation step. When the CFA image comes in, the area classification step is primarily divided into an edge area and a pattern area according to a region judgment criterion using an interpolation equation obtained by a local pixel approximation method using a taylor series. Separate. In the interpolation value generation step, the final interpolation value is generated by appropriately fusion of the interpolation values obtained by using the interpolation equations obtained in the vertical and horizontal directions according to the divided regions.

The present invention can be applied to an optical imaging device to reduce hardware size. In addition, considering that image quality is determined according to a color interpolation algorithm, the present invention may obtain a high resolution color image from a camera.

Today, with the ever-increasing market demand, it is urgent to develop a color interpolation algorithm that can obtain high-resolution color images in order to secure technological competitiveness. The present invention is expected to meet these market needs.

1 is a block diagram schematically showing a color interpolation apparatus according to a preferred embodiment of the present invention.

2 is a reference diagram for explaining a process of determining a first interpolation equation in the present embodiment.

3 is a reference diagram for explaining a color difference assumption between fixed channels to be applied in the present embodiment.

4 is a reference diagram for explaining a second interpolation decision process in the present embodiment.

5 is a flowchart illustrating a color interpolation method according to a preferred embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

100: color interpolation device 110: interpolation prediction unit

120: temporary interpolation unit 130: first edge direction determination unit

140: second edge direction determination unit 150: interpolation value determination unit

160: color interpolation unit 170: power unit

180: control unit

Claims (14)

A first edge direction determination unit determining a direction of a first edge in the input image in consideration of a first edge direction determination criterion derived from color difference values of the same colors interpolated by each of the predetermined color interpolation equations; At least three images interpolated by each of the color interpolation equations determine a direction of a second edge in an area not including the first edge in consideration of a second edge direction determination criterion derived from a dispersion value of colors; 2 edge direction determination unit; And An interpolation value determiner which determines a color interpolation value for color interpolation of the input image in consideration of the direction of the first edge or the second edge. Color interpolation apparatus comprising a. The method of claim 1, The first edge direction determination unit may determine a first color difference value between the same colors interpolated in the first direction and a second color difference value between the same colors interpolated in the second direction with the color difference values of the same colors. Color interpolation apparatus, characterized in that used. The method of claim 1, The second edge direction determination unit may include a first color derived from a color difference value between first colors interpolated in a first direction and a color difference value between second colors interpolated in the first direction as a dispersion value between the different colors. And a second dispersion value derived from a variance value and color difference values between the first colors interpolated in the second direction and color difference values between the second colors interpolated in the second direction. The method of claim 1, The interpolation value determiner uses two color interpolations as a color interpolation equation to determine the color interpolation value, wherein the two color interpolations are based on an interpolation value of a first color and at least one color adjacent to the first color. A color interpolation apparatus, characterized in that it takes into account weights derived from the relationship between the main values. The method of claim 2, When the first direction is a horizontal direction, the sum of the first color difference value and the gradient in the same colors interpolated in the first direction is a horizontal edge determination reference value, and the second direction is a vertical direction. When the sum of the gradient between the second color difference value and the same colors interpolated in the second direction is a vertical edge determination reference value, The first edge direction determination unit, The first multiplication value of the variable in the range of 0 to 1 and the vertical edge determination reference value is compared with the horizontal edge determination reference value, or the second multiplication value of the variable and the horizontal edge determination reference value is determined by the vertical edge determination reference value. A reference value comparison unit for comparing with; And The edge direction is determined as the horizontal edge when the first multiplication value is greater than the horizontal edge determination reference value, and the edge direction is determined as the vertical edge when the second multiplication value is greater than the vertical edge determination reference value. part Color interpolation apparatus comprising a. The method of claim 3, wherein The first dispersion value is a horizontal edge determination reference value when the first direction is horizontal, and the second dispersion value is a vertical edge determination reference value when the second direction is vertical. The second edge direction determination unit, The first multiplication value of the variable in the range of 0 to 1 and the vertical edge determination reference value is compared with the horizontal edge determination reference value, or the second multiplication value of the variable and the horizontal edge determination reference value is determined by the vertical edge determination reference value. A reference value comparison unit for comparing with; And The edge direction is determined as the horizontal edge when the first multiplication value is greater than the horizontal edge determination reference value, and the edge direction is determined as the vertical edge when the second multiplication value is greater than the vertical edge determination reference value. part Color interpolation apparatus comprising a. The method of claim 4, wherein And the relationship value includes a color difference value between the interpolation value of the first color and the main value, and the weight is inversely proportional to the relationship value. The method of claim 1, A non-edge area extractor for extracting a non-edge area not including the first edge and the second edge from the input image. Further includes, The interpolation value determiner determines a color interpolation value of the non-edge area in consideration of a weight derived from a color difference value between an interpolation value of a selected color and a main value of at least one color neighboring the selected color. . The method of claim 1, The color interpolation apparatus may be mounted on an image sensor or an image signal acquisition apparatus. determining a direction of a first edge in the input image in consideration of a first edge direction determination criterion derived from color difference values of the same colors interpolated by each of the predetermined color interpolation equations; (b) the direction of the second edge in the region not including the first edge in consideration of the second edge direction determination criterion derived from the dispersion value of the colors between the at least three images interpolated by each of the color interpolation equations; Determining; And (c) determining a color interpolation value for color interpolation of the input image in consideration of the direction of the first edge or the second edge; Color interpolation method comprising a. 11. The method of claim 10, The step (a) uses a first color difference value between the same colors interpolated in the first direction and a second color difference value between the same colors interpolated in the second direction with the color difference values of the same colors. And taking into account the gradients in the same colors when obtaining the first edge direction determination criterion. 11. The method of claim 10, In the step (b), the first dispersion resulting from the color difference values of the first colors interpolated in the first direction and the color difference values of the second colors interpolated in the first direction as the dispersion values of the different colors. And a second dispersion value derived from a color difference value between the first colors interpolated in the second direction and a color difference value between the second colors interpolated in the second direction. 11. The method of claim 10, Step (c) uses two color interpolations to determine the color interpolation value, The two color interpolation equations consider weights derived from a color difference value between an interpolation value of a first color and a main value of at least one color neighboring the first color, wherein the weight is an interpolation value of the first color and the main value. Color interpolation method characterized in that inversely proportional to the color difference value. In a computer-readable recording medium, A recording medium having recorded thereon a program for implementing the method according to any one of claims 10 to 13.

Images