KR20110039988A - Interpolation method - Google Patents

Abstract

PURPOSE: An interpolation method is provided to increase image resolution through an interpolation procedure to reduce a driving time of an image apparatus. CONSTITUTION: An interpolation method comprise the steps of: mapping n*n pixel data with m*m grids, wherein the n and m are natural numbers; and obtaining the selected A pixel value from m*m grids by using the plural kinds of pixel data adjacent to the A pixel in the n*n pixels.

Description

Interpolation Method {INTERPOLATION METHOD}

The present invention relates to an interpolation method of image data, and more particularly, to an interpolation method capable of increasing resolution.

In general, an image sensor such as a charge coupled device (CCD) type image sensor, a CMOS image sensor (CIS), or the like is a device or an electronic component that recognizes subject information and converts it into an electrical image signal. . The image sensor stores the image information input through the microlens through each color filter array (CFA) and stores the image information in each pixel constituting one image.

There are several types of color filter arrays, the most widely used being Bayer filters. Bayer filters consist of an array of red (R), blue (B) and green (G) filters, with a red color filter and a blue color on one pixel. One color filter is located among the filter and the green color filter. The green color filter occupies 50%, and the red color filter and blue color filter occupy 25%.

The Bayer filter receives the light and passes only the light of the color corresponding to the color filter. Therefore, the image data passing through the Bayer filter has only one color component for each color filter. However, the light must be a combination of three colors, that is, red (R), blue (B), and green (G) to realize all colors. Therefore, in order to realize a lifelike image, each pixel constituting the image must have red (R), blue (B), and green (G) information. The red (R), blue (B) and green (G) information is reproduced in every pixel.

The process of reproducing the red (R), blue (B), and green (G) information of the corresponding pixel using the surrounding color information is called interpolation.

The present invention provides an interpolation method capable of increasing the resolution in the process of performing the interpolation process.

According to an embodiment of the present invention, there is provided a method of interpolating an image of n * n pixels, comprising: mapping n * n pixel data to an m * m grid (n, m being a natural number); And obtaining a selected A pixel value of the m * m grid by using a plurality of pixel data in the periphery of the A pixel of the n * n pixels.

When the image is processed by the present invention, since the resolution of the image is naturally increased during the interpolation process, there is no need to proceed with an additional resolution increase. Therefore, since the processing time of the image can be greatly reduced, the driving time of the imaging device can be reduced.

In addition, by naturally increasing the resolution in the interpolation process, there is no need to additionally provide hardware for increasing the resolution of the image. Therefore, manufacturing cost can be reduced when implementing an image processing apparatus capable of increasing resolution.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, in order to facilitate a person skilled in the art to easily carry out the technical idea of the present invention. do.

The present invention relates to a high-resolution CFA interpolation method that allows the resolution to be naturally increased while performing color interpolation for CFA, unlike a general interpolation method applied to a color filter array (CFA).

1 is a view showing an interpolation process to illustrate the present invention.

Referring to FIG. 1, light incident from the outside in a CCD or CMOS image sensor passes through a CFA to form a single image through the CFA. The color information in each pixel constituting the image is converted into an electrical signal and is generally output as a digital image after passing through an analog digital conversion (ADC).

An ISP (Image Signal Processing) block is a block that is placed in a CCD or CMOS image sensor to correct the image information that has passed through the CFA to produce a higher quality image. In general, the ISP block performs an interpolation process, which is described in detail with a Bayer pattern.

CFA has only one element of RGB component per pixel and its arrangement varies. For example, a Bayer pattern, a CMY pattern, etc. are mentioned. The Bayer pattern is characterized by a series of basic patterns consisting of one R pixel and one B pixel, and two G pixels.

Since we only have color information for one image in one pixel, the other two color values are generated using the values in adjacent pixels, which is called interpolation. There are various ways to reconstruct the color information of the desired pixel using the values in the adjacent pixels. 1 shows a Bayer pattern, and at the bottom thereof, image information after the interpolation process is performed.

2 is a view showing an image state before and after the interpolation process according to the present invention.

Referring to FIG. 2, the interpolation process of the present invention is implemented so that the resolution is increased at the same time when color interpolation is performed to implement a resolution higher than that of the image sensor. In general, to increase the resolution of an image, an interpolation process is performed in an image sensor, and then the resolution is increased in another block that processes image information output from the image sensor. However, the present invention can naturally increase the resolution while performing the interpolation process in the IPS block.

A device with an image sensor can perform better in the ISP block than to apply the function of increasing the resolution to post-processing after the ISP block. Since the image data coming after the ISP block is a pixel value that is already modified by not only color interpolation but also other functions of various ISP blocks, there is a disadvantage in that the modified pixel value is used in improving the resolution.

Many functions are performed in the ISP stage. Color Interpolation, Lens Shading Correction, Noise Reduction, Color Correction, Gamma Correction, Hue / Saturation Control, Gain Control Control). The function of processing by ISP block type may be different, but all of the above functions apply. The most important part is the color interpolation process, and the image quality is different according to the color interpolation performance. In the present invention, in order to improve image quality and resolution, resolution is simultaneously improved when color interpolation is performed in an ISP block, so that an image can be processed more effectively.

For example, as shown in FIG. 2, when generating the R, G, and B values for each pixel in the Bayer pattern, a pixel memory space of 1.25 times each in the horizontal and vertical directions is provided, and each RGB is reconstructed in each pixel. In this case, images with a resolution of 1.25 times horizontally and 1.25 times vertically are produced. If the sensor has a resolution of 8M pixels, a resolution of 12.5M pixels can be achieved.

3 to 12, the process of improving the resolution of the image by 1.25 times by 1.25 times while performing the interpolation process will be described in detail. Specifically, a process of simultaneously performing interpolation while increasing the resolution of a 5 * 5 pixel image to a 6 * 6 image will be described.

3 is a diagram illustrating high grid mapping of image data of a Bayer pattern.

As shown in FIG. 3, the interpolation method according to the present embodiment first maps a 5 * 5 Bayer pattern to a 6 * 6 high resolution grid (hereinafter referred to as a high grid). Bayer pattern data is positioned at an interval of 1.25 horizontally with respect to one pixel in a 6 * 6 high grid. The final goal is to determine the R, G, and B values for each pixel on a 6 * 6 high grid.

4 is a diagram illustrating high grid mapping of image data of a CMY pattern.

As shown in Fig. 4, the mapping of image data of a cyan-magenta-yellow (CMY) pattern to a high resolution grid is similar to that shown in Fig. 3. When the interpolation process is performed on the image data of the CMY pattern, values of C, M, and Y may be determined for all pixels. Since the CMY pattern is similar to the interpolation process of the Bayer pattern, the following description will focus on the Bayer pattern.

5 is a flowchart illustrating an interpolation method according to a preferred embodiment of the present invention.

As shown in Figure 5, the interpolation method according to a preferred embodiment of the present invention first checks what pattern the pattern (S1). Subsequently, an image pattern to be interpolated is mapped corresponding to the high grid. That is, the position is confirmed on the high grid with respect to the RGB information of the image pattern to be interpolated (S2). Subsequently, a full high grid reconstruction is performed, starting with the G channel with the most information (S3). High grid reconstruction for the G channel calculates and reflects geometric closeness. When the G channel is completed, high grid reconstruction of the R channel and the B channel is performed, respectively (S4 and S5).

The geometric closeness is calculated by using the R values of respective positions referenced in the R channel configuration, and the integrated weight is configured by calculating the photometric similarity of the G values corresponding to the respective positions. This weight is normalized and used as the final weight. This normalization is shown in detail in FIGS. 6 and 7.

FIG. 6 is a flowchart showing the R channel high grid reconstruction step shown in FIG. 5 in detail, and FIG. 7 is a flowchart showing the B channel high grid reconstruction step shown in FIG. 5 in detail.

As shown in Fig. 6, the weight of the geometric closeness for the R channel is calculated (11), the weight of the photometric similarity of the R channel is calculated using the G channel (12), and then the two weights are generalized. (13, 14). FIG. 7 illustrates a generalization process for the B channel, and a description thereof will be omitted since it is similar to FIG. 6.

8 is a diagram illustrating high grid mapping using geometric closeness of G values according to a preferred embodiment of the present invention.

In the process of converting an image into a 6 * 6 pixel by using 5 * 5 pixel information, that is, an interpolation process in which resolution is increased, the value of 2 * 2 of 6 * 6 pixels of the high grid is found.

As shown in Fig. 8, to find the G value at the 2 * 2 position, n G values are used around the 2 * 2 position. Here, six G values are used around the 2 * 2 position. The surrounding G value referred to herein may vary depending on a situation in which resolution is increased, but a method using at least one pixel value in four directions of the corresponding pixel may be used.

In addition, instead of using all six G values uniformly, the weight is used by using a distance from the G value referenced at the 2 * 2 position. This uses the following equation (1).

Wk = D (Hx-HGx-k)

 Where Wk represents the k th weight. D () is a function that represents a distance. Hx is the position of the high grid grating and HGx-k represents the G value of the pixel at the kth closest to Hx. The information about the position is confirmed when the resolution increase magnification is determined, so it is not necessary to calculate the data every time, but only in the form of a lookup table (LUT). There are various ways to represent distance, for example, Euclid L2 norm or L1 norm can be used using Equation 2.

D (T) = | T |

Continuing look at, the G value at any pixel Hx can be expressed as Equation 3 below.

Referring to Equation 3, Hx to G value (G _Hx ) in any pixel is obtained by multiplying the G value of the pixel referred to by the weight value corresponding to the pixel and adding them all to obtain a first value. The second value is obtained by adding the weighted values of all the pixels referred to by and then dividing the second value by the first value.

The weight is determined according to the geometric closeness of how far the G value of the pixel to be referred to is from the pixel to which the current interpolation is to be performed. The information of the pixel data in the near position has a large weight to have a large influence, and the information of the pixel data in the relatively far position has a small weight to reduce the influence.

Therefore, in FIG. 8, six G values are referred to in the process of finding the G value of the X point at the position of 2 * 2 by interpolation process. It affects getting the value. In FIG. 8, the interpolation process is performed by referring to six neighboring G values. In FIG. 8, the interpolation process is performed by referring to six neighboring G values. Is showing.

After the G value is obtained for all the pixels using the aforementioned method, the R and B values are obtained. 9 and 10 illustrate a method of obtaining R values, and FIGS. 11 and 12 illustrate a method of obtaining B values.

9 is a diagram illustrating high grid mapping using geometric closeness of an R value according to a preferred embodiment of the present invention.

In FIG. 9, reference is made to refer to four R data in the periphery to obtain an R value of the pixel at the X position, and in FIG. 9, <B> refers to another four R data at the X position. Is shown.

As in the case of obtaining the above-described G value, as in the case of using the geometric closeness, in the case of obtaining the R value, the R value of the reference pixel is used by the weight according to the distance between the pixels for which the pixel is to obtain the current value. The information of the pixel data in the near position has a large weight to have a large influence, and the information of the pixel data in the far position has a small weight to reduce the influence. The equation of geometric clossness is given by Equation 4 below.

Wk_geoR = D (Hx -HRx-k)

Wk_geoR represents kth geometric clossness. D () is a function that represents a distance. Hx is a position of a high grid grating, and HRx-k represents position information of a pixel having an R value closest to H. Similar to the case of G, information about the position is confirmed when the magnification is determined, so it is only necessary to have the LUT form without calculating each time. The method of representing the distance is also the same as that of G. Of course, you can use different methods to get the weight of G and the value of R.

In addition, when calculating the R value, it is possible to additionally determine whether the image is a flat surface or an edge and obtain a photometric similarity to find a more accurate R value. The photometric similarity is obtained using the G value.

10 is a diagram illustrating an example of applying a photometric similiarity weight of an R value according to a preferred embodiment of the present invention.

In order to obtain a phtometric similarity for referencing the R value of the X position, the G value corresponding to the R value position of the reference pixel is reconstructed using the G channel value. In this case, there may be various reconstruction methods used. For example, a linear interpolation method may be used. In addition to linear interpolation, polynomial interpolation or spline interpolation can be used.

In detail, we use the reconstructed G value to calculate the photometric similarity of the R value. Photometric similarity compares the G value of the desired position to be reconstructed with the G value to be referred to, and if the difference is large, the weight is decreased, and if the difference is small, the weight is increased. This is represented by the following equation (5). It can also be expressed using a Gaussian function or some other generic function.

Where Gx is the G value of the pixel for which the R value is to be obtained, and Gx-k is a reconstructed G value of the reference pixel. That is, the Gx-k value is a G value obtained at the R pixel position which is now referred to by linear interpolation using n * n pixel information.

As described above, the equation of R value finally obtained using both geometric closeness and photometric similarity is shown in Equation 6 below.

To obtain the R value of the X position of FIG. 10, as shown in Equation 6, the geometric closeness weight, the photometric similarity weight, and the R value of each pixel to be referred to are multiplied and then added to obtain a first value. After multiplying a pixel's geometric closeness weight by a photometric similarity weight and then adding the value to the second value and dividing the first value by the second value, the R value of the desired pixel can be finally obtained. Dividing the first value by the second value thus normalizes the geometric closeness and photometric similarity weights.

In FIG. 10, the B value of the pixel at the X position may be similar to the case of obtaining the R value. 11 is a diagram illustrating high grid mapping using geometric closeness of a B value according to an exemplary embodiment of the present invention. 12 illustrates an example of applying a photometric similiarity weight of B value according to an embodiment of the present invention.

The equation for obtaining the B value of the pixel at the X position is shown in equation (7).

As described above, when another interpolation method is performed in this embodiment, all RGB information of 6 * 6 high grid pixels can be obtained using data of 5 * 5 bait patterns. Since one example of increasing the resolution while performing the interpolation process described in this embodiment is 1.25 times, the 5 * 5 grid is converted into a 6 * 6 grid. Such a pattern may be used for high grid lattice 0, 5, 10, 15,... High grid lattice 0, 5, 10, 15,... You can continue to perform repetitive tasks each time.

Herein, a 5 * 5 grid is described as being converted to a 6 * 6 grid, but in some cases, pixels of a unit grid may vary, and according to the technical details of the present embodiment, resolution of various sizes may be increased. Applicable Specifically, the resolution can be increased by more than twice while performing the interpolation process. In addition, as described above, the technical content according to the present embodiment may be applied to various patterns such as a CMY pattern as well as a Bayer pattern.

Although the present invention has been described in detail with reference to exemplary embodiments above, those skilled in the art to which the present invention pertains can make various modifications to the above-described embodiments without departing from the scope of the present invention. Will understand. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the scope of the appended claims, as well as the appended claims.

1 illustrates an interpolation process to illustrate the present invention.

Figure 2 shows the state of the image before and after the interpolation process according to the present invention.

3 is a diagram showing high grid mapping of image data of a Bayer pattern.

4 shows high grid mapping of image data of a CMY pattern.

5 is a flowchart showing an interpolation method according to the present invention;

6 is a flowchart detailing the R channel high grid reconstruction step shown in FIG.

7 is a flowchart detailing the B channel high grid reconstruction step shown in FIG.

8 illustrates high grid mapping using Geometric closeness of G values in accordance with a preferred embodiment of the present invention.

9 illustrates high grid mapping using Geometric closeness of R values in accordance with a preferred embodiment of the present invention.

10 illustrates an example of applying a photometric similiarity weight of an R value according to a preferred embodiment of the present invention.

Figure 11 illustrates high grid mapping using Geometric closeness of B values in accordance with a preferred embodiment of the present invention.

12 illustrates an example of applying a photometric similiarity weight of B value according to a preferred embodiment of the present invention.

Explanation of symbols on the main parts of the drawings

G: Green value of pixel

B: the blue value of the pixel

R: Red value of the pixel

Claims (11)

In a method of interpolating an image of n * n pixels, mapping n * n pixel data to an m * m grid, where n and m are natural numbers; And Obtaining a selected A pixel value of the m * m grid using a plurality of pixel data in the periphery of the A pixel of the n * n pixels Interpolation method comprising a. The method of claim 1, And the n * n pixel data is in a Bayer pattern or a CMY pattern. The method of claim 1, Obtaining the A pixel value Obtaining a G value of the A pixel by using a G value of X pixel data around the A pixel among the n * n pixel data; Obtaining an R value of the A pixel by using an R value of Y pixel data around the A pixel among the n * n pixel data; And Obtaining a B value of the A pixel by using a B value of Z pixel data around the A pixel among the n * n pixel data. The method of claim 3, wherein And obtaining the G values of the X pixel data using Equation 8 below.

(G _Hx is the G value of the A pixel to be obtained, Wk is the distance between the position of the A pixel and the pixel position of the G value among the n * n pixels, Gx-k is the G value of the pixel to be referred to, N is a natural number, and the number of pixels to refer to when _obtaining the G _Hx value of A pixels.) The method of claim 3, wherein The method of obtaining the R pixel of the Y pixel data in the periphery of the A pixel may use Equation 9 below.

(R _Hx is the R value of A pixel to be obtained, WkgeoR is the distance between the position of A pixel and the pixel position of B value among n * n pixels to be referred to, WkphotR is the photometric scalability of the pixel to be referred to ( photometric similarity) weight, Rx-R k is a value of the pixel to be reference, M is the number of pixels to see if you want to obtain the R value of _Hx a pixel with a natural number) The method of claim 5, The photometric similarity weight is determined by Equation 10 below.

Where Gx is the G value of the pixel for which the R value is to be obtained, and Gx-k is the G value of the position corresponding to the reference pixel. Binary G value)) The method of claim 6, The G value of the position corresponding to the reference pixel is a linear interpolation method, a polynomial interpolation method or a spline interpolation method characterized in that the. The method of claim 3, wherein The method of obtaining using the B values of the Z pixel data in the periphery of the A pixel using the following equation (11).

(B _Hx is the B value of the A pixel to be obtained, WkgeoR is the distance between the position of the A pixel and the pixel position of the B value to be referred to among n * n pixels, and WkphotB is the photometric scalability of the pixel to be referred to. photometric similarity) weight, Bx-k B is the value of the pixel to be reference, M is the number of pixels to see if you want to obtain the B _Hx values of the pixel a as a natural number) The method of claim 8, The photometric similarity weight is determined by Equation 12 below.

Where Gx is the G value of the pixel for which the B value is to be obtained, and Gx-k is the G value of the position corresponding to the reference pixel. Binary G value)) The method of claim 9, The G value of the position corresponding to the reference pixel is a linear interpolation method, a polynomial interpolation method or a spline interpolation method characterized in that the. The method of claim 1, The plurality of pixel data in the periphery of the A pixel And at least one pixel data in at least four directions of left and right and up and down of the A pixel.

Images