CN1992909A

CN1992909A - Reconstruction method of picture-element color information

Info

Publication number: CN1992909A
Application number: CN 200510135580
Authority: CN
Inventors: 郭俊廷
Original assignee: Altek Corp
Current assignee: Altek Corp
Priority date: 2005-12-29
Filing date: 2005-12-29
Publication date: 2007-07-04
Anticipated expiration: 2025-12-29
Also published as: CN100559885C

Abstract

The invention relates to a method for rebuilding color information, wherein it comprises that: via the position weight of pixels near each red/blue pixel, calculating out the color signal difference at horizontal and vertical directions; then via said difference, distributing weight values reversed; averaging the nearby two green pixels vertically and horizontally to obtain two average values; multiplying two average values with relative direction weight values, summing them into the green information of each blue/green pixel; then using dual-linear interpolation method to compensate the lost red and blue elements; using one middle-value selector to correct aberration. The invention has high accuracy, high resolution and uniform color.

Description

Pixel color information reconstruction method

Technical Field

The present invention relates to a method for reconstructing pixel color, and more particularly, to a method for reconstructing pixel color information with weighting and color difference correction added to provide an accurate color interpolation value.

Background

With the prevalence of video and audio multimedia and the successive introduction of digital image devices, the requirements of users on image colors are becoming more stringent. Among them, the image sensor array is a device for recording image signals in digital photography, and the minimum unit constituting a picture on the image sensor array is a photo-electric body, which is generally called a pixel (pixel). Since the photo-electric bodies themselves only react to the intensity of light and have no ability to separate and release the color of light, filters or filters of primary colors (red (R), green (G), and blue (B) must be added in front of each photo-electric body of the image sensor array to pass only red light, green light, and blue light, respectively, to obtain a color image.

However, in the image pixel array obtained by the image sensing array and the filter, each image pixel only has a single color element, and therefore, the brightness of the image pixel must be processed by image processing software, so that each image pixel contains three color elements, namely red, blue and green. The conventional pixel color reconstruction method uses interpolation (interpolation) to reconstruct color elements, and usually uses bilinear (i.e. four-point average) to calculate the data of the remaining two colors for each primary color pixel, as shown in fig. 1, four blue pixels 12 and four green pixels 14 are respectively located around each red pixel 10, and the pixel interpolation system uses the luminosity average of the nearest four blue pixels 12 and the luminosity average of the four green pixels 14 to calculate the missing blue and green data for the central red pixel 10, so that the red pixel obtains a complete RGB luminosity data. Similarly, the interpolation principle for the blue and green pixels 12, 14 is the same. However, this conventional method does not consider the edge (edge) and the variation of the signal trend, when some images with fine color lines or fine color variation are captured, if one of four averaged pixels is located at the edge (edge) or the sharp part of the signal, the overall average value will be affected by this point to generate a large error, so that the resolution at this point is deteriorated, and a sawtooth-shaped color pattern or a false color (aliasing) problem is generated, which results in poor image quality and easy distortion of the overall image.

To solve this problem, an algorithm system is known, such as US patent US 4,774,565, which first uses bilinear method (bilinear) to find out G color (channel), and then uses median filter (median filter) to calculate color difference domain (color difference domain) of R and B color; however, since the patent only uses a bilinear method without adding any step of determining the directivity when obtaining the G color frequency, the problem that the R and B color frequencies obtained based on the G color frequency have large errors is still caused, and the resolution is still poor, in addition to the disadvantage that the interpolated G color frequency has large errors.

Another known technique, for example, US 5,373,322, is to determine the signal difference of the central pixel in the horizontal and vertical directions, and then calculate the interpolation value in the direction with smaller difference, because the proper reconstruction of R and B color difference domains is not added, there is still a possibility of misjudgment of the direction, and the interpolation value is not correct, which results in serious false color problem.

Therefore, the present invention provides a pixel color information reconstruction method aiming at the above problems.

Disclosure of Invention

The main objective of the present invention is to provide a pixel color information reconstruction method, which distributes a weight ratio according to signal differences in a plurality of specific directions to perform weighted summation on interpolation values in the directions to obtain an accurate green interpolation value, and establishes accurate red and blue information by cooperating with a median filter, so that the reconstructed color information has a high accuracy effect, thereby effectively overcoming the defect of large error of the known interpolation color.

Another objective of the present invention is to provide a pixel color information reconstruction method to achieve the effects of high resolution and uniform color, so as to substantially improve the problems of the conventional interpolation system that is prone to generate false color and color distortion.

In order to achieve the above object, the present invention provides a pixel color information reconstruction method for complementing missing color elements in a red pixel, a blue pixel and a green pixel in an image pixel array, so that each pixel contains three elements, namely red, blue and green, the pixel color information reconstruction method comprises the following steps:

selecting a plurality of directions for each red/blue pixel of a green element to be reconstructed, and distributing weight proportions according to the positions of the pixels around the red/blue pixel so as to calculate the color signal difference in each direction according to the weight proportions;

distributing weighted values in the equal directions according to the inverse proportion of the color signal difference, averaging two adjacent green pixel values of the red/blue pixel along the equal directions to obtain a plurality of average values, further multiplying the average values by the weighted values in the directions to which the average values belong, and then adding the average values to obtain a weighted sum, wherein the weighted sum is the green element information in the red/blue pixel to be reconstructed; and

the red and blue elements lost by each pixel to be reconstructed are complemented by a bilinear interpolation method, so that each pixel contains three elements of green, red and blue.

Wherein the directions include a horizontal direction and a vertical direction.

In the step of calculating the color signal difference in the equal direction, for each pixel to be reconstructed, the surrounding pixels farther away are assigned with smaller weight ratio, and the surrounding pixels closer are assigned with larger weight ratio.

In the step of calculating the color signal difference amount according to the directions, the color signal difference amount is calculated according to the values of the green pixel and the red/blue pixel around the pixel to be reconstructed.

After complementing the red element and blue element lost by each pixel to be reconstructed, the method also includes a step of selecting a plurality of pixels arranged in a matrix around the pixel to be reconstructed by using a median filter based on each pixel to be reconstructed to calculate color difference values respectively, obtaining the red and green difference values and the blue and green difference values of each pixel in the matrix, and further selecting at least one intermediate value from the red and green difference values and the blue and green difference values to add with the green element in the pixel so as to further obtain accurate information of the red element and the blue element.

Wherein the matrix selected by the median filter is selected from one of 3 × 3, 4 × 4 and 5 × 5.

The image pixel array is formed by arranging a green pixel row, a blue pixel row and a red pixel row and a green pixel row in sequence.

The invention relates to a pixel color information reconstruction method, which is applied to the color reconstruction of an image pixel array, wherein the image pixel array comprises a first pixel, a second pixel and a third pixel, and color elements lost by the pixels in the image pixel array are complemented by the reconstruction method, so that each pixel comprises a first element, a second element and a third element, and the pixel color information reconstruction method is characterized by comprising the following steps:

selecting a plurality of directions for each second pixel/third pixel of the first pixel to be reconstructed, and distributing weight proportions according to the positions of the pixels around the red/blue pixels so as to calculate the color signal difference in each direction according to the weight proportions;

distributing weighted values in the equal direction according to the inverse proportion of the color signal difference, averaging two adjacent first pixel values of the red/blue pixel along the equal direction to obtain a plurality of average values, further multiplying the average values by the weighted values in the direction to which the average values belong, and then adding the average values to obtain a weighted sum, wherein the weighted sum is the first pixel information in the second/third pixel to be reconstructed; and

the second element and the third element lost by each pixel to be reconstructed are complemented by a bilinear interpolation method, so that each pixel respectively comprises the first element, the second element and the third element.

The first, second and third pixels are respectively green pixel, red pixel and blue pixel, and the image pixel array is formed by arranging a green pixel row, a blue pixel row and a red pixel row and a green pixel row in sequence.

Wherein the directions include a horizontal direction and a vertical direction.

Wherein, in the step of calculating the color signal difference according to the directions, the calculation is performed according to the values of the first pixel and the second/third pixels around the pixel to be reconstructed.

After complementing the second element and the third element lost by each pixel to be reconstructed, the method also comprises a step of selecting a plurality of pixels arranged in a matrix around the pixel to be reconstructed by using a median filter based on each pixel to be reconstructed to calculate color difference values respectively, and then selecting at least one intermediate value from the color difference values to add with the first element in the pixel so as to further obtain accurate information of the second element and the third element.

Drawings

The following detailed description of the embodiments, taken in conjunction with the accompanying drawings, will make it easier to understand the objects, technical contents, features, and effects achieved by the present invention, wherein:

FIG. 1 is a diagram illustrating color interpolation in an image pixel array according to the prior art.

FIG. 2 is a schematic diagram of an interpolated image pixel array according to the present invention.

FIG. 3 is a flow chart of reconstructing pixel color information according to the present invention.

FIG. 4 is a diagram illustrating the reconstruction of color information in an image pixel array according to the present invention.

Detailed description of the invention

Before a pixel is reconstructed by interpolation, the color signal difference quantity in the horizontal direction and the vertical direction is calculated according to the position near and far distribution weight proportion of the surrounding pixels, the weight value is distributed according to the inverse proportion of the color signal difference quantity, then the green elements obtained in the horizontal direction and the vertical direction are multiplied by the weight value in the direction to which the green elements belong, and the obtained weighted sum becomes the green element interpolation value in the red/blue pixel.

Referring to fig. 2, an image pixel array 20 generated after light passes through a color filter is composed of a plurality of pixels 22, including red pixels 24, blue pixels 26 and green pixels 28, and is formed by arranging a row of green and blue pixels G, B and a row of red and green pixels R, G in sequence, thereby forming an array having four green pixels 28 around each red pixel 24 or blue pixel 26. Since each primary pixel 22 has only a single color element, the other two color elements lost by each pixel 22 must be complemented so that each pixel 22 contains three elements, red, blue and green.

Because the luminance value of the green primary color is the highest among the three primary colors of red, blue and green, and more accurate luminance data can be provided, the interpolation method of the present invention firstly reconstructs the green color frequency (G channel) by proper weight calculation, and then calculates the color difference domain (color difference domain) of red and blue by accurate green color frequency data, so as to obtain accurate green, red and blue color frequency data.

The pixel color information reconstruction method of the invention is software, and is usually realized by an image processing device. Referring to fig. 3, a flow chart of the pixel color information reconstruction method is shown, which applies interpolation and adds appropriate weighting judgment in the interpolation process, and with reference to fig. 2, the method of the present invention includes the following steps: first, the process of reconstructing pixel color information is started in step S10, a signal of the image pixel array 20 transmitted from the image sensor array is received, and then the image pixel array 20 is scanned from left to right in order to reconstruct the green element first, as shown in step S12, wherein each scan of a blue pixel 26 or a red pixel 24 selects the horizontal and vertical directions for the pixel, and the weight ratio is assigned according to the positions of the surrounding pixels, the surrounding pixels farther away are assigned with smaller weight ratio, and the surrounding pixels closer are assigned with larger weight ratio; next, as shown in step S14, the color signal difference Δ H and Δ V in the horizontal and vertical directions are calculated according to the weight ratios, and the color signal difference is calculated according to the values of the green pixel 28, the red pixel 24 and the blue pixel 26 around the pixel to be reconstructed.

After the color signal differences in the horizontal and vertical directions are calculated, in step S16, the horizontal and vertical weighted values are assigned according to the inverse proportion of the color signal differences to perform a weighted sum, i.e., the one with the larger difference is assigned the smaller weighted value, and then the two adjacent green pixel values of the pixel 24/26 to be reconstructed are averaged along the horizontal and vertical directions to obtain two average values, and the two average values are multiplied by the weighted values in the corresponding directions and added together, so that the weighted sum becomes the green element information in the red/blue pixel 24/26. The steps S12 to S16 are repeated to sequentially scan and reconstruct the missing green elements in each of the red and

blue pixels

24, 26 until the reconstruction of the green elements in all the red and

blue pixels

24, 26 is completed. The scanning direction can also be from right to left or from top to bottom.

Then, step S18 is performed to sequentially complement the red and blue elements lost by each pixel to be reconstructed by a bilinear (bilinear) interpolation method, which uses four or two surrounding points to calculate the value to be interpolated by a linear interpolation method, so that each pixel 22 in the image pixel array 20 contains three elements, namely green, red and blue.

After step S18 is completed, each pixel 22 has three color information of green, blue and red, so as to make the blue and red element values obtained by the bilinear method more accurate, therefore, the present invention further includes a color difference correction step after step S18 is completed, as shown in step S20, a median filter (median filter) is used to select a plurality of pixels arranged in a matrix around the pixel, and a color difference (color difference) and a blue-green difference of each pixel 22 in the matrix are obtained, and then an intermediate value is selected from the red-green difference and the blue-green difference, and the intermediate value is added to the green element in the pixel to be reconstructed, so as to further obtain accurate information of the red element and the blue element. After the color difference correction of each red and blue interpolation value is completed in sequence, each pixel 22 can have accurate green, red and blue color information, and the whole color reconstruction process is ended in step S22. The matrix selected by the median filter is typically 3 × 3, 4 × 4, or 5 × 5.

Now that the spirit of the pixel color information reconstruction method of the present invention has been described, the interpolation method of the present invention is described in detail with a specific example, so that those skilled in the art can obtain sufficient knowledge to implement the method with reference to the description of the example.

The practical application of the pixel color information reconstruction method of the present invention as color reconstruction in an image processing system is that, as shown in the fourth figure, an image passes through a Color Filter Array (CFA) and an image sensor array to form an image pixel array as shown in the figure, and the lost color component is required to be reconstructed before the image is subjected to subsequent processing.

The method comprises the following steps: the green (G) color values are reconstructed from the weighted sums.

In FIG. 4, G₄₄Originally absent, R₄₄Position G₄₄Is reconstructed by reference to R₄₄G of the periphery₃₄、G₅₄、G₄₃、G₄₅Four green luminance values are processed by a certain weighting, and the calculation formula of the weighting principle is as follows:

ΔH＝w₁×abs(G₄₃-G₄₅)+w₂×[abs(R₄₄-R₄₂)+abs(R₄₄-R₄₆)] (1)

ΔV＝w₁×abs(G₃₄-G₅₄)+w₂×[abs(R₄₄-R₂₄)+abs(R₄₄-R₆₄)] (2)

wherein Δ H represents R₄₄The difference of the signals of the surrounding pixels in the horizontal direction, i.e. the variation of the signal trend in the horizontal direction, and Δ V represents R₄₄The signal difference of the peripheral pixels in the vertical direction; w is a₁And w₁Each represents a weight value of w₁1/2 and w₁＝1/4。

As shown in the figure, the first and second,for R₄₄In the horizontal direction of (1), due to G₄₃、G₄₅Closer, so the weight value is 1/2; and R is₄₂And R₄₆The weighting value is 1/4, so the difference Δ H and Δ V of the horizontal and vertical signals can be closer to the real condition by the way of assigning the weighting ratio according to the position, and the disadvantage of determining Δ H and Δ V without adding proper judgment or processing can be effectively solved.

After the horizontal and vertical signal variations Δ H and Δ V are calculated, the interpolation values are weighted and calculated as follows:

wherein,

in the formula (3), k₁And k₂The values are weighted values of horizontal and vertical components, which vary with the difference Δ H, Δ V of the horizontal and vertical signals, respectively, such that the value G is the inverse of the difference Δ H, Δ V of the signalsThe weight values of the horizontal and vertical components are proportionally assigned to perform weighted summation, so that the G component in the direction of large difference amount has a smaller weight value, and the G component in the direction of small difference amount has a larger weight value. Therefore, the formula (3) is used to reconstruct the pixel G along the horizontal and vertical directions₄₄Two adjacent green pixel values G₄₃、G₄₅And G₃₄、G₅₄Respectively averaging to obtain two average values, i.e. horizontal component and vertical component, and multiplying the two average values by the weight value k of the direction to which the two average values belong₁And k₂Then adding the weighted values to obtain a weighted sum G₄₄I.e. becomes the red pixel R to be reconstructed₄₄Green element information of (1), and further B₄₄Loss of position G₄₄To be complemented.

The missing G values at the R and B positions in each row are then calculated by the weighted interpolation method, so that the R pixels and the B pixels in the image pixel array each contain the missing green element G. At this point, the reconstruction of the green color of all pixels in the image pixel array has been completed.

Step two: the red (G) and blue (B) color values are determined by a bilinear method.

After the G color reconstruction is completed, the missing red (G) and blue (B) color values are then determined by bilinear method, and referring to fig. 4 again, R₃₃Originally absent, B₃₃R of position₃₃Is reference B₃₃The nearest 4R values are calculated as follows:

R_{33} = \frac{R_{22} + R_{24} + R_{42} + R_{44}}{4}

due to B₃₃The nearest 4R positions are respectively R₂₂、R₂₄、R₄₂、R₄₄So that the four points are averaged to obtain R through interpolation₃₃The value is obtained.

The interpolation of pixels at the edge of the image is averaged with the R values at the horizontal or vertical sides, for example:

R_{23} = \frac{R_{22} + R_{24}}{2}

R_{32} = \frac{R_{22} + R_{42}}{2}

thus, the missing R values of the G and B pixels in the image pixel array are calculated one by one in sequence, so that each G position and each B position have a red element R.

Similarly, the reconstruction of the blue element is also the same, and therefore is not described herein. Thus, the preliminary reconstruction of the blue (B) color and the red (R) color in the image pixel array is completed.

Step three: and solving the color difference information by a median filter.

After the interpolation calculation of the green, blue and red elements of each pixel is completed, in order to make the blue and red element values obtained by utilizing the bilinear method more accurate, therefore, the invention uses each pixel to be reconstructed as a reference, utilizes a median filter (median filter) to select 5 × 5 blocks (pixels) from an image pixel array, firstly establishes a color difference domain (color difference domain) for each pixel individually, and then selects a color difference median value from the color difference domain, and the calculation procedure is as follows:

(R_i-G_i)_median＝median filter(R-G) (4)

(B_i-G_i)_median＝median filter(B-G) (5)

the above-mentioned procedures (4) and (5) are used to match the median selection of the median filter based on the precise G value, so as to select the best median of the red color difference domain and the median of the blue color difference domain.

Step four: the exact R and B values were calculated.

After the optimal red and blue difference domain median values are selected by the median filter, the exact R and B values are then calculated using the following equations (6) and (7).

<math> <mrow> <mover> <msub> <mi>R</mi> <mi>i</mi> </msub> <mo>&OverBar;</mo> </mover> <mo>=</mo> <msub> <mrow> <mo>(</mo> <msub> <mi>R</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>G</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> <mi>median</mi> </msub> <mo>+</mo> <msub> <mi>G</mi> <mi>i</mi> </msub> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>6</mn> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <msub> <mover> <mi>B</mi> <mo>&OverBar;</mo> </mover> <mi>i</mi> </msub> <mo>=</mo> <msub> <mrow> <mo>(</mo> <msub> <mi>B</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>G</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> <mi>median</mi> </msub> <mo>+</mo> <msub> <mi>G</mi> <mi>i</mi> </msub> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>7</mn> <mo>)</mo> </mrow> </mrow> </math>

Thus, each R pixel and each B pixel interpolated in the image pixel array are subjected to chromatic aberration correction one by one, so that each pixel contains accurate RGB three-color information, and the whole image reconstruction procedure is further completed.

Therefore, before interpolation calculation, the invention calculates the signal difference quantity of the peripheral of the position of the G value to be interpolated in the horizontal and vertical directions according to the position near and far distribution weight proportion, then distributes the horizontal weight value and the vertical weight value by the signal difference quantity, provides the weight value of the horizontal G value and the vertical G value obtained by interpolation multiplied by the corresponding directions, obtains the green element interpolation value by using the weighted sum as the red/blue pixel, then carries out interpolation calculation of the red element and/or the blue element lost by other pixels, and establishes more accurate red and blue information by a median filter. Therefore, the invention can make the calculation of the signal value in the whole interpolation process closer to the real condition to provide accurate interpolation color value, thereby achieving the effects of high resolution and uniform color, and further improving the problems of false color, distortion and the like caused by large error of the known interpolation system.

While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A pixel color information reconstruction method is used for complementing lost color elements in a red pixel, a blue pixel and a green pixel in an image pixel array, so that each pixel respectively comprises three elements of red, blue and green, and the pixel color information reconstruction method is characterized by comprising the following steps:

2. The pixel color information reconstruction method of claim 1, wherein the directions comprise a horizontal direction and a vertical direction.

3. The method according to claim 1, wherein in the step of calculating the difference of color signals in the equal directions, for each pixel to be reconstructed, the pixels around the pixel farther away are assigned with smaller weight ratio, and the pixels around the pixel closer to the pixel closer are assigned with larger weight ratio.

4. The method as claimed in claim 1, wherein in the step of calculating the color signal difference according to the directions, the color signal difference is calculated according to the values of the green pixel and the red/blue pixel around the pixel to be reconstructed.

5. The method as claimed in claim 1, wherein the method further comprises a step of calculating color difference values by using a median filter to select a plurality of pixels arranged in a matrix around each pixel to be reconstructed based on each pixel to be reconstructed, and further calculating accurate information of red and blue elements by selecting at least one intermediate value from the red and green difference values and the blue and green difference values to be added to the green elements in the pixels.

6. The pixel color information reconstruction method of claim 5, wherein the matrix selected by the median filter is selected from one of 3 x 3, 4 x 4 and 5 x 5.

7. The method of claim 1, wherein the image pixel array is formed by sequentially arranging a green and blue pixel row and a red and green pixel row.

8. A pixel color information reconstruction method is applied to color reconstruction of an image pixel array, the image pixel array comprises a first pixel, a second pixel and a third pixel, and color elements lost by the pixels in the image pixel array are complemented by the reconstruction method, so that each pixel comprises a first element, a second element and a third element, and the pixel color information reconstruction method is characterized by comprising the following steps:

9. The method of claim 8, wherein the first, second and third pixels are green, red and blue pixels, respectively, and the image pixel array is an array of green and blue pixels arranged in sequence.

10. The pixel color information reconstruction method of claim 8, wherein the directions include a horizontal direction and a vertical direction.

11. The method according to claim 8, wherein in the step of calculating the difference of color signals in the directions, for each pixel to be reconstructed, the more distant surrounding pixels are assigned with smaller weight ratio, and the more distant surrounding pixels are assigned with larger weight ratio.

12. The method as claimed in claim 8, wherein the step of calculating the difference of color signals according to the directions is performed based on the values of the first pixel and the second/third pixels around the pixel to be reconstructed.

13. The method as claimed in claim 8, wherein after the second and third elements missing from each pixel to be reconstructed are compensated, the method further comprises a step of selecting a plurality of pixels arranged in a matrix around the pixel to be reconstructed by using a median filter based on each pixel to be reconstructed to calculate color difference values, and further selecting at least one intermediate value from the color difference values to add to the first element in the pixel to further obtain accurate information of the second and third elements.

14. The pixel color information reconstruction method of claim 13, wherein the matrix selected by the median filter is selected from one of 3 x 3, 4 x 4 and 5 x 5.