JP4334151B2 - Image interpolation device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image interpolation device provided to improve the image quality of a color image, for example, in a digital camera.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, a digital camera is known in which a Bayer array color filter is provided in front of an image sensor such as a CCD in order to detect a color image. This color filter is configured by arranging red (R), green (G) and blue (B) color filter elements in a checkered pattern, and these color filter elements correspond to the photodiodes of the image sensor, respectively. Yes. Therefore, a pixel signal of a color corresponding to each color filter element is generated by the photodiode, and for example, an R pixel signal is generated by the photodiode corresponding to the R color filter element.
[0003]
In order to display a high-quality color image rather than using the pixel signal output from the image sensor as it is, interpolation processing may be performed on the pixel signal. In a normal interpolation process, a G signal is generated by taking an arithmetic average of G signals of pixels located in the periphery of a pixel from which an R signal is obtained by a photodiode. The B signal is generated by taking the arithmetic mean of the B signals of the pixels to be processed. However, in an image having a high spatial frequency, for example, the actual color of the pixel to be obtained by the interpolation process and the actual color of the surrounding pixels may be greatly different. Depending on the value of the color signal of the pixel, color blur occurs in the reproduced image.
[0004]
[Problems to be solved by the invention]
Accordingly, an object of the present invention is to reduce the occurrence of color blur in a reproduced image due to interpolation processing.
[0005]
[Means for Solving the Problems]
An image interpolation apparatus according to the present invention includes a color filter, an image sensor, a pattern setting unit, a G interpolation processing unit, an R / B interpolation processing unit, a first interpolation / correction processing unit, and a second interpolation / correction processing unit. Prepare.
The color filter includes a first row in which red (R) and green (G) color filter elements are alternately arranged in the horizontal direction, and adjacent to the upper and lower sides of the first row, and in the horizontal direction G and blue (B ) Color filter elements are alternately arranged in the second row. The image sensor generates R, G, and B signals that are pixel signals corresponding to each color filter element. The pattern setting means includes a first pattern in which an R signal is located on the upper left side in a 2 × 2 pixel matrix, and a second pattern in which a G signal is located on the upper right side from the R, G, and B signals generated by the image sensor. Images belonging to the third pattern in which the G signal is located in the lower left and the fourth pattern in which the B signal is located in the lower right are extracted. The G interpolation processing unit obtains a G signal for each pixel of the image belonging to the first and fourth patterns using the G signal of the adjacent pixel. The R / B interpolation processing means obtains R and B signals for each pixel of the images belonging to the second and third patterns using the R and B signals of adjacent pixels. For each pixel of the image belonging to the first pattern, the first interpolation / correction processing means extracts a similar pixel having the closest luminance value from adjacent pixels, and determines the change rate of the luminance value between the similar pixels. Based on the corresponding first correction value and the B, G, and R signals of similar pixels, the B signal is obtained, and the G signal obtained by the G interpolation processing unit and the R signal generated by the image sensor are corrected. To do. For each pixel of the image belonging to the fourth pattern, the second interpolation / correction processing means extracts a similar pixel having the closest luminance value from adjacent pixels, and determines the change rate of the luminance value between the similar pixels. Based on the corresponding second correction value and the R, G, B signals of similar pixels, the R signal is obtained, and the G signal obtained by the G interpolation processing means and the B signal generated by the image sensor are corrected. To do.
[0006]
In the G interpolation processing means, adjacent pixels are included in images belonging to the second and third patterns. In the R / B interpolation processing means, adjacent pixels are included in the images belonging to the first and fourth patterns. In the first interpolation / correction processing means and the second interpolation / correction processing means, adjacent pixels are included in the images belonging to the second and third patterns.
[0007]
The first interpolation / correction processing unit and the second interpolation / correction processing unit preferably extract a similar pixel having the closest luminance value by using the value of the G signal in the adjacent pixel. By estimating that the hue of the similar pixel is close to the hue of the pixel to be obtained by the interpolation process, the color blur is reduced.
[0008]
The first interpolation / correction processing means obtains a B signal and adds a G signal to each pixel of an image belonging to the first pattern by adding a first correction value to the B, G, and R signals of similar pixels. Correct the R signal.
[0009]
For example, for each pixel of the image belonging to the first pattern, the first correction value is obtained by multiplying the change rate of the G signal with the similar pixel by the luminance value of the similar pixel.
[0010]
The first interpolation / correction processing means obtains the B signal and corrects the G signal and the R signal according to the following formula, for example.
Y = 0.299 × R (x ′, y ′) + 0.587 × G (x ′, y ′) + 0.114 × B (x ′, y ′)
YG = Y × (G (x, y) −G (x ′, Y ′)) / G (x ′, y ′)
r ′ = YG + R (x ′, y ′)
g ′ = YG + G (x ′, y ′)
b = YG + B (x ′, y ′)
Where Y is the luminance value of the similar pixel, YG is the first correction value, R (x ′, y ′), G (x ′, y ′), B (x ′, y ′) are the R, G, B signals, G (x, y) are G signals of the pixels of the image belonging to the first pattern, b is B signals of the pixels belonging to the first pattern obtained by the first interpolation / correction processing means, r ′ Is a modified R signal and g ′ is a modified G signal.
[0011]
The second interpolation / correction processing means obtains the R signal and adds the G signal to each of the pixels of the image belonging to the fourth pattern by adding the second correction value to the R, G, B signals of the similar pixels. Correct the B signal.
[0012]
For example, for each pixel of the image belonging to the fourth pattern, the second correction value is obtained by multiplying the change rate of the G signal with the similar pixel by the luminance value of the similar pixel.
[0013]
The second interpolation / correction processing means obtains the R signal according to, for example, the following formula and corrects the G signal and the B signal.
Y = 0.299 × R (x ′, y ′) + 0.587 × G (x ′, y ′) + 0.114 × B (x ′, y ′)
YG = Y × (G (x, y) −G (x ′, y ′)) / G (x ′, y ′)
r = YG + R (x ', y')
g ′ = YG + G (x ′, y ′)
b '= YG + B (x', y ')
Where Y is the luminance value of the similar pixel, YG is the second correction value, R (x ′, y ′), G (x ′, y ′), and B (x ′, y ′) are R, G, B signals, G (x, y) are G signals of the pixels of the image belonging to the fourth pattern, r is an R signal of the pixels of the image belonging to the fourth pattern obtained by the second interpolation / correction processing means, g ′ is a modified G signal and b ′ is a modified B signal.
[0014]
The first interpolation / correction processing means extracts similar pixels using the values of the G signal and R signal in adjacent pixels. Further, the second interpolation / correction processing means is configured to output the G signal in the adjacent pixel and B Similar pixels are extracted using the value of the signal.
[0015]
The first correction value is based on the first reference value obtained based on the G signal and R signal in the pixel and similar pixels and the G signal and R signal of the similar pixel for each pixel of the image belonging to the first pattern. The luminance value is obtained by multiplying the ratio with the second reference value obtained in this way.
[0016]
The first interpolation / correction processing means may obtain the B signal according to the following formula and modify the G signal and the R signal.
Y = 0.299 × R (x ′, y ′) + 0.587 × G (x ′, y ′) + 0.114 × B (x ′, y ′)
YG = Y × (0.587 × (G (x, y) −G (x ′, y ′) + 0.299 × (R (x, y) −R (x ′, y ′)))
/(0.587×G(x',y')+0.299×R(x',y '))
r ′ = YG + R (x ′, y)
g ′ = YG + G (x ′, y ′)
b = YG + B (x ′, y ′)
Where Y is the luminance value of the similar pixel, YG is the first correction value, R (x ′, y ′), G (x ′, y ′), B (x ′, y ′) are the R, G, B signals, G (x, y) are G signals of pixels of the image belonging to the first pattern, b is B signals of pixels of the image belonging to the first pattern obtained by the first interpolation / correction processing means, r ′ is a modified R signal and g ′ is a modified G signal.
[0017]
The second interpolation / correction processing means relates to each pixel of the image belonging to the fourth pattern, the first reference value obtained based on the G signal and B signal in the pixel and similar pixels, and the G signal and B of similar pixels. The luminance value is obtained by multiplying the ratio with the second reference value obtained based on the signal.
[0018]
The second interpolation / correction processing means may obtain the R signal according to the following equation and correct the G signal and the B signal.
Y = 0.299 × R (x ′, y ′) + 0.587 × G (x ′, y ′) + 0.114 × B (x ′, y ′)
YG = Y × (0.587 × (G (x, y) −G (x ′, y ′)) + 0.114 × (B (x, y) −B (x ′, y ′))
/(0.587×G(x',y')+0.114×B(x',y '))
r = YG + R (x ', y')
g ′ = YG + G (x ′, y ′)
b '= YG + B (x', y ')
Where Y is the luminance value of the similar pixel, YG is the second correction value, R (x ′, y ′), G (x ′, y ′), and B (x ′, y ′) are R, G, B signals, G (x, y) are G signals of the pixels of the image belonging to the fourth pattern, r is an R signal of the pixels of the image belonging to the fourth pattern obtained by the second interpolation / correction processing means, g ′ is a modified G signal and b ′ is a modified B signal.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows a schematic configuration of a digital camera 10 provided with an image interpolation device according to the first embodiment of the present invention.
[0020]
A CCD 12 as an image pickup device is disposed behind the photographing lens 11, and a color filter 13 is provided on the light receiving surface of the CCD 12. That is, the optical image of the subject S obtained by the photographing lens 11 is formed on the light receiving surface of the CCD 12, and a color pixel signal constituting the subject image is generated in the photodiode of the CCD 12. The pixel signal is read from the CCD 12 and converted into a digital signal by the A / D converter 14. The pixel signal is subjected to white balance adjustment in the white balance adjustment circuit 15 and stored in the image memory 16.
[0021]
The pixel signal is read from the image memory 16 and input to the pattern setting unit 17. In the pattern setting unit 17, as will be described later, pixel signals constituting a predetermined array pattern are extracted from all pixel signals and classified into first, second, third and fourth patterns. In the interpolation processing unit 18, the pixel signal extracted in the pattern setting unit 17 is subjected to interpolation processing according to the pattern. The interpolated pixel signal output from the interpolation processing unit 18 is recorded on a recording medium D such as a memory card. Note that the white balance adjustment circuit 15 may be provided not in the subsequent stage of the A / D converter 14 but in the subsequent stage of the interpolation processing unit 18.
[0022]
FIG. 2 shows an array of color filter elements in the color filter 13 and pixel signals of images belonging to each pattern extracted by the pattern setting unit 17. The color filter 13 is composed of red (R), green (G), and blue (B) color filter elements arranged according to the Bayer array. That is, the color filter 13 includes a first row 13a in which R and G color filter elements are alternately arranged in the horizontal direction, and adjacent to the upper and lower sides of the first row 13a, and the G and B color filter elements in the horizontal direction. Are alternately arranged in the second row 13b, and the CCD 12 generates R, G, and B signals that are pixel signals corresponding to the respective color filter elements.
[0023]
The color filter 13 is divided into a 2 × 2 pixel matrix M1 in which color filter elements R are arranged in the upper left, G in the upper right and lower left, and B in the lower right. In the pattern setting unit 17, from the R, G, and B signals generated by the CCD 12, the first pattern in which the R signal is located at the upper left in the pixel matrix M1, the second pattern in which the G signal is located in the upper right, and the G in the lower left. Images belonging to the third pattern in which the signal is located and the fourth pattern in which the B signal is located in the lower right are extracted.
[0024]
In the pattern setting unit 17, the first pattern is extracted from all the pixel signals constituting one image according to the logical expression (1) expressed in C language.
! (X% 2) &&! (Y% 2) (1)
In the logical expression (1), “x” and “y” respectively indicate an abscissa and ordinate with the left corner of the image as the origin. That is, the coordinates of the R signal at the left corner are (0, 0) and the coordinates of the G signal at the right are (1,0). In the logical expression (1), “!” Indicates a logical negation, “%” indicates a remainder, and “&&” indicates a logical product. Therefore, according to the logical expression (1), an image composed of pixel signals (that is, R signals) whose x coordinate and y coordinate are both even numbers is extracted.
[0025]
Similarly, the second pattern, the third pattern, and the fourth pattern are extracted according to the logical expressions (2), (3), and (4).
(X% 2) &&! (Y% 2) (2)
! (X% 2) && (y% 2) (3)
(X% 2) && (y% 2) (4)
[0026]
Processing in the interpolation processing unit 18 will be described with reference to FIGS. FIG. 3 shows the coordinates and colors of the pixels constituting one image stored in the image memory 16. As will be described later, the interpolation processing unit 18 performs the G interpolation processing routine (FIG. 4), the R / B interpolation processing routine (FIG. 5), the first interpolation / correction processing routine (FIGS. 7 and 8), the first 2 interpolation / correction processing routine (FIG. 9) is executed. In these processing routines, the coordinates of the target pixel for which pixel data is to be obtained by interpolation or the like are (x, y).
[0027]
FIG. 4 is a flowchart of the G interpolation processing routine. In the G interpolation processing routine, interpolation processing relating to images belonging to the first and fourth patterns is performed.
[0028]
In step 101, the absolute value of the difference between G (x-1, y), which is the G signal of the pixel adjacent to the left side of the target pixel, and G (x + 1, y), which is the G signal of the pixel adjacent to the right side. Is the absolute value of the difference between G (x, y-1) which is the G signal of the pixel adjacent to the upper side of the target pixel and G (x, y + 1) which is the G signal of the pixel adjacent to the lower side It is also determined whether or not is smaller. The case where the coordinates of the target pixel is (2, 2) (that is, the image of the first pattern) will be described as an example. The target pixel is R (2, 2). In step 101, | G (1, 2) − It is determined whether G (3,2) | is smaller than | G (2,1) -G (2,3) |.
[0029]
When the target pixel is included in the image belonging to the first pattern, that is, the R signal, the pixels adjacent to the left and right are included in the image belonging to the second pattern, and the pixels adjacent to the top and bottom thereof are the third pattern. include. On the other hand, when the target pixel is included in the image belonging to the fourth pattern, that is, a B signal, the pixels adjacent to the left and right are included in the image belonging to the third pattern, and the pixels adjacent to the top and bottom thereof are the second. Included in the pattern.
[0030]
When it is determined in step 101 that the absolute value of the difference between the pixel signals adjacent to the left and right of the target pixel is smaller than the absolute value of the difference between the pixel signals adjacent to the upper and lower sides, step 102 is executed. An arithmetic average of G (x−1, y) and G (x + 1, y) that are pixel signals to be obtained is obtained as the G signal g of the target pixel. On the other hand, when it is determined that the absolute value of the difference between the pixel signals adjacent to the left and right of the target pixel is not smaller than the absolute value of the difference between the pixel signals adjacent vertically, in step 103, the pixels adjacent vertically An arithmetic average of the signals G (x, y-1) and G (x, y + 1) is obtained as the G signal g of the target pixel. Execution of step 102 or 103 ends the G interpolation processing routine.
[0031]
As described above, in the G interpolation processing routine, for each pixel of the image belonging to the first and fourth patterns, the average value of the two G signals of the smaller difference group among the four G signals adjacent to the left and right and the top and bottom is obtained. It is obtained as the G signal of the target pixel. When the G interpolation processing routine is executed for all the pixels of the images belonging to the first and fourth patterns, the G signals for all the pixels constituting one image are obtained.
[0032]
FIG. 5 is a flowchart of the R / B interpolation processing routine. In the R / B interpolation processing routine, interpolation processing relating to images belonging to the second and third patterns is performed.
[0033]
In step 201, it is determined whether or not the target pixel is included in an image belonging to the second pattern. When the target pixel is included in the image belonging to the second pattern, that is, when the target pixel is the upper right G signal of the pixel matrix M1 (FIG. 2), step 202 is executed. On the other hand, when the target pixel is not included in the second pattern, that is, is included in the third pattern and is the lower left G signal of the pixel matrix M1 (FIG. 2), step 203 is executed.
[0034]
In step 202, the arithmetic mean of R (x-1, y) and R (x + 1, y), which are pixel signals (first pattern image) adjacent to the left and right of the target pixel, is the R signal r of the target pixel. As required. Also, an arithmetic average of B (x, y-1) and B (x, y + 1), which are pixel signals (fourth pattern image) adjacent to the upper and lower sides of the target pixel, is obtained as the B signal b of the target pixel. .
[0035]
In step 203, the arithmetic mean of R (x, y-1) and R (x, y + 1), which are pixel signals (first pattern image) adjacent to the upper and lower sides of the target pixel, is the R signal r of the target pixel. As required. Further, an arithmetic average of B (x−1, y) and B (x + 1, y), which are pixel signals (fourth pattern image) adjacent to the left and right of the target pixel, is obtained as the B signal b of the target pixel. .
[0036]
Execution of step 202 or 203 ends the R / B interpolation processing routine. As described above, in the R / B interpolation processing routine, the R and B signals of the target pixel are obtained by interpolating the pixel signals adjacent to the left and right and top and bottom for each pixel of the image belonging to the second and third patterns. When the G interpolation processing routine (FIG. 4) is completed and the R / B interpolation processing routine is executed for all the pixels of the image belonging to the second and third patterns, as understood from FIG. The G signal is obtained for all the pixels constituting one image, and the R signal r and the B signal b are obtained for each pixel of the image belonging to the second and third patterns. That is, the B signal has not yet been obtained for the image belonging to the first pattern, and the R signal has not yet been obtained for the image belonging to the fourth pattern.
[0037]
7 and 8 show the B signal for each pixel of the image belonging to the first pattern, and the G signal obtained by the G interpolation processing routine (FIG. 4) and the R signal generated by the CCD 12 are corrected. 2 is a flowchart of a first interpolation / correction processing routine.
[0038]
In step 301, the absolute value of the difference between G (x−1, y), which is the G signal of the pixel adjacent to the left side of the target pixel, and G (x, y), which is the G signal of the target pixel, is calculated. It is determined whether or not the absolute value of the difference between G (x + 1, y), which is the G signal of the pixel adjacent to the right side, and G (x, y), which is the G signal of the target pixel, is smaller. In other words, it is determined whether the G signal of any pixel adjacent to the left and right of the target pixel has a value close to the G signal of the target pixel. When the G signal of the left pixel is closer to the G signal of the target pixel, in step 302, the parameter p is set to -1. On the other hand, when the G signal of the right pixel is closer to or equal to the G signal of the target pixel, the parameter p is set to 1 in step 303.
[0039]
In step 304, the absolute value of the difference between G (x, y-1), which is the G signal of the pixel adjacent to the upper side of the target pixel, and G (x, y), which is the G signal of the target pixel, is It is determined whether or not the absolute value of the difference between G (x, y + 1) which is the G signal of the pixel adjacent on the lower side and G (x, y) which is the G signal of the target pixel is smaller. That is, it is determined whether or not the G signal of any pixel adjacent to the top and bottom of the target pixel has a value close to the G signal of the target pixel. When the upper G signal is closer to the G signal of the target pixel, in step 305, the parameter q is set to -1. On the other hand, when the lower G signal is closer to or equal to the G signal of the target pixel, the parameter q is set to 1 in step 306.
[0040]
In step 307, the magnitude of the G signal is compared with respect to a pixel adjacent to any one of the left and right of the target pixel, a pixel adjacent to any one of the top and bottom, and the target pixel. The G signal of the left pixel of the target pixel is closer to the G signal of the target pixel than the G signal of the right pixel, and the G signal of the upper pixel of the target pixel is higher than the G signal of the lower pixel. When close to the signal, the parameters p and q are both -1, and the absolute value of the difference between G (x-1, y) and G (x, y) is G (x, y-1) and G ( It is determined whether or not the absolute value of the difference from x, y) is smaller. When the left G signal is closer to the G signal of the target pixel, in step 308, the parameter s is set to p, that is, -1. On the other hand, when the upper G signal is closer or equal to the G signal of the target pixel, the parameter s is set to 2 × q, that is, −2 in step 309.
[0041]
When the parameters p and q are both 1, the parameters p and q are −1 and 1 respectively, and the parameters p and q are respectively 1 and −1, the same steps 307, 308 or 309 are executed. Is done. In this way, in the first interpolation / correction processing routine, regarding the G signal of the target pixel, the value of the G signal of the adjacent pixel included in the images belonging to the second and third patterns is examined, and is adjacent to the left side. The parameter s is -1 if the G signal of the pixel is closest, the parameter s is 1 if the G signal of the pixel adjacent on the right side is closest, and the parameter s is -2 if the G signal of the pixel adjacent on the upper side is closest. In addition, the parameter s is set to 2 if the G signal of the adjacent pixel on the lower side is closest.
[0042]
In this specification, among four adjacent pixels, a pixel having a G signal closest to the G signal of the target pixel is referred to as a similar pixel. The similar pixel preferably has a luminance value closest to the luminance value of the target pixel. However, since the luminance value of the target pixel is unknown, the similar pixel is extracted by determining the luminance value using the G signal.
[0043]
In step 310, it is determined whether or not the parameter s is -1, that is, whether or not the similar pixel is a pixel adjacent to the left side of the target pixel. When the parameter s is -1, step 311 is executed, and R (x-1, y), G (x-1, y), B (x-1, y) is used to determine the luminance value Y, and G (x−1, y) which is the G signal of the similar pixel, G (x, y) which is the G signal of the target pixel, and the luminance value Y are used. A correction value YG is obtained.
[0044]
The luminance value Y and the correction value YG are calculated according to the equations (5) and (6), respectively.
Y = 0.299 × R (x-1, y) + 0.587 × G (x-1, y) + 0.114 × B (x-1, y) (5)
YG = Y × (G (x, y) -G (x-1, y)) / G (x-1, y) (6)
[0045]
The formula (5) is conventionally known. As understood from the equation (6), the correction value YG is obtained by multiplying the change rate of the G signal between the target pixel and the similar pixel by the luminance value Y of the similar pixel.
[0046]
In step 312, the G signal and R signal of the target pixel are calculated according to equations (7) and (8), and a corrected G signal g ′ and a corrected R signal r ′ are obtained. Further, the B signal of the target pixel is calculated according to the equation (9), and the interpolated B signal b is obtained. Thereby, the first interpolation / correction processing routine ends.
r '= YG + R (x', y ') (7)
g ′ = YG + G (x ′, y ′) (8)
b = YG + B (x ′, y ′) (9)
[0047]
Note that R (x ′, y ′), G (x ′, y ′), and B (x ′, y ′) are R (x−1, y), when step 312 is executed after step 311. G (x-1, y), B (x-1, y), and when step 312 is executed after step 314, R (x + 1, y), G (x + 1, y), B (x + 1, y), and when step 312 is executed after step 316, R (x, y-1), G (x, y-1), B (x, y-1) Yes, when step 312 is executed after step 317, R (x, y + 1), G (x, y + 1), B (x, y + 1).
[0048]
When it is determined in step 310 that the parameter s is not -1, the process proceeds to step 313, where it is determined whether or not the parameter s is 1, and whether or not the similar pixel is a pixel adjacent to the right side of the target pixel. Is determined. When the parameter s is 1, step 314 is executed, and the luminance value is calculated using the R, G, B signals of similar pixels. Y And a correction value YG is obtained.
[0049]
Luminance value Y and correction value YG are calculated according to equations (5a) and (6a), respectively.
Y = 0.299 × R (x + 1, y) + 0.587 × G (x + 1, y) + 0.114 × B (x + 1, y) (5a)
YG = Y * (G (x, y) -G (x + 1, y)) / G (x + 1, y) (6a)
[0050]
Step 312 is then executed. That is, the G signal and R signal of the target pixel are calculated according to the equations (7) and (8), and the corrected G signal g ′ and the corrected R signal r ′ are obtained. Further, the B signal of the target pixel is calculated according to the equation (9), and the interpolated B signal b is obtained. This completes the first interpolation / correction processing routine.
[0051]
When it is determined in step 313 that the parameter s is not 1, the process proceeds to step 315, where it is determined whether the parameter s is −2, and whether the similar pixel is a pixel adjacent to the upper side of the target pixel. Is determined. When the parameter s is −2, step 316 is executed, and the luminance value Y and the correction value YG are obtained using the R, G, B signals of similar pixels.
[0052]
The luminance value Y and the correction value YG are calculated according to the equations (5b) and (6b), respectively.
Y = 0.299 × R (x, y-1) + 0.587 × G (x, y-1) + 0.114 × B (x, y-1) (5b)
YG = Y * (G (x, y) -G (x, y-1)) / G (x, y-1) (6b)
[0053]
Next, as in the case of steps 311 and 314, step 312 is executed, the G signal and R signal of the target pixel are corrected according to equations (7) and (8), and the B signal of the target pixel is changed to (9). The first interpolation / correction processing routine is completed according to the equation.
[0054]
When it is determined in step 315 that the parameter s is not −2, the parameter s is 2, that is, the similar pixel is considered to be a pixel adjacent to the lower side of the target pixel. A luminance value Y and a correction value YG are obtained using the R, G, and B signals.
[0055]
The luminance value Y and the correction value YG are calculated according to the equations (5c) and (6c), respectively.
Y = 0.299 × R (x, y + 1) + 0.587 × G (x, y + 1) + 0.114 × B (x, y + 1) (5c)
YG = Y * (G (x, y) -G (x, y + 1)) / G (x, y + 1) (6c)
[0056]
Next, as in steps 311, 314, and 316, step 312 is executed, the G signal and R signal of the target pixel are corrected according to equations (7) and (8), and the B signal of the target pixel is ( 9), the first interpolation / correction processing routine is completed.
[0057]
As described above, in the first interpolation / correction processing routine, the correction signal YG is added to the R, G, and B signals of the similar pixels to obtain the B signal of the target pixel, and the G signal that has been obtained so far. The R signal is corrected.
[0058]
The first interpolation / correction processing routine is executed for all pixels of the image belonging to the first pattern. When the G interpolation processing routine (FIG. 4), the R / B interpolation processing routine (FIG. 5), and the first interpolation / correction processing routine (FIGS. 7 and 8) are completed, the first, second, and third patterns R, G, B signals are obtained for the images belonging to, and the R signals in the images belonging to the fourth pattern have not yet been determined.
[0059]
The R signal of the fourth pattern image is obtained by the second interpolation / correction processing routine. In the second interpolation / correction processing routine, the G signal obtained by the G interpolation processing routine (FIG. 4) and the B signal generated by the CCD 12 are corrected. FIG. 9 shows the latter half of the flowchart of the second interpolation / correction processing routine. The first half of the second interpolation / correction processing routine is the same as the first interpolation / correction processing routine shown in FIG.
[0060]
The processing contents of steps 410, 413, and 415 are the same as those of steps 310, 313, and 315 (FIG. 8). The processing contents of steps 411, 414, 416, and 417 are the same as those of steps 311, 314, 316, and 317.
[0061]
In step 412, the R signal of the target pixel is calculated according to equation (10), and an interpolated R signal r is obtained. The G signal and B signal of the target pixel are calculated according to the equations (11) and (12), and the corrected G signal g ′ and the corrected B signal b ′ are obtained.
r = YG + R (x ′, y ′) (10)
g ′ = YG + G (x ′, y ′) (11)
b '= YG + B (x', y ') (12)
[0062]
The second interpolation / correction processing routine is executed for all pixels of the image belonging to the fourth pattern.
[0063]
As described above, in the second interpolation / correction processing routine, as in the first interpolation / correction processing routine, the correction value YG is added to the R, G, and B signals of similar pixels to obtain the R signal of the target pixel. At the same time, the G signal and the B signal that have been obtained so far are corrected.
[0064]
As described above, in the present embodiment, in particular in the first interpolation / correction processing routine (FIGS. 7 and 8) and the second interpolation / correction processing routine (FIG. 9), the target pixel is positioned on the left and right and top and bottom. A pixel having a luminance value closest to the pixel to be extracted is extracted as a similar pixel, and interpolation processing is performed. In this interpolation process, the closer the luminance values of the two pixels are, the stronger the color correlation is. Therefore, by using the color difference signal of the target pixel as the color difference signal of the similar pixel, a pixel signal closer to the actual color can be obtained. Therefore, according to the present embodiment, it is possible to reduce color bleeding in the reproduced image.
[0065]
Furthermore, in the present embodiment, the G signal obtained by the G interpolation processing routine and the R signal and B signal generated by the CCD 12 are corrected according to the equations (7), (8), (11) and (12). The color of the target pixel is close to the color of the surrounding pixels, and the color blur is further reduced in the entire image.
[0066]
A second embodiment will be described with reference to FIGS. 10 and 11. FIG. 10 shows the first half of the first interpolation / correction processing routine, and FIG. 11 shows the first half of the second interpolation / correction processing routine. Other configurations are the same as those of the first embodiment, and thus are omitted.
[0067]
In step 501, it is determined whether or not the inequality (13) is established as a comparison of the luminance values of the pixel adjacent to the left and right sides of the target pixel and the target pixel.
| 0.587 × (G (x-1, y) -G (x, y)) + 0.299 × (R (x-1, y) -R (x, y)) | <
| 0.587 × (G (x + 1, y) −G (x, y)) + 0.299 × (R (x + 1, y) −R (x, y)) | (13)
When the inequality (13) is true, that is, when the luminance value of the left pixel is closer to the luminance value of the target pixel, the parameter p is set to −1 in step 502. On the other hand, when the inequality (13) is false, that is, when the luminance value of the right pixel is closer to or equal to the luminance value of the target pixel, the parameter p is set to 1 in step 503.
[0068]
In step 504, it is determined whether or not the inequality (14) is established as a comparison between the luminance values of the pixel adjacent to the upper side and the lower side of the target pixel and the target pixel.
| 0.587 × (G (x, y-1) -G (x, y)) + 0.299 × (R (x, y-1) -R (x, y)) | <
| 0.587 × (G (x, y + 1) −G (x, y)) + 0.299 × (R (x, y + 1) −R (x, y)) | (14)
When the inequality (14) is true, that is, when the luminance value of the upper pixel is closer to the luminance value of the target pixel, in step 505, the parameter q is set to -1. On the other hand, when the inequality (14) is false, that is, when the luminance value of the lower pixel is closer to or equal to the luminance value of the target pixel, the parameter q is set to 1 in step 506.
[0069]
In step 507, the luminance value is compared according to the inequality (15) with respect to the pixel adjacent to one of the left and right of the target pixel, the pixel adjacent to one of the top and bottom, and the target pixel.
| 0.587 × (G (x + p, y) -G (x, y)) + 0.299 × (R (x + p, y) -R (x, y)) | <
| 0.587 × (G (x, y + q) -G (x, y)) + 0.299 × (R (x, y + q) -R (x, y)) | (15)
When the parameters p and q are both −1 and the inequality (15) is true, the luminance value of the left pixel is closest to the luminance value of the target pixel. In this case, in step 508, the parameter s is p, that is, −1. Determined. On the other hand, when the inequality (15) is false, that is, when the luminance value of the upper pixel is closer to or equal to the luminance value of the target pixel, the parameter s is set to 2 × q, that is, −2 in step 509.
[0070]
When the parameters p and q are both 1, the parameters p and q are −1 and 1 respectively, and the parameters p and q are respectively 1 and −1, the same steps 507, 508 or 509 are executed. Is done. In this way, in the first interpolation / correction processing routine, the G signal and R signal of the target pixel and the G signal and R signal of adjacent pixels included in the images belonging to the second and third patterns are used. The luminance value is examined. The parameter s is -1 if the luminance value of the pixel adjacent on the left side is closest, the parameter s is 1 if the luminance value of the pixel adjacent on the right side is closest, and the luminance of the pixel adjacent on the upper side. If the value is closest, the parameter s is set to -2, and if the luminance value of the adjacent pixel on the lower side is closest, the parameter s is set to 2.
[0071]
After the similar pixel having the luminance value closest to the luminance value of the target pixel is extracted in this way, the luminance is obtained using the R, G, and B signals of the similar pixel by the same process as the flowchart shown in FIG. The value Y and the correction value YG are obtained, the B signal of the target pixel is obtained by interpolation, and the G signal and the R signal are corrected. The correction value YG is obtained according to the equation (16) using the G signal and R signal of the target pixel and similar pixels. Other formulas are the same as those in the first embodiment.
YG = Y × (0.587 × (G (x, y) −G (x ′, y ′)) + 0.299 × (R (x, y) −R (x ′, y ′)))
/(0.587×G(x',y')+0.299×R(x',y ')) (16)
However, G (x ′, y ′) indicates the G signal of the similar pixel, and R (x ′, y ′) indicates the R signal of the similar pixel.
[0072]
In the second interpolation / correction processing routine (FIG. 11), the R signal of the fourth pattern image is obtained by interpolation, and the G signal and the B signal are corrected. In step 601, it is determined whether or not the inequality (17) is established as a comparison of the luminance values of the pixel adjacent to the left and right sides of the target pixel and the target pixel.
| 0.587 × (G (x-1, y) -G (x, y)) + 0.144 × (B (x-1, y) -B (x, y)) | <
| 0.587 × (G (x + 1, y) -G (x, y)) + 0.144 × (B (x + 1, y) -B (x, y)) | (17)
When the inequality (17) is true, the parameter p is set to −1 at step 602, and when the inequality (17) is false, the parameter p is set to 1 at step 603.
[0073]
In step 604, it is determined whether or not the inequality (18) is established as a comparison of the luminance values of the target pixel and the pixels adjacent to the upper and lower sides of the target pixel.
| 0.587 × (G (x, y-1) -G (x, y)) + 0.144 × (B (x, y-1) -B (x, y)) | <
| 0.587 × (G (x, y + 1) -G (x, y)) + 0.144 × (B (x, y + 1) -B (x, y)) | (18)
When the inequality (18) is true, the parameter q is set to −1 at step 605, and when the inequality (18) is false, the parameter q is set to 1 at step 606.
[0074]
In step 607, the luminance value is compared according to the inequality (19) with respect to the pixel adjacent to any of the left and right of the target pixel, the pixel adjacent to either of the top and bottom, and the target pixel.
| 0.587 × (G (x + p, y) -G (x, y)) + 0.144 × (B (x + p, y) -B (x, y)) | <
| 0.587 × (G (x, y + q) -G (x, y)) + 0.144 × (B (x, y + q) -B (x, y)) | (19)
When the parameters p and q are both −1 and the inequality (19) is true, the luminance value of the left pixel is closest to the luminance value of the target pixel. In this case, in step 608, the parameter s is p, that is, −1. Determined. On the other hand, when the inequality (19) is false, that is, when the luminance value of the upper pixel is closer to or equal to the luminance value of the target pixel, the parameter s is set to 2 × q, that is, −2 in step 609.
[0075]
When the parameters p and q are both 1, the parameters p and q are −1 and 1, respectively, and the parameters p and q are respectively 1 and −1, steps 607, 608 or 609 are executed in the same manner. Is done. In this way, in the second interpolation / correction processing routine, the G signal and B signal of the target pixel and the G signal and B signal of the adjacent pixels included in the images belonging to the second and third patterns are used. The luminance value is examined. The parameter s is -1 if the luminance value of the pixel adjacent on the left side is closest, the parameter s is 1 if the luminance value of the pixel adjacent on the right side is closest, and the luminance of the pixel adjacent on the upper side. If the value is closest, the parameter s is set to -2, and if the luminance value of the adjacent pixel on the lower side is closest, the parameter s is set to 2.
[0076]
After the similar pixel having the luminance value closest to the luminance value of the target pixel is extracted in this way, the luminance value is obtained using the R, G, B signals of the similar pixel by the same processing as the flowchart shown in FIG. Y and the correction value YG are obtained, the R signal of the target pixel is obtained by interpolation, and the G signal and the B signal are corrected. The correction value YG is obtained according to the equation (20) using the G signal and B signal of the target pixel and similar pixels.
YG = Y × (0.587 × (G (x, y) −G (x ′, y ′)) + 0.114 × (B (x, y) −B (x ′, y ′)))
/(0.587×G(x',y')+0.114×B(x',y ')) (20)
However, G (x ′, y ′) indicates the G signal of the similar pixel, and B (x ′, y ′) indicates the B signal of the similar pixel.
[0077]
As described above, in this embodiment, in the first interpolation / correction processing routine (FIG. 10) and the second interpolation / correction processing routine (FIG. 11), the G signal and the R signal or the B signal are used as luminance values. Therefore, the luminance signal can be examined more accurately. Therefore, the color blur in the reproduced image can be reduced as compared with the first embodiment.
[0078]
【The invention's effect】
As described above, according to the present invention, it is possible to reduce color bleeding that occurs in a reproduced image by interpolation processing.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a schematic configuration of a digital camera including an image interpolation device according to a first embodiment of the present invention.
FIG. 2 is a diagram illustrating an array of color filter elements in a color filter and pixel signals of an image belonging to each pattern extracted by a pattern setting unit.
FIG. 3 is a diagram illustrating coordinates and colors of pixels constituting one image.
FIG. 4 is a flowchart of a G interpolation processing routine.
FIG. 5 is a flowchart of an R / B interpolation processing routine.
FIG. 6 is a diagram illustrating pixel signals obtained by executing a G interpolation process routine and an R / B interpolation process routine.
FIG. 7 is a first half of a flowchart of a first interpolation / correction processing routine.
FIG. 8 is the latter half of the flowchart of the first interpolation / correction processing routine.
FIG. 9 is the latter half of the flowchart of the second interpolation / correction processing routine.
FIG. 10 is a first half of a flowchart of a first interpolation / correction processing routine in the second embodiment.
FIG. 11 is a first half of a flowchart of a second interpolation / correction processing routine in the second embodiment.
[Explanation of symbols]
12 Image sensor
13 Color filter

Claims (16)

A first column in which red (R) and green (G) color filter elements are arranged alternately in the horizontal direction, and a color filter of G and blue (B) in the horizontal direction adjacent to the upper and lower sides of the first column A color filter composed of a second row of alternating elements;
An image sensor that generates R, G, and B signals that are pixel signals corresponding to the color filter elements;
From the R, G, B signals generated by the imaging device, a first pattern in which the R signal is located at the upper left in the 2 × 2 pixel matrix, a second pattern in which the G signal is located in the upper right, and a G signal in the lower left Pattern setting means for extracting images belonging to the third pattern in which B is located and the fourth pattern in which the B signal is located in the lower right,
G interpolation processing means for obtaining a G signal using the G signal of an adjacent pixel for each pixel of the image belonging to the first and fourth patterns;
R / B interpolation processing means for obtaining R and B signals using R and B signals of adjacent pixels for each pixel of the images belonging to the second and third patterns;
For each pixel of the image belonging to the first pattern, a similar pixel having the closest luminance value is extracted from adjacent pixels, and a first correction value corresponding to a change rate of the luminance value between the similar pixels and A first interpolation that obtains a B signal based on the B, G, and R signals of the similar pixels and that corrects the G signal obtained by the G interpolation processing unit and the R signal generated by the imaging device. / Correction processing means,
For each pixel of the image belonging to the fourth pattern, a similar pixel having the closest luminance value is extracted from adjacent pixels, and a second correction value corresponding to the change rate of the luminance value between the similar pixels and Based on the R, G, and B signals of the similar pixels, a second interpolation that obtains the R signal and corrects the G signal obtained by the G interpolation processing means and the B signal generated by the image sensor. / Correction processing means ,
The first interpolation / correction processing means adds the first correction value to each of the B, G, R signals of the similar pixels for each pixel of the image belonging to the first pattern, thereby obtaining the B signal. An image interpolating apparatus characterized in that the G signal and the R signal are corrected and obtained.   The image interpolation apparatus according to claim 1, wherein in the G interpolation processing unit, the adjacent pixels are included in an image belonging to the second and third patterns.   2. The image interpolation apparatus according to claim 1, wherein in the R / B interpolation processing means, the adjacent pixels are included in an image belonging to the first and fourth patterns.   2. The first interpolation / correction processing unit and the second interpolation / correction processing unit, wherein the adjacent pixels are included in an image belonging to the second and third patterns. Image interpolation device.   2. The similar pixel according to claim 1, wherein the first interpolation / correction processing unit and the second interpolation / correction processing unit extract the similar pixel by using a value of a G signal in the adjacent pixel. Image interpolation device.   The first correction value is obtained for each pixel of an image belonging to the first pattern by multiplying a change rate of a G signal with the similar pixel by a luminance value of the similar pixel. Item 2. The image interpolation device according to Item 1. The image interpolation apparatus according to claim 6 , wherein the first interpolation / correction processing unit obtains the B signal according to the following formula and corrects the G signal and the R signal.
Y = 0.299 × R (x ′, y ′) + 0.587 × G (x ′, y ′) + 0.114 × B (x ′, y ′)
YG = Y × (G (x, y) −G (x ′, y ′ )) / G (x ′, y ′)
r ′ = YG + R (x ′, y ′)
g ′ = YG + G (x ′, y ′)
b = YG + B (x ′, y ′)
Where Y is the luminance value of the similar pixel, YG is the first correction value, and R (x ′, y ′), G (x ′, y ′), and B (x ′, y ′) are the similar pixels. R, G, B signals, G (x, y) are G signals of the pixels of the image belonging to the first pattern, and b is a pixel of the first pattern obtained by the first interpolation / correction processing means. B signal, r ′ is a modified R signal, and g ′ is a modified G signal.   The second interpolation / correction processing means adds the second correction value to the R, G, and B signals of the similar pixels for each pixel of the image belonging to the fourth pattern, thereby obtaining the R signal. 2. The image interpolating apparatus according to claim 1, wherein the G signal and the B signal are corrected and obtained.   The second correction value is obtained for each pixel of the image belonging to the fourth pattern by multiplying a change rate of a G signal with the similar pixel by a luminance value of the similar pixel. Item 2. The image interpolation device according to Item 1. 10. The image interpolation apparatus according to claim 9 , wherein the second interpolation / correction processing unit obtains the R signal according to the following formula and corrects the G signal and the B signal.
Y = 0.299 × R (x ′, y ′) + 0.587 × G (x ′, y ′) + 0.114 × B (x ′, y ′)
YG = Y × (G (x, y) −G (x ′, y ′)) / G (x ′, y ′)
r = YG + R (x ', y')
g ′ = YG + G (x ′, y ′)
b '= YG + B (x', y ')
Where Y is the luminance value of the similar pixel, YG is the second correction value, and R (x ′, y ′), G (x ′, y ′), and B (x ′, y ′) are the similar pixels. R, G, B signals, G (x, y) are G signals of pixels of an image belonging to the fourth pattern, and r is an image of the image belonging to the fourth pattern obtained by the second interpolation / correction processing means. The R signal of the pixel, g ′ is a modified G signal, and b ′ is a modified B signal.   The image interpolation apparatus according to claim 1, wherein the first interpolation / correction processing unit extracts the similar pixels by using values of a G signal and an R signal in the adjacent pixels.   The image interpolation apparatus according to claim 1, wherein the second interpolation / correction processing unit extracts the similar pixels by using values of a G signal and a B signal in the adjacent pixels.   For each pixel of the image belonging to the first pattern, the first correction value is obtained based on a G signal and an R signal in the pixel and the similar pixel, and a G signal of the similar pixel and The image interpolation apparatus according to claim 1, wherein the luminance value is obtained by multiplying a ratio with a second reference value obtained based on the R signal. 14. The image interpolating apparatus according to claim 13 , wherein the first interpolation / correction processing means obtains the B signal according to the following equation and corrects the G signal and the R signal.
Y = 0.299 × R (x ′, y ′) + 0.587 × G (x ′, y ′) + 0.114 × B (x ′, y ′)
YG = Y × (0.587 × (G (x, y) −G (x ′, y ′) + 0.299 × (R (x, y) −R (x ′, y ′)) / (0.587 × G (x ', y') + 0.299 × R (x ', y'))
r '= YG + R (x', y ' )
g ′ = YG + G (x ′, y ′)
b = YG + B (x ′, y ′)
Where Y is the luminance value of the similar pixel, YG is the first correction value, and R (x ′, y ′), G (x ′, y ′), and B (x ′, y ′) are the similar pixels. R, G, B signals, G (x, y) are G signals of pixels of an image belonging to the first pattern, and b is an image of the image belonging to the first pattern obtained by the first interpolation / correction processing means. The B signal of the pixel, r ′ is the modified R signal, and g ′ is the modified G signal.   The second interpolation / correction processing means, for each pixel of the image belonging to the fourth pattern, the first reference value obtained based on the G signal and the B signal in the pixel and the similar pixel, and the similar pixel The image interpolation apparatus according to claim 1, wherein the luminance value is obtained by multiplying a ratio with a second reference value obtained based on the G signal and the B signal. 16. The image interpolation apparatus according to claim 15 , wherein the second interpolation / correction processing unit obtains the R signal according to the following formula and corrects the G signal and the B signal.
Y = 0.299 × R (x ′, y ′) + 0.587 × G (x ′, y ′) + 0.114 × B (x ′, y ′)
YG = Y × (0.587 × (G (x, y) −G (x ′, y ′)) + 0.114 × (B (x, y) −B (x ′, y ′)) / (0.587 × G ( x ', y') + 0.114 × B (x ', y'))
r = YG + R (x ', y')
g ′ = YG + G (x ′, y ′)
b '= YG + B (x', y ')
Where Y is the luminance value of the similar pixel, YG is the second correction value, and R (x ′, y ′), G (x ′, y ′), and B (x ′, y ′) are the similar pixels. R, G, B signals, G (x, y) are G signals of pixels of an image belonging to the fourth pattern, and r is an image of the image belonging to the fourth pattern obtained by the second interpolation / correction processing means. The R signal of the pixel, g ′ is a modified G signal, and b ′ is a modified B signal.

Images