KR101003681B1 - Pixel interpolation circuit, pixel interpolation method, and recording medium - Google Patents

Abstract

Using the pixel values of the R pixels, the G pixels, and the B pixels in the Bayer data, the horizontal and vertical correlations in the peripheral region of the interpolation target pixel are calculated. The pixel value of the G pixel to be interpolated to the interpolation target pixel is determined based on the pixel value of the pixel adjacent to the interpolation target pixel in either the horizontal direction or the vertical direction showing a stronger correlation among the calculated correlations. . A G pixel composed of the determined pixel values is interpolated to the interpolation target pixel in the Bayer data.

Description

Pixel interpolation circuit, pixel interpolation method and recording medium {PIXEL INTERPOLATION CIRCUIT, PIXEL INTERPOLATION METHOD, AND RECORDING MEDIUM}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pixel interpolation circuit, a pixel interpolation method, and a recording medium for use in an imaging device or the like employing a single-plate color imaging method.

In a digital camera, a single plate type imaging method using an imaging device comprising a Bayer array pixel array has become a mainstream. When the image signal is obtained from the Bayer data which is the image data of the subject imaged by the image pickup device of the Bayer array, it is necessary to interpolate the color information (pixel value) other than the color filter arranged in each pixel in the Bayer data. In particular, since the information of the G pixel affects the luminance information, the information of the G pixel affects the resolution of the image signal and the sharpness of the outline.

Here, in the Bayer data, Japanese Patent Laid-Open No. 2000-278703 or Japanese Laid-Open Patent Publication No. 2006-135564, which interpolates a G pixel to an interpolation target pixel (an R pixel or a B pixel in which G pixel information does not exist). There is a technique described in the call. In these prior arts, the G pixel is interpolated to the interpolation target pixel in consideration of the correlation of pixels around the interpolation target pixel in the Bayer data.

However, all of these conventional techniques refer only to the G pixels around the interpolation target pixel when interpolating the G pixel to the interpolation target pixel in the Bayer data. Therefore, the information of the pixel referred to when interpolating the G pixel to the interpolation target pixel is small. As a result, there was a problem that the correlation between the pixels in the periphery of the interpolation target pixel could not be appropriately determined and that the G pixel appropriate for the interpolation target pixel could not be interpolated.

Therefore, an object of the present invention is to appropriately interpolate G pixel information with Bayer data.

An object of the present invention is a pixel value of a G pixel interpolated to the interpolation target pixel using only the pixel value of the G pixel in the peripheral region of the interpolation target pixel, which is a pixel in which Bayer data does not have a pixel value of the G pixel. A pixel interpolation unit for generating an interpolation image signal by interpolating the determined pixel value of the G pixel to the interpolation target pixel, and the pixel of interest in the R pixel, the G pixel, and the B pixel in the Bayer data. By performing the calculation between the pixel value of the pixel and the pixel value of the peripheral pixel of the other pixel component as the pixel value of the peripheral pixel of the peripheral region of the pixel of interest, the peripheral region of the interpolation target pixel. A stronger image from among the correlations calculated by the horizontal and vertical correlations and the correlations calculated by the correlation calculator. Pixel value determination for determining a second G interpolation pixel value to be interpolated to the interpolation target pixel based on the pixel value of the pixel adjacent to the interpolation target pixel in either the horizontal direction or the vertical direction indicating inertia. A G image signal acquisition unit for acquiring a G image signal consisting of only a G pixel consisting of a second G interpolation pixel value determined by the pixel value determination unit, and the G image signal acquired by the G image signal acquisition unit. A high pass filter for removing low frequency components of an image coder and a second G interpolation pixel value in a G image signal from which low frequency components have been removed by the high pass filter are added to the interpolated image signal generated by the pixel interpolation unit. An adder which adds; the pixel value determiner includes any one of a horizontal direction and a vertical direction indicating a stronger correlation among the correlations calculated by the correlation calculator. A pixel value obtained by adding an average value of pixel values of pixels adjacent to each of the interpolation target pixels in one direction and an average value of pixels adjacent to the interpolation target pixels in a horizontal direction and a vertical direction according to a predetermined ratio, A pixel interpolation circuit may be determined as the second G interpolation pixel value.

A first reference image generation unit for performing a calculation using pixel values of pixels adjacent in the horizontal direction in the peripheral region of the interpolation target pixel and generating a first reference image, and a vertical direction in the peripheral region of the interpolation target pixel And a second reference image generation unit for performing an operation using pixel values of adjacent pixels and generating a second reference image, wherein the correlation calculator comprises a first reference generated by the first reference image generation unit. The correlation in the horizontal direction is calculated on the basis of the image, and the correlation in the vertical direction is calculated on the basis of the second reference image generated by the second reference image generating unit.

The first reference image generating unit includes a pixel value of the first pixel, an average value of respective pixel values of the second pixel and the third pixel adjacent to the first pixel in the horizontal direction in the peripheral region of the interpolation target pixel. A horizontal difference calculating unit for acquiring a difference value of; and generating the first reference image based on each difference value obtained by the horizontal difference calculating unit, wherein the second reference image generating unit is arranged in the peripheral region of the interpolation target pixel. And a vertical difference calculating section for obtaining a difference value between a pixel value of the first pixel and an average value of respective pixel values of the fourth pixel and the fifth pixel adjacent to the first pixel in the vertical direction. The second reference image may be generated based on the difference values obtained by the difference calculating unit.

The first reference image generating section includes a first horizontal difference calculating section that obtains a difference value between the pixel value of the R pixel and the average value of the pixel value of the G pixel adjacent to the R pixel in the horizontal direction, and the pixel value of the B pixel. And a second horizontal difference calculating unit for obtaining a difference value between the average value of pixel values of the G pixels adjacent to the B pixel in the horizontal direction, and the average value of pixel values of the R pixels adjacent to the G pixels in the horizontal direction. A third horizontal difference calculating section for acquiring the difference value of the pixel value of the G pixel, and a fourth value for acquiring the difference value of the pixel value of the B pixel adjacent to the G pixel and the pixel value of the G pixel in the horizontal direction A horizontal difference calculating unit, and generating the first reference image based on each difference value obtained by the first to fourth horizontal difference calculating units, wherein the second reference image generating unit is perpendicular to the pixel value of the R pixel; In direction A first vertical difference calculating section for obtaining a difference value between an average value of pixel values of the G pixels adjacent to the R pixel, a pixel value of the B pixel, and a pixel value of the G pixel adjacent to the B pixel in the vertical direction. A second vertical difference calculating section for obtaining a difference value from an average value, a third vertical difference calculating section for obtaining a difference value between a pixel value of an R pixel adjacent to a G pixel and a pixel value of the corresponding G pixel in a vertical direction; And a fourth vertical difference calculating section for obtaining a difference value between the pixel value of the B pixel adjacent to the G pixel and the pixel value of the G pixel in the vertical direction, and obtained by the first to fourth vertical difference calculating sections. The second reference image may be generated based on each difference value.

The correlation calculating section includes a first calculating section that calculates a first correlation value that is a sum of a value obtained by multiplying an absolute value of each difference value obtained by the first to fourth horizontal difference calculating sections by a predetermined coefficient, and the first to fourth sections. A first calculation unit configured to calculate a second correlation value which is a sum of a value obtained by multiplying each of an absolute value of each difference value obtained by the fourth vertical difference calculation unit by a predetermined coefficient; The correlation in the horizontal direction and the correlation in the vertical direction may be calculated based on the correlation value and the second correlation value calculated by the second calculation unit.

The pixel value determination unit is configured to calculate an average value of pixel values of a pixel adjacent to the interpolation target pixel in any one of a horizontal direction and a vertical direction showing stronger correlation among the correlations calculated by the correlation calculation unit. It may be determined as a second G interpolation pixel value.

The predetermined ratio may be a ratio based on a first correlation value calculated by the first calculator and a second correlation value calculated by the second calculator.

As another category, an object of the present invention is to interpolate to the interpolation target pixel using only the pixel value of the G pixel in the peripheral region of the interpolation target pixel, which is a pixel in which the pixel value of the G pixel does not exist in the Bayer data. Determining the pixel value of the G pixel, and interpolating the determined pixel value of the G pixel to the interpolation target pixel to generate an interpolated image signal, and the pixel of interest in the R pixel, the G pixel, and the B pixel in the Bayer data. The calculation is performed between the pixel value of the pixel to be the pixel value of the pixel of interest and the pixel value of the peripheral pixel of the other color component as the pixel value of the peripheral pixel of the peripheral region of the pixel of interest. A step of calculating the correlation between the horizontal direction and the vertical direction in the peripheral region, and the horizontal direction showing a stronger correlation among the calculated correlations And an average value of pixel values of the pixel adjacent to the interpolation target pixel in one of the vertical directions and an average value of pixels adjacent to the interpolation target pixel in the horizontal and vertical directions, according to a predetermined ratio. Determining a second G interpolation pixel value for interpolating a pixel value to the interpolation target pixel, acquiring a G image signal composed of only a G pixel consisting of the determined second G interpolation pixel value, and And removing the low frequency component of the G image coral, and adding the second G interpolated pixel value in the G image signal from which the low frequency component has been removed to the generated interpolated image signal. It can be achieved by an interpolation method.

Still another object of the present invention is to interpolate to the interpolation target pixel using only the pixel value of the G pixel in the peripheral region of the interpolation target pixel, which is a pixel in which the pixel value of the G pixel does not exist in the Bayer data. The pixel value of the G pixel to be determined and the interpolation image signal are generated by interpolating the determined pixel value of the G pixel to the interpolation target pixel, and the attention of the R pixel, the G pixel, and the B pixel in the Bayer data. The interpolation target pixel is performed by performing an operation between the pixel value of the pixel serving as the pixel and the pixel value of the peripheral pixel of the other pixel component as the pixel value of the peripheral pixel of the peripheral region of the pixel of interest. Processing for calculating the horizontal and vertical correlations in the peripheral region of the horizontal direction, and the horizontal direction showing a stronger correlation among the calculated correlations. A pixel obtained by adding an average value of pixel values of a pixel adjacent to the interpolation target pixel in one of the direct directions and an average value of pixels adjacent to the interpolation target pixel in a horizontal direction and a vertical direction according to a predetermined ratio. A process for determining a second G interpolation pixel value for interpolating a value to the interpolation target pixel, a process for acquiring a G image signal composed only of a G pixel consisting of the determined second G interpolation pixel value, and the obtained A computer readout that stores a program that executes a process of removing low frequency components of a G image coral and a process of adding a second G interpolation pixel value in a G image signal from which low frequency components have been removed to the generated interpolated image signal. It can be achieved as a possible recording medium.

According to the present invention, the information of the G pixel can be appropriately interpolated with respect to the Bayer data.

1 is a view showing the overall configuration of the digital camera 100.
2 is a diagram illustrating a configuration of the image processing unit 20.
3 is a diagram illustrating a configuration of the pixel interpolation unit 30.
4 is a configuration diagram of the G pixel extraction circuit 40.
5 is a diagram illustrating an example of a Bayer arrangement.
6 is a diagram illustrating an example of a horizontal reference image.
7 is a diagram illustrating an example of a vertical reference image.
8 is a configuration diagram of the G pixel extraction circuit 40 in the modification.

Embodiment of this invention is described. First, with reference to FIG. 1, the whole structure of the digital camera 100 which concerns on this embodiment is demonstrated.

The imaging lens 62 includes a focus lens and a zoom lens, and a lens driving block 63 is connected. The lens driving block 63 includes a focus motor and a zoom motor for driving a focus lens and a zoom lens (not shown) in the optical axis direction parallel to the imaging plane, respectively, and the focus motor and the zoom motor in accordance with control signals from the CPU 70, respectively. It consists of a focus driver and a zoom motor driver for driving.

The diaphragm combined shutter 64 includes a drive circuit (not shown), which operates the diaphragm combined shutter in accordance with a control signal sent from the CPU 70. This diaphragm combined shutter 64 functions as an aperture and a shutter.

The CCD 50, which is an imaging device, is provided with a Bayer array color filter on the light receiving surface. The CCD 50 converts the light of the subject projected through the imaging lens 62 and the diaphragm shutter 64 into an electrical signal, and outputs the converted electrical signal to the image processing unit 20 as the imaging signal. In addition, the CCD 50 is driven in accordance with a timing signal of a predetermined frequency generated by the TG 66 in accordance with a control signal from the CPU 70.

The memory card 80 is detachably connected to the card I / F 76.

The memory 69 stores programs and data necessary for controlling each part of the digital camera 100 by the CPU 70.

The DRAM 68 is used as a buffer memory for temporarily storing image data sent to the CPU 70 after being picked up by the CCD 50 and also used as a work memory of the CPU 70.

The image display section 71 is composed of a liquid crystal display and its driving circuit. When the digital camera 100 is in the shooting standby state, the subject captured by the CCD 50 is displayed on the liquid crystal display as a through image. The image display unit 71 displays the image read out from the memory card 80 on the liquid crystal display when the digital camera 100 reproduces the image.

The key input unit 72 includes a plurality of operation keys such as a shutter button, a mode key, a setting key, a cross key, a shooting mode selection key and the like, and outputs an operation signal corresponding to a user's key operation to the CPU 70.

The CPU 70 controls the entire digital camera 100. The CPU 70 executes various operations in cooperation with a program read from the memory 69.

Next, with reference to FIG. 2, the structure of the image processing part 20 is demonstrated. The image processing unit 20 includes analog processing units (CDS, AGC, A / D, 21), a black level adjusting unit 22, a white balance adjusting unit 23, a gamma correction unit 24, and a pixel interpolation unit 30. ), A YUV conversion processing unit 27, and a JPEG conversion unit 28.

In the image processing unit 20, the correlated double sampling circuit CDS reduces noise included in the image pickup signal, which is an analog signal output from the CCD 50. The automatic gain control circuit AGC adjusts the gain of the image pickup signal after noise reduction. The A / D converter A / D converts the captured image signal after gain adjustment into a digital signal, that is, R (pixel data representing red information), G (pixel data representing green information), and B (blue) according to the Bayer arrangement of the color filter. To Bayer data, which is pixel data of pixel data representing information.

The black level adjustment section 22 clamps the Bayer data to a predetermined black level. The white balance adjustment unit 23 performs gain adjustment for each of R, G, and B for the Bayer data after the clamp, and performs the white balance adjustment. The gamma correction unit 24 performs correction of the gamma characteristic (gradation characteristic) on the Bayer data after the white balance adjustment. The pixel interpolation unit 30 interpolates the pixel value information with respect to the Bayer data after the correction of the gamma characteristic, and generates an image signal. Details of the pixel interpolation unit 30 will be described later.

The YUV conversion processing unit 27 generates an image signal consisting of the luminance signal Y and the color difference signals U and V from the interpolated image signal of the pixel information. The JPEG converter 28 compresses and encodes the image signal in JPEG format. The compressed coded image signal is recorded in the memory card 80.

Next, the pixel interpolation unit 30 will be described in detail with reference to FIG. 3. The pixel interpolation unit 30 is composed of two circuit portions of the first circuit portion 30a and the second circuit portion 30b, a low pass filter 26, and an adder 36. The first circuit section 30a is composed of a wideband LPF 31, a G interpolation circuit 32, an R interpolation circuit 33, and a B interpolation circuit 34. The second circuit section 30b is composed of a G pixel extraction circuit 40 and an HPF (High Pass Filter) 37.

In the first circuit unit 30a, signal processing for interpolating the missing pixel information is performed on the Bayer data sent from the gamma correction unit 24, and the image signal generated in the pixel interpolation unit 30 is executed. A color image signal as a reference is generated.

The signal processing in the first circuit section 30a is the same as the signal processing disclosed in Japanese Patent Laid-Open No. 2006-135564. In other words, about the signal processing in the first circuit section 30a, the pixel of the G pixel is located in the R pixel or the B pixel (hereinafter referred to as the "interpolation target pixel") in which the G pixel information does not exist in the Bayer data. When interpolating a value, only the pixel value of the G pixel around the interpolation target pixel is referred to. Therefore, the information of the pixel value referred to when interpolating the pixel value of the G pixel to the interpolation target pixel is reduced. As a result, the correlation between the pixel values around the interpolation target pixels cannot be appropriately determined, and the appropriate pixel values of the G pixels may not be interpolated in the interpolation target pixels.

This improper G pixel becomes an isolated point, and noise may occur in an image signal. Therefore, the high frequency component (noise component, etc.) of the image signal is removed by passing the image signal through the LPF 26, which is a low pass filter, in order to remove this noise component. However, if the image signal is passed through the LPF 26, the noise component can be removed, but the sharpness of the outline in the image signal is also lowered, and the reproduced image based on the image signal is blurred.

Therefore, appropriate information of the G pixel is separately extracted from the Bayer data sent from the gamma correction unit 24 by the second circuit unit 30b. In the adder 38, the extracted G pixel information is added to the image signal passed through the LPF 26. Since the information of the G pixel affects the sharpness of the outline in the reproduced image, the outline of the reproduced image based on the Bayer data is emphasized, and the reproduced image is not blurred.

Next, the signal processing in the second circuit section 30b will be described in detail. The G pixel extraction circuit 40 in the second circuit section 30b is configured to interpolate the information of the G pixels to be interpolated with respect to the interpolation target pixels (R pixels and B pixels in which G pixel information does not exist) in the Bayer data. Circuit for extracting pixel values). As shown in Fig. 4, the G pixel extraction circuit 40 includes a horizontal reference image generating unit 41, a vertical reference image generating unit 42, a horizontal correlation calculating unit 43, and a vertical direction. The correlation degree calculation section 44, the pixel value determination section 45, and the pixel value extraction section 46 are constituted.

The horizontal reference image generation unit 41 performs a calculation on the Bayer data using pixel values of pixels adjacent in the horizontal direction, and generates a horizontal reference image. The horizontal reference image is information for obtaining a direction of strong correlation between pixels in the periphery of each interpolation target pixel of the Bayer data.

Specifically, the horizontal reference image generating unit 41 performs horizontal processing by sequentially executing any one of the following (A1) to (A4) on each pixel in the periphery of each interpolation target pixel of the Bayer data. Create a direction reference image.

(A1) Process of subtracting the average value of pixel values of the R pixel adjacent to the R pixel in the horizontal direction from the pixel value of the R pixel adjacent to the G pixel in the horizontal direction

(A2) Processing for subtracting the pixel value of the G pixel from the average value of the pixel values of the R pixel adjacent to the G pixel in the horizontal direction with respect to the G pixel adjacent to the R pixel in the horizontal direction

(A3) A process of subtracting the pixel value of the G pixel from the average value of the pixel values of the B pixel adjacent to the G pixel in the horizontal direction with respect to the G pixel adjacent to the B pixel in the horizontal direction.

(A4) Process of subtracting the average value of pixel values of the B pixels adjacent to the B pixels in the horizontal direction from the pixel values of the B pixels adjacent to the G pixels in the horizontal direction

5 shows an example of pixel values of respective pixels in Bayer data. In Fig. 5, the positions denoted by B, such as B0, B2, ..., B48, are positions where B pixels are arranged in the Bayer data. The positions indicated by G, such as G1, G3, ..., G47, are positions where G pixels are arranged in the Bayer data. The positions denoted by R, such as R8, R10, ..., R40, are positions where the R pixels are arranged in the Bayer data.

In Fig. 5, B0, B2, ... B48 indicate pixel values of each B pixel. G1, G3, ..., G47 represent pixel values of each G pixel. R8, R10, ... R40 represent pixel values of each R pixel.

The horizontal reference image is generated in the peripheral area of each interpolation target pixel in the Bayer data. 6 shows a horizontal reference image 40H as an example of the horizontal reference image. The horizontal reference image 40H is a horizontal reference image for the pixel (interpolation target pixel) at the position "R24" shown in FIG.

For example, B0, G1, B2, ... B48 which are 7x7 pixels in the peripheral region of the pixel at the position of "R24" shown in FIG. 5 regarding the generation of the horizontal reference image 40H. It focuses on the area | region which consists of each pixel in the position of. In addition, the peripheral region to be implanted may not be a region consisting of 7x7 pixels.

Next, each component in the horizontal reference image 40H will be described as an example of each component in the horizontal reference image.

The component "h8" in the horizontal reference image 40H becomes the following value by the process of (A1) using the pixel values of G7, R8, and G9 shown in FIG.

h8 = R8- (G7 + G9) / 2

The component "h9" in the horizontal reference image 40H becomes the following value by the process of (A2) using the pixel values of R8, G9, and R10 shown in FIG.

h9 = (R8 + R10) / 2-G9

The component "h15" in the horizontal reference image 40H becomes the following value by the process of (A3) using the pixel values of B14, G15, and B16 shown in FIG.

h15 = (B14 + B16) / 2-G15

The component "h16" in the horizontal reference image 40H becomes the following value by the process of (A4) using the pixel values of G15, B16, and G17 shown in FIG.

h16 = B16- (G15 + G17)

Similarly, the other components in the horizontal reference image 40H can also be obtained by performing the processing of (A1) to (A4).

Instead of the processing of (A1) to (A4), the horizontal reference image is executed by sequentially executing any of the following processing (A5) to (A8) for each pixel in the periphery of the interpolation target pixel of the Bayer data. May be generated.

(A5) A process of subtracting the pixel value of the R pixel from the average value of the pixel values of the G pixel adjacent to the R pixel in the horizontal direction with respect to the R pixel adjacent to the G pixel in the horizontal direction.

(A6) Processing for subtracting the average value of pixel values of the R pixels adjacent to the G pixels in the horizontal direction from the pixel values of the G pixels adjacent to the R pixels in the horizontal direction

(A7) Process of subtracting the average value of pixel values of the G pixels adjacent to the G pixels in the horizontal direction from the pixel values of the G pixels adjacent to the B pixels in the horizontal direction

(A8) A process of subtracting the pixel value of the B pixel from the average value of the pixel values of the G pixel adjacent to the B pixel in the horizontal direction with respect to the B pixel adjacent to the G pixel in the horizontal direction

On the other hand, the vertical reference image generation unit 42 performs a calculation on the Bayer data using pixel values of pixels adjacent in the vertical direction, and generates a vertical reference image. The vertical reference image is information for obtaining a direction in which the correlation between pixels is strong around each interpolation target pixel of the Bayer data. Specifically, the vertical reference image generating section 42 sequentially executes any one of the following operations (B1) to (B4) for each pixel around the interpolation target pixel of the Bayer data in the vertical direction. Create a reference image.

(B1) A process of subtracting an average value of pixel values of the R pixel adjacent to the R pixel in the vertical direction from the pixel value of the R pixel adjacent to the G pixel in the vertical direction.

(B2) Processing for subtracting the pixel value of the G pixel from the average value of the pixel values of the R pixel adjacent to the G pixel in the vertical direction with respect to the G pixel adjacent to the R pixel in the vertical direction

(B3) For a G pixel adjacent to the B pixel in the vertical direction, a process of subtracting the pixel value of the G pixel from the average value of the pixel values of the B pixel adjacent to the G pixel in the vertical direction.

(B4) A process of subtracting the average value of pixel values of the B pixels adjacent to the B pixels in the vertical direction from the pixel values of the B pixels adjacent to the G pixels in the vertical direction.

7 shows a vertical reference image 40V as an example of the vertical reference image. Here, each component in the vertical reference image 40V will be described.

In the vertical reference image 40V, the component "v8" becomes the following value by the process of (B1) using the pixel values of G1, R8, and G15 shown in FIG.

v8 = R8- (G1 + G15) / 2

In the vertical reference image 40V, the component "v15" becomes the following value by the process of (B2) using the pixel values of R8, G15, and R22 shown in FIG.

v15 = (R8 + R22) / 2-G15

In the vertical reference image 40V, the component "v9" becomes the following values by the processing of (B3) using the pixel values of B2, G9, and B16 shown in FIG.

v9 = (B2 + B16) / 2-G9

In the vertical reference image 40V, the component "v16" becomes the following value by the process of (B4) using the pixel values of G9, B16, G23 shown in FIG.

v16 = B16- (G9 + G23) / 2

Similarly, the other components in the vertical reference image 40V can also be obtained by performing the processing of (B1) to (B4).

Instead of the processing of (B1) to (B4), the vertical reference image is executed by sequentially executing any one of the following (B5) to (B8) on each pixel in the periphery of the interpolation target pixel of the Bayer data. May be generated.

(B5) For an R pixel adjacent to the G pixel in the vertical direction, a process of subtracting the pixel value of the R pixel from the average value of the pixel values of the G pixel adjacent to the R pixel in the vertical direction.

(B6) Processing for subtracting the average value of the pixel values of the R pixels adjacent to the G pixels in the vertical direction from the pixel values of the G pixels in the vertical direction for the G pixels adjacent to the R pixels in the vertical direction.

(B7) Processing for subtracting the average value of pixel values of the B pixel adjacent to the G pixel value in the vertical direction from the pixel value of the G pixel in the vertical direction for the G pixel adjacent to the B pixel.

(B8) A process of subtracting the pixel value of the B pixel from the average value of the pixel values of the G pixel adjacent to the B pixel in the vertical direction with respect to the B pixel adjacent to the G pixel in the vertical direction.

The horizontal correlation calculator 43 calculates a horizontal correlation diagram Hr, which is a correlation degree in the horizontal direction, from each horizontal interpolation target image in the Bayer data. The horizontal correlation diagram calculating section 43 calculates the horizontal correlation diagram Hr by executing the processing of (C3) in the following (C1).

(C1) The difference value between the components in the horizontal reference image is obtained.

(C2) The absolute value of each difference value calculated | required in (C1) is calculated | required.

(C3) The sum of the values obtained by multiplying the absolute values obtained in (C2) by a predetermined coefficient is calculated as the horizontal correlation diagram (Hr).

As an example of the processing from (C1) to (C3), the horizontal correlation diagram Hr relating to the pixel at the position of "R24" shown in FIG. 5 is as follows.

Hr = abs (h8-h9) + 2 × abs (h9-h10) + 2 × abs (h10-h11) + abs (h11-h12) + 2 × (abs (h15-h16) + 2 × abs (h16- h17) + 2 × abs (h17-h18) + abs (h18-h19)} + 2 × {abs (h22-h23) + 2 × abs (h23-h24) + 2 × abs (h24-h25) + abs ( h25-h26)) + 2 × (abs (h29-h30) + 2 × abs (h30-h31) + 2 × abs (h31-h32) + abs (h32-h33)} + abs (h36-h37) +2 × abs (h37-h38) + 2 × abs (h38-h39) + abs (h39-h40)

Here, abs () represents an operation for finding an absolute value.

The smaller the value of the horizontal correlation diagram Hr, the stronger the correlation in the horizontal direction.

The vertical correlation chart calculating section 44 calculates a vertical correlation chart Vd, which is a correlation chart in the vertical direction, from the vertical reference image for each interpolation target pixel in the Bayer data. The vertical correlation calculation section 44 calculates the vertical correlation diagram Vd by executing the processing of (D3) in the following (D1).

(D1) The difference value between the components in the vertical reference image is obtained.

(D2) The absolute value of each difference value calculated | required in (D1) is calculated | required.

(D3) The sum of the values obtained by multiplying each absolute value obtained in (D2) by a predetermined coefficient is calculated as the vertical correlation diagram Vd.

As an example of the processing from (D1) to (D3), the vertical correlation diagram Vd with respect to the pixel at the position of "R24" shown in FIG. 5 is as follows.

Vd = abs (v8-v15) + 2 × abs (v15-v22) + 2 × abs (v22-v29) + abs (v29-v36) + 2 × (abs (v9-v16) + 2 × abs (v16- v23) + 2 × abs (v23-v30) + abs (v30-v37)} + 2 × {abs (v10-v17) + 2 × abs (v17-v24) + 2 × abs (v24-v31) + abs ( v31-v38)) + 2 × (abs (v11-v18) + 2 × abs (v18-v25) + 2 × abs (v25-v32) + abs (v32-v39)} + abs (v12-v19) +2 × abs (v19-v25) + 2 × abs (v26-v33) + abs (v33-v40)

Here, abs () represents an operation for finding an absolute value.

The smaller the value of the vertical correlation diagram Vd, the stronger the correlation in the vertical direction.

The pixel value determination unit 45 determines the pixel value of the G pixel to be interpolated with respect to the interpolation target pixel based on the horizontal correlation diagram Hr and the vertical correlation diagram Vd.

In other words, when the horizontal correlation diagram Hr is smaller than the vertical correlation diagram Vd, in other words, when the horizontal correlation is stronger than the vertical correlation, the pixel value determination unit 45 is adjacent to each other in the horizontal direction of the interpolation target pixel. The average value of the pixel values of the G pixels is determined as the pixel values of the G pixels to be interpolated with respect to the interpolation target pixel.

In this case, for example, the pixel value determination unit 45 is the pixel value "G24" of the G pixel interpolated with respect to the pixel (interpolation target pixel) at the position "R24" shown in FIG. The following pixel values are determined using the pixel values "G23" and "G25" of the G pixel adjacent to the pixel at the "R24" position.

G24 = (G23 + G25) / 2

On the other hand, in the case where the vertical correlation diagram Vd is smaller than the horizontal correlation diagram Hr, in other words, when the vertical correlation is stronger than the horizontal correlation, the pixel value determination unit 45 is connected to the vertical direction of the pixel to be interpolated. The average value of pixel values of adjacent G pixels is determined as the pixel value of the G pixels to be interpolated with respect to the interpolation target pixel.

In this case, for example, the pixel value determination unit 45 is the pixel value "G24" of the G pixel interpolated with respect to the pixel (interpolation target pixel) at the position "R24" shown in FIG. The following pixel values are determined using the pixel values "G17" and "G31" of the G pixel adjacent to the pixel at the "R24" position.

G24 = (G17 + G31) / 2

The pixel value extraction unit 46 extracts the pixel value determined by the pixel value determination unit as the pixel value of the G pixel to be interpolated with respect to the interpolation target pixel in the Bayer data.

The G pixel extraction circuit 40 carries out the above-described processing on all interpolation target pixels (R pixels without G pixel information) in addition to the pixel at the position of "R24". The pixel values of the G pixels to be interpolated for each interpolation object are extracted. That is, the G pixel extraction circuit 40 generates a horizontal reference image and a vertical reference image for each of all interpolation target pixels in the Bayer data, and generates the horizontal reference image and the vertical reference image. Using the component, the correlation between the pixel values in the horizontal direction and the pixel values in the vertical direction is calculated to extract pixel values of G pixels to be interpolated for each interpolation object. By this processing, an image signal (hereinafter referred to as a "G image signal") consisting of only the information of the appropriate G pixels extracted from the Bayer data is obtained.

Returning to FIG. 3, the 2nd circuit part 30b passes the G image signal obtained by the G pixel extraction circuit 40 through the HPF 37 which is a high pass filter, and removes the low frequency component of a G image signal.

In the adder 36, the signal processing is performed in the first circuit portion 30a, and the G image signal from which the low frequency component is removed is added to the color image signal passed through the LPF (low pass filter) 26, That is, interpolated.

As described above, according to the present embodiment, the pixel values of the G pixels to be interpolated with respect to the Bayer data by referring not only to the G pixels around the interpolation target pixels but also to the R pixels and the B pixels around the interpolation target pixels. Decided. This increases the information of pixels to be referred to when interpolating G pixels with respect to the Bayer data. As a result, an appropriate pixel value of the G pixel can be interpolated with respect to the image signal.

According to the present embodiment, the horizontal reference image and the vertical reference image are generated using the pixel values of the G pixels, the R pixels, and the B pixels around the interpolation target pixel, and these horizontal reference images are generated. The correlation between the pixel values in the horizontal direction and the pixel values in the vertical direction was calculated using the component in the vertical reference image. In this way, the information of the pixel to be referred when calculating the correlation regarding the pixel value increases. As a result, it is possible to reduce the possibility of judging by mistake the direction of strong correlation with the pixel value.

In addition, according to the present embodiment, the signal processing is performed in the first circuit section 30a, and the G pixels extracted by the second circuit section 30b for the color image signal passing through the LPF (low pass filter) 26. Properly interpolated the information. Since the information of the G pixel affects the sharpness of the outline in the reproduced image, in this way, the outline of the reproduced image based on the Bayer data can be emphasized to improve the image quality of the reproduced image.

< Variant ＞

Next, the modification of this embodiment is demonstrated. The difference in the correlation between the horizontal direction and the vertical direction may be small with respect to the pixel around the interpolation target pixel. That is, the gradation may be gentle around the interpolation pixel. In this case, when the G pixel is interpolated with respect to the interpolation target pixel of the Bayer data by the processing of the above embodiment, the pixel value of the interpolation target pixel becomes a prominent value (isolating point) and the image quality of the reproduced image based on the image signal is deteriorated. There is problem to say. This modification copes with such a problem.

In this modification, as shown in FIG. 8, only the point in which the mixing ratio determination section 47 is provided in the G pixel extraction circuit 40 differs from the above embodiment. Other configurations and processes are the same as in the above embodiment.

The mixing ratio determination unit 47 determines the mixing ratio from the horizontal correlation diagram Hr calculated by the horizontal correlation calculator 43 and the vertical correlation diagram Vd calculated by the vertical correlation calculator 44.

When the horizontal correlation diagram Hr is smaller than the vertical correlation diagram Vd, that is, when the horizontal correlation is stronger than the vertical correlation, the mixing ratio determination unit 47 determines the value consisting of Hr / Vd as the mixing ratio. . On the other hand, when the vertical correlation diagram Vd is smaller than the horizontal correlation diagram Hr, that is, the correlation in the vertical direction is stronger than the correlation in the horizontal direction, the mixing ratio determination unit 47 sets the value composed of Vd / Hr as the mixing ratio. Decide

When the correlation in the horizontal direction is stronger than the correlation in the vertical direction, the pixel value determination unit 45 interpolates the average value of the pixel values of the G pixels adjacent to the interpolation target pixel in the horizontal direction, and the interpolation target pixels in the horizontal direction and the vertical direction. The G pixel of the pixel value obtained by adding the average value of four G pixels adjacent to the ratio based on the mixing ratio is determined as the pixel value of the G pixel to be interpolated with respect to the interpolation target pixel.

In this case, for example, the pixel value "G24" of the G pixel interpolated with respect to the pixel at the "R24" position shown in FIG. 5 becomes the following pixel values.

G24 = (1.0- (Hr / Vd)) × (G23 + G25) / 2 + (Hr / Vd) × (G17 + G31 + G23 + G25) / 4

When the correlation in the vertical direction is stronger than the correlation in the horizontal direction, the pixel value determination unit 45 interpolates the average value of the pixel values of the G pixels adjacent to the interpolation target pixel in the vertical direction, and the interpolation target pixels in the horizontal direction and the vertical direction. The G pixel of the pixel value obtained by adding the average value of four G pixels adjacent to the ratio based on the mixing ratio is determined as the pixel value of the G pixel to be interpolated with respect to the interpolation target pixel.

In this case, for example, the pixel value "G24" of the G pixel interpolated with respect to the pixel at the "R24" position shown in FIG. 5 becomes the following pixel values.

G24 = (1.0- (Vd / Hr)) × (G17 + G31) / 2 + (Vd / Hr) × (G17 + G31 + G23 + G25) / 4

The pixel value extraction unit 46 extracts the pixel value determined by the pixel value determination unit as the pixel value of the G pixel to be interpolated with respect to the interpolation target pixel in the Bayer data. As a result, an image signal (G image signal) consisting only of information of appropriate G pixels extracted from Bayer data is obtained.

3, in the adder 36, the signal processing is performed in the first circuit portion 30a, and the G image signal is added to the color image signal passed through the LPF (low pass filter) 26. That is, interpolated.

According to the modification described above, in the horizontal direction and the vertical direction, as well as the average value of two pixels adjacent to the interpolation target pixel in the direction of high correlation in determining the pixel value of the G pixel to be interpolated for the Bayer data. The average value of four pixels adjacent to the interpolation target pixel was also used. In this way, even when the difference in the horizontal and vertical correlations is small, the pixel value of the interpolation target pixel does not become a prominent value (isolating point), and deterioration in image quality of the reproduced image based on the image signal can be prevented. Can be.

This invention is not limited to said embodiment and modification, The design change of the range which does not deviate from the summary of this invention is also included.

30: pixel interpolation unit 31: wideband LPF
32: G interpolation circuit 33: R interpolation circuit
34: B interpolation circuit 36: Adder
37: LPF 38: Subtractor
40: pixel interpolation circuit 41: horizontal reference image generation unit
42: vertical reference image generation unit 43: horizontal correlation calculation unit
44: vertical correlation calculator 45: pixel value determiner
46: pixel value extraction unit 47: mixing ratio determination unit
100: digital camera

Claims (9)

The pixel value of the G pixel to be interpolated to the interpolation target pixel is determined and determined using only the pixel value of the G pixel in the peripheral area of the interpolation target pixel that is the pixel in which the pixel value of the G pixel does not exist in the Bayer data. A pixel interpolation unit which generates an interpolation image signal by interpolating a pixel value of a G pixel to the interpolation target pixel;
In the Bayer data, a pixel value of a pixel that is a pixel of interest in an R pixel, a G pixel, and a B pixel, and a pixel value of a peripheral pixel of a color component different from the color component of the pixel of interest as a pixel value of a peripheral pixel of a peripheral region of the pixel of interest. A correlation calculating section for calculating a correlation between the horizontal direction and the vertical direction in the peripheral region of the interpolation target pixel by performing an operation between pixel values;
The interpolation target pixel may be interpolated based on the pixel values of pixels adjacent to the interpolation target pixel in either the horizontal direction or the vertical direction showing a stronger correlation among the correlations calculated by the correlation calculation unit. A pixel value determination unit which determines a second G interpolation pixel value,
A G-image signal acquisition unit for acquiring a G-image signal composed of only G pixels consisting of second G-interpolated pixel values determined by the pixel value determination unit;
A high pass filter for removing low frequency components of the G image coral obtained by the G image signal acquisition unit;
An adder for adding a second G interpolation pixel value in the G image signal from which low frequency components have been removed by the high pass filter to an interpolation image signal generated by the pixel interpolation unit,
The pixel value determiner,
In each of the correlations calculated by the correlation calculating section, in either the horizontal direction or the vertical direction showing a stronger correlation, the average value of pixel values of pixels adjacent to each of the interpolation target pixels, and in the horizontal and vertical directions. Wherein the pixel value obtained by adding an average value of pixels adjacent to the interpolation target pixel according to a predetermined ratio is determined as the second G interpolation pixel value. The method of claim 1,
A first reference image generation unit for performing a calculation using pixel values of pixels adjacent in the horizontal direction in the peripheral region of the interpolation target pixel, and generating a first reference image;
And a second reference image generation unit configured to perform a calculation using pixel values of pixels adjacent to each other in the vertical direction in the peripheral region of the interpolation target pixel, and generate a second reference image.
The correlation calculation unit,
Calculating the correlation in the horizontal direction based on the first reference image generated by the first reference image generating unit,
And a correlation in the vertical direction is calculated based on the second reference image generated by the second reference image generating unit. The method of claim 2,
The first reference image generating unit,
In the peripheral region of the interpolation target pixel, a difference value between a pixel value of the first pixel and an average value of respective pixel values of the second pixel and the third pixel adjacent to the first pixel in the horizontal direction is obtained. A horizontal difference calculating unit,
Generate the first reference image based on each difference value obtained by the horizontal difference calculator;
The second reference image generating unit,
In the peripheral region of the interpolation target pixel, a difference value between a pixel value of the first pixel and an average value of respective pixel values of the fourth pixel and the fifth pixel adjacent to the first pixel in the vertical direction is obtained. A vertical difference calculating unit,
And the second reference image is generated based on each difference value obtained by the vertical difference calculator. The method of claim 2,
The first reference image generating unit,
A first horizontal difference calculating section for obtaining a difference value between the pixel value of the R pixel and the average value of the pixel value of the G pixel adjacent to the R pixel in the horizontal direction;
A second horizontal difference calculator which acquires a difference value between the pixel value of the B pixel and the average value of the pixel value of the G pixel adjacent to the B pixel in the horizontal direction;
A third horizontal difference calculating section for acquiring a difference value between the pixel value of the R pixel adjacent to the G pixel and the pixel value of the G pixel in the horizontal direction;
And a fourth horizontal difference calculating section for acquiring the difference value between the pixel value of the B pixel adjacent to the G pixel and the pixel value of the G pixel in the horizontal direction.
Generating the first reference image based on each difference value obtained by the first to fourth horizontal difference calculation units,
The second reference image generating unit,
A first vertical difference calculating section for obtaining a difference value between the pixel value of the R pixel and the average value of the pixel value of the G pixel adjacent to the R pixel in the vertical direction;
A second vertical difference calculator which acquires a difference value between the pixel value of the B pixel and the average value of the pixel value of the G pixel adjacent to the B pixel in the vertical direction;
A third vertical difference calculator which acquires a difference value between the pixel value of the R pixel adjacent to the G pixel and the pixel value of the G pixel in the vertical direction;
A fourth vertical difference calculating section for acquiring the difference value between the pixel value of the B pixel adjacent to the G pixel and the pixel value of the G pixel in the vertical direction;
And the second reference image is generated based on the difference values obtained by the first to fourth vertical difference calculating units. The method of claim 4, wherein
The correlation calculation unit,
A first calculating unit for calculating a first correlation value that is a sum of a value obtained by multiplying an absolute value of each difference value obtained by the first to fourth horizontal difference calculating units by a predetermined coefficient;
And a second calculating section for calculating a second correlation value that is a sum of a value obtained by multiplying a predetermined coefficient by each of the absolute values of the difference values obtained by the first to fourth vertical difference calculating sections,
A pixel interpolation circuit for calculating a horizontal correlation and a vertical correlation based on a first correlation value calculated by the first calculation unit and a second correlation value calculated by the second calculation unit. . The method of claim 1,
The pixel value determiner,
Among the correlations calculated by the correlation calculator, the average value of pixel values of pixels adjacent to the interpolation target pixel in either the horizontal direction or the vertical direction showing a stronger correlation is used as the second G interpolation pixel. A pixel interpolation circuit characterized in that it is determined as a value. The method of claim 5, wherein
And said predetermined ratio is a ratio based on a first correlation value calculated by said first calculator and a second correlation value calculated by said second calculator. The pixel value of the G pixel to be interpolated to the interpolation target pixel is determined and determined using only the pixel value of the G pixel in the peripheral area of the interpolation target pixel that is the pixel in which the pixel value of the G pixel does not exist in the Bayer data. Generating an interpolation image signal by interpolating a pixel value of a G pixel to the interpolation target pixel;
In the Bayer data, a pixel value of a pixel that is a pixel of interest in an R pixel, a G pixel, and a B pixel, and a pixel value of a peripheral pixel of a color component different from the color component of the pixel of interest as a pixel value of a peripheral pixel of a peripheral region of the pixel of interest. Calculating a correlation between the horizontal direction and the vertical direction in the peripheral region of the interpolation target pixel by performing a calculation between pixel values;
Among the calculated correlations, an average value of pixel values of pixels adjacent to the interpolation target pixel in either the horizontal direction or the vertical direction showing a stronger correlation is applied to the interpolation target pixel in the horizontal and vertical directions. Determining a second G interpolation pixel value for interpolating the pixel value obtained by adding the average value of adjacent pixels in a predetermined ratio to the interpolation target pixel;
Acquiring a G image signal composed of only the G pixels consisting of the determined second G interpolation pixel values;
Removing the low frequency component of the obtained G image signal;
And adding a second G interpolation pixel value in the G image signal from which the low frequency component has been removed to the generated interpolation image signal. On your computer,
The pixel value of the G pixel to be interpolated to the interpolation target pixel is determined and determined using only the pixel value of the G pixel in the peripheral area of the interpolation target pixel that is the pixel in which the pixel value of the G pixel does not exist in the Bayer data. A process of generating an interpolation image signal by interpolating a pixel value of a G pixel to the interpolation target pixel;
In the Bayer data, a pixel value of a pixel that is a pixel of interest in an R pixel, a G pixel, and a B pixel, and a pixel value of a peripheral pixel of a color component different from the color component of the pixel of interest as a pixel value of a peripheral pixel of a peripheral region of the pixel of interest. A process of calculating the correlation between the horizontal direction and the correlation between the vertical direction in the peripheral region of the interpolation target pixel by performing an operation between pixel values;
Among the calculated correlations, an average value of pixel values of pixels adjacent to the interpolation target pixel in either the horizontal direction or the vertical direction showing a stronger correlation is applied to the interpolation target pixel in the horizontal and vertical directions. A process of determining a second G interpolation pixel value for interpolating the pixel value obtained by adding the average value of adjacent pixels in a predetermined ratio to the interpolation target pixel;
Processing for acquiring a G image signal composed of only the G pixels consisting of the determined second G interpolation pixel values;
A process of removing the low frequency component of the obtained G image signal;
And a program for executing a process of adding a second G interpolation pixel value in the G image signal from which the low frequency component has been removed to the generated interpolation image signal.

Images