JP2018056653A - Image processing device, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device that can reduce a block-shaped false pattern for a subject having a slant striped shape in filter processing.SOLUTION: An image processing device comprises first color difference signal calculation means (305) that calculates a color difference signal in a vertical direction of an input image, second color difference signal calculation means (306) that calculates the color difference signal in a horizontal direction of the input image, first filter processing means (310, 311) that performs filter processing to the color difference signal in the vertical direction, and second filter processing means (312, 313) that performs filter processing to the color difference signal in the horizontal direction, and the first filter processing means and the second filter processing means change strength of the filter processing based on the color difference signal in the vertical direction and the color difference signal in the horizontal direction.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  In Patent Document 1, when interpolating a G signal that is a main component of a luminance signal, a correlation in the vertical direction or the horizontal direction is determined, and if there is a correlation in the vertical direction, LPF processing is performed in the vertical direction. A technique for performing LPF processing in the horizontal direction when there is a correlation in the horizontal direction is disclosed. If there is no correlation in either the vertical direction or the horizontal direction, two-dimensional LPF processing is performed so that the vertical and horizontal edges are not blurred.

  However, in the technique of Patent Document 1, since there is a two-dimensional LPF process when there is no correlation in the vertical or horizontal direction, for example, an edge in an oblique direction is blurred compared to an edge in the vertical direction or the horizontal direction.

  Therefore, Non-Patent Document 1 discloses a method for calculating the G signal at the position of the R or B pixel. In Non-Patent Document 1, color difference signals between G pixels and pixels of the same color as the target pixel are calculated in the upper, lower, left, and right directions including the target pixel, and a color difference signal obtained by synthesizing the color difference signals in each direction is obtained. The G signal is calculated by adding to the pixel of interest. At this time, in Non-Patent Document 1, an image with high resolution is generated regardless of the direction by synthesizing the color difference signals in each direction according to the inclination of the color difference signal in the corresponding direction.

Japanese Patent No. 3862506

I. Pekkucuksen, Y. Altunbasak, “Gradient based threshold free color filter array interpolation”, ICIP 2010

  However, in Non-Patent Document 1, when calculating the color difference signals in the upper, lower, left, and right directions, the color difference signals are filtered in each direction. A block-like false pattern may occur in a striped subject.

  An object of the present invention is to provide an image processing apparatus, an image processing method, and a program capable of reducing a block-like false pattern with respect to an oblique striped subject during filtering.

  The image processing apparatus of the present invention includes a first color difference signal calculation unit that calculates a color difference signal in the vertical direction of an input image, a second color difference signal calculation unit that calculates a color difference signal in the horizontal direction of the input image, A first filter processing unit that performs a filtering process on the color difference signal in the vertical direction; and a second filter processing unit that performs a filtering process on the color difference signal in the horizontal direction. The means and the second filter processing means change the intensity of the filter processing based on the color difference signal in the vertical direction and the color difference signal in the horizontal direction.

  According to the present invention, it is possible to reduce a block-like false pattern with respect to an oblique striped subject during the filtering process.

It is a figure which shows 1 unit of a primary color Bayer arrangement | sequence. It is a block diagram which shows the structural example of the image processing apparatus by this embodiment. It is a block diagram which shows the structural example of a G interpolation circuit. It is a flowchart which shows an example of the flow of a process of G interpolation circuit. It is a figure which shows an example of the input-output characteristic of an HV correlation value calculation circuit.

  FIG. 1 is a diagram illustrating an imaging device having a Bayer array color filter according to an embodiment of the present invention. The imaging device has an imaging element such as a CMOS image sensor. The imaging device can be applied to a smartphone, a tablet, an industrial camera, a medical camera, and the like in addition to a digital camera and a video camera. The imaging element has a plurality of pixels arranged in a two-dimensional matrix, and each of the plurality of pixels is a color filter (for example, among three colors of red (R), green (G), and blue (B)). One color filter). FIG. 1 shows one unit of color filter in a primary color Bayer array. The image sensor performs photoelectric conversion and analog-digital conversion for each pixel, and outputs a digital R signal (red signal), G1 signal (green signal), G2 signal (green signal), or B signal (blue signal). To do. The G1 signal and the G2 signal are G signals (green signals). Since an image pickup apparatus having a Bayer array color filter can obtain only one color signal of R, G, and B in each pixel, when obtaining all RGB color signals in each pixel, FIG. It is necessary to perform an interpolation process in the luminance signal generation unit 200 of the second.

  FIG. 2 is a diagram illustrating a configuration example of the luminance signal generation unit 200 according to the present embodiment. The luminance signal generation unit 200 is an image processing apparatus, and inputs digital R, G, and B signals from the image sensor shown in FIG. The luminance signal generation unit 200 includes a WB circuit (white balance circuit) 201, a G interpolation circuit 202, an R interpolation circuit 203, a B interpolation circuit 204, an APC circuit 205, a luminance signal generation circuit 206, and an addition circuit 207. And have.

  The WB circuit 201 inputs digital image signals (R signal, G signal, and B signal) from the image sensor, and corrects the white balance of the image signal. The G interpolation circuit 202 receives the image signal output from the WB circuit 201, calculates the G signal at the positions of the R pixel and the B pixel in FIG. 1 by interpolation, and outputs the G signal of all the pixels. The R pixel is a pixel provided with a red filter, the B pixel is a pixel provided with a blue filter, and the G pixel is a pixel provided with a green filter. Details of the processing of the G interpolation circuit 202 will be described later.

  The R interpolation circuit 203 is a red interpolation unit, and interpolates R signals at the positions of the G pixel and B pixel in the input image of FIG. 1 based on the input image from the WB circuit 201 and the G signal from the G interpolation circuit 202. To calculate the R signal of all pixels. For example, the R interpolation circuit 203 calculates a color difference signal between the R pixel and the pixel of the same color as the target pixel in the upper, lower, left, and right directions including the target pixel, and combines the color difference signals in each direction. The R signal is calculated by adding the signal to the pixel of interest. The R interpolation circuit 203 may calculate the R signal by two-dimensional LPF (low pass filter) processing.

  The B interpolation circuit 204 is blue interpolation means, and interpolates the B signal at the position of the R pixel and the G pixel in the input image of FIG. 1 based on the input image from the WB circuit 201 and the G signal from the G interpolation circuit 202. And B signals of all pixels are output. For example, the B interpolation circuit 204 calculates a color difference signal between the B pixel and a pixel of the same color as the target pixel in the upper, lower, left, and right directions including the target pixel, and combines the color difference signals in each direction. The B signal is calculated by adding the signal to the pixel of interest. Note that the B interpolation circuit 204 may calculate the B signal by two-dimensional LPF processing.

The APC circuit 205 generates an aperture correction signal for the G signal output from the G interpolation circuit 202. The adder circuit 207 adds the G signal output from the G interpolation circuit 202 and the output signal of the APC circuit 204, and outputs a G signal for all pixels. The luminance signal generation circuit 206 is a luminance signal generation unit, and based on the G signal output from the adder circuit 207, the R signal output from the R interpolation circuit 203, and the B signal output from the B interpolation circuit 204, The luminance signal Y for all pixels is generated by the following equation (1).
Y = 0.3R + 0.59G + 0.11B (1)

  FIG. 3 is a block diagram illustrating a configuration example of the G interpolation circuit 202 of FIG. The G interpolation circuit 202 includes a G pixel V interpolation circuit 301, a G pixel H interpolation circuit 302, an R and B pixel V interpolation circuit 303, an R and B pixel H interpolation circuit 304, a V color difference calculation circuit 305, an H And a color difference calculation circuit 306. Further, the G interpolation circuit 202 includes a V color difference inclination calculation circuit 307, an H color difference inclination calculation circuit 308, an HV correlation value calculation circuit 309, an N filter circuit 310, an S filter circuit 311, and a W filter circuit 312. E filter circuit 313. Further, the G interpolation circuit 202 includes an N weight calculation circuit 314, an S weight calculation circuit 315, a W weight calculation circuit 316, an E weight calculation circuit 317, a synthesis circuit 318, and an addition circuit 319.

FIG. 4 is a flowchart showing the flow of the image processing method of the G interpolation circuit 202. In step S401, the G pixel V interpolation circuit 301 calculates the G signal by performing interpolation processing in the vertical direction on the Bayer array input image output from the WB circuit 201, and outputs the G signal of all pixels. . Specifically, the G pixel V interpolation circuit 301 outputs the G signal as it is when the target pixel is a G pixel, and interpolates the G signal in the vertical direction when the target pixel is an R pixel or a B pixel. The G signal is calculated by performing processing. For example, when the X and Y coordinates of the pixel of interest in the image are (j, i) and the pixel of interest is an R pixel, the G pixel V interpolation circuit 301 calculates the G signal Gv _{i, j} using the following equation (2). Calculated by Further, the G pixel V interpolation circuit 301 calculates the G signal by the same method as when the target pixel is the R pixel even when the target pixel is the B pixel.
Gv _{i, j} = (G _{i−1, j} + G _{i + 1, j} ) / 2
+ (2 × R _{i, j} −R _{i−2, j} + R _{i + 2, j} ) / 4 (2)

Next, in step S402, the G pixel H interpolation circuit 302 calculates a G signal by performing an interpolation process in the horizontal direction on the input image output from the WB circuit 201, and outputs the G signal of all pixels. . Specifically, the G pixel H interpolation circuit 302 outputs the G signal as it is when the target pixel is a G pixel, and interpolates the G signal in the horizontal direction when the target pixel is an R pixel or a B pixel. The G signal is calculated by performing processing. For example, when the target pixel is an R pixel, the G pixel H interpolation circuit 302 calculates the G signal Gh _{i, j} by the following equation (3). The G pixel H interpolation circuit 302 calculates the G signal by the same method as that when the target pixel is the R pixel even when the target pixel is the B pixel.
Gh _{i, j} = (G _{i, j-1} + G _{i, j + 1} ) / 2
+ (2 × R _{i, j} −R _{i, j−2} + R _{i, j + 2} ) / 4 (3)

Next, in step S403, the R and B pixel V interpolation circuit 303 calculates an R signal and a B signal by performing interpolation processing in the vertical direction on the input image output from the WB circuit 201, and calculates all the pixels. R signal and B signal are output. Specifically, when the target pixel is an R pixel or a B pixel, the R and B pixel V interpolation circuit 303 outputs the R signal or the B signal as it is. When the pixel of interest is a G pixel and the vertical direction of the pixel of interest is an R pixel, the R and B pixel V interpolation circuit 303 calculates an R signal by performing interpolation processing in the vertical direction. When the vertical direction is a B pixel, the B signal is calculated by performing interpolation processing in the vertical direction. For example, when the target pixel is a G pixel and the vertical direction of the target pixel is an R pixel, the R and B pixel V interpolation circuit 303 calculates the R signal Rv _{i, j} by the following equation (4). The R and B pixel V interpolation circuit 303 calculates the B signal by the same method as that when the vertical direction of the target pixel is the R pixel even when the vertical direction of the target pixel is the B pixel.
Rv _{i, j} = (R _{i−1, j} + R _{i + 1, j} ) / 2
+ (2 × G _{i, j} −G _{i−2, j} + G _{i + 2, j} ) / 4 (4)

In step S404, the R and B pixel H interpolation circuit 304 calculates an R signal and a B signal by performing an interpolation process in the horizontal direction on the input image output from the WB circuit 201, and calculates all the pixels. R signal and B signal are output. Specifically, when the target pixel is an R pixel or a B pixel, the R and B pixel H interpolation circuit 304 outputs the R signal or the B signal as it is. When the target pixel is a G pixel and the horizontal direction of the target pixel is an R pixel, the R and B pixel H interpolation circuit 304 calculates an R signal by performing an interpolation process in the horizontal direction, When the horizontal direction is B pixels, the B signal is calculated by performing interpolation processing in the horizontal direction. For example, when the target pixel is a G pixel and the horizontal direction of the target pixel is an R pixel, the R and B pixel H interpolation circuit 304 calculates the R signal Rh _{i, j} by the following equation (5). The R and B pixel H interpolation circuit 304 calculates the B signal by the same method as that when the horizontal direction of the target pixel is the R pixel even when the horizontal direction of the target pixel is the B pixel.
Rh _{i, j} = (R _{i, j-1} + R _{i, j + 1} ) / 2
+ (2 × G _{i, j} −G _{i, j−2} + G _{i, j + 2} ) / 4 (5)

  In steps S401 to S404, the equations (2) to (5) are used as the vertical and horizontal interpolation methods for the G pixel and the R and B pixels. However, the present invention is not limited to this. May be calculated by linear interpolation.

  The G pixel V interpolation circuit 301 and the R and B pixel V interpolation circuits 303 are first interpolation means, and output R signals, G signals, and B signals of all pixels by interpolation processing in the vertical direction with respect to the input image. . The G pixel H interpolation circuit 302 and the R and B pixel H interpolation circuits 304 are second interpolation means, and output an R signal, a G signal, and a B signal of all the pixels by horizontal interpolation processing on the input image. .

Next, in step S405, the V color difference calculation circuit 305 subtracts the R signal or B signal output from the R and B pixel V interpolation circuit 303 from the G signal output from the G pixel V interpolation circuit 301. The color difference signal in the vertical direction of the input image is calculated. The V color difference calculation circuit 305 is a first color difference signal calculation unit. In the V color difference calculation circuit 305, the signal output from the G pixel V interpolation circuit 301 is the G signal Gv _{i, j} and the signal output from the R and B pixel V interpolation circuit 303 is R in the target pixel (j, i). In the case of the signal Rv _{i, j} , the color difference signal Diff_v in the vertical direction is calculated by the following equation (6). The V color difference calculation circuit 305 uses the same method as that when the signal output from the R and B pixel V interpolation circuit 303 is the R signal even when the signal output from the R and B pixel V interpolation circuit 303 is the B signal. , A vertical color difference signal between the G signal and the B signal is calculated.
Diff_v _{i, j} = Gv _{i, j} −Rv _{i, j} (6)

Next, in step S406, the H color difference calculation circuit 306 subtracts the R signal or B signal output from the R and B pixel H interpolation circuit 304 from the G signal output from the G pixel H interpolation circuit 302. The horizontal color difference signal of the input image is calculated. The H color difference calculation circuit 306 is a second color difference signal calculation unit. In the H color difference calculation circuit 306, in the pixel of interest (j, i), the signal output from the G pixel H interpolation circuit 302 is the G signal Gh _{i, j} , and the signal output from the R, B pixel H interpolation circuit 304 is R In the case of the signal Rv _{i, j} , the color difference signal Diff_h in the horizontal direction is calculated by the following equation (7). The H color difference calculation circuit 306 uses the same method as when the signal output from the R and B pixel H interpolation circuit 304 is an R signal even when the signal output from the R and B pixel H interpolation circuit 304 is a B signal. The horizontal color difference signals of the G signal and the B signal are calculated.
_{Diff_h i, j = Gh i,} j -Rh i, j ··· (7)

Next, in step S407, the V color difference inclination calculation circuit 307 calculates the inclination of the color difference in the vertical direction based on the color difference signal output from the V color difference calculation circuit 305. The V color difference inclination calculation circuit 307 is a first inclination calculation unit. Specifically, the V color difference inclination calculation circuit 307 calculates a vertical color difference inclination signal Grad_v based on the color difference signal Diff_v output from the V color difference calculation circuit 305 in the pixel of interest (j, i) as follows: 8).
Grad_v _{i, j} = | Diff_v _{i−1, j} −Diff_v _{i + 1, j} | (8)

In step S <b> 408, the H color difference inclination calculation circuit 308 calculates the horizontal color difference inclination based on the color difference signal output from the H color difference calculation circuit 306. The H color difference inclination calculation circuit 308 is a second inclination calculation unit. Specifically, the H color difference inclination calculation circuit 308 generates a horizontal color difference inclination signal Grad_h based on the color difference signal Diff_h output from the H color difference calculation circuit 306 in the pixel of interest (j, i) as follows: 9).
Grad_h _{i, j} = | Diff_h _{i, j−1} −Diff_h _{i, j + 1} | (9)

Next, in step S409, the HV correlation value calculation circuit 309 calculates a correlation value Ratio_HV in the horizontal direction and the vertical direction based on the gradient signal Grad_v in the vertical direction and the gradient signal Grad_h in the horizontal direction. The HV correlation value calculation circuit 309 is a correlation value calculation unit. Specifically, first, the HV correlation value calculation circuit 309 includes the vertical color difference inclination signal Grad_v input from the V color difference inclination calculation circuit 307 and the horizontal color difference input from the H color difference inclination calculation circuit 308. An absolute value signal of the difference between the inclination signals Grad_h is calculated. Here, the absolute value signal of the difference in the pixel of interest (j, i) is defined as Diff_Grad. The HV correlation value calculation circuit 309 calculates the absolute value signal Diff_Grad of the difference between the vertical color difference inclination signal Grad_v and the horizontal color difference inclination signal Grad_h by the following equation (10).
Diff_Grad _{i, j} = | Grad_v _{i, j} −Grad_h _{i, j} | (10)

For example, when there is a color edge in the horizontal direction, the color difference changes in the vertical direction, and the color difference does not change in the horizontal direction. Therefore, the inclination Grad_v _i chrominance _{vertical, j} becomes a large value, the slope Grad_h _i chrominance _{horizontal, j} becomes a small value, the absolute value signal Diff_Grad results to the difference becomes a large value.

Similarly, when there is a color edge in the vertical direction, the gradient grad_v _{i, j} of the color difference in the vertical direction becomes a small value, and the gradient grad_h _{i, j} of the color difference in the horizontal direction becomes a large value, resulting in the absolute difference. The value signal Diff_Grad has a large value.

In addition, when there is an oblique color edge, the change in the color difference in the vertical direction and the change in the color difference in the horizontal direction are approximately the same. Therefore, the inclination Grad_v _{i, j} of the color difference in the vertical direction, the inclination Grad_h _i chrominance _{horizontal, j} becomes a comparable value, the absolute value signal Diff_Grad results to the difference is a small value.

When there is no color edge, the vertical color difference and the horizontal color difference do not change, and therefore the vertical color difference gradient Grad_v _{i, j} and the horizontal color difference gradient Grad_h _{i, j} are small values. The difference absolute value signal Diff_Grad also becomes a small value.

  Next, the HV correlation value calculation circuit 309 calculates an HV correlation value Ratio_HV for each pixel based on the difference absolute value signal based on Diff_Grad.

  FIG. 5 is a diagram illustrating an example of a conversion table when the HV correlation value calculation circuit 309 calculates the HV correlation value Ratio_HV from the difference absolute value signal Diff_Grad. In FIG. 5, the horizontal axis represents the difference absolute value signal Diff_Grad, and the vertical axis represents the HV correlation value Ratio_HV. When the absolute value signal Diff_Grad of the difference is smaller than the first threshold value Th1, the HV correlation value calculation circuit 309 can be regarded as having no color edge in the horizontal direction and the vertical direction, so 0.0 is set as the HV correlation value Ratio_HV. Output. Further, when the absolute value signal Diff_Grad of the difference is larger than the second threshold value Th2, the HV correlation value calculation circuit 309 can consider that there is a color edge in the horizontal direction and the vertical direction, so that the HV correlation value Ratio_HV is 1.0. Is output. Here, the second threshold Th2 is larger than the first threshold Th1. When the difference absolute value signal Diff_Grad is greater than or equal to the first threshold value Th1 and less than or equal to the second threshold value Th2, the HV correlation value calculation circuit 309 linearly varies according to the magnitude of the difference absolute value signal Diff_Grad. An HV correlation value Ratio_HV between .0 and 1.0 is output.

  Next, in step S410, the N filter circuit 310 performs an upward filter process on the color difference signal output from the V color difference calculation circuit 305 based on the HV correlation value Ratio_HV output from the HV correlation value calculation circuit 309. I do. Specifically, first, the N filter circuit 310 uses the color difference signal Diff_v output from the V color difference calculation circuit 305 to calculate the result Fil_n of the upward filter processing on the pixel of interest (j, i) as follows: ).

Next, the N filter circuit 310 calculates and outputs the color difference signal Diff_n by the following equation (12) based on the HV correlation value Ratio_HV.
Diff_n _{i, j} = Ratio_HV _{i, j} × Fil_n _{i, j}
+ (1.0-Ratio_HV _{i, j} ) × Diff_v _{i, j} (12)

  Next, in step S411, the S filter circuit 311 performs a downward filter process on the color difference signal output from the V color difference calculation circuit 305 based on the HV correlation value output from the HV correlation value calculation circuit 309. Do. Specifically, first, the S filter circuit 311 uses the color difference signal Diff_v output from the V color difference calculation circuit 305 to obtain a filter result Fil_s in the downward direction for the pixel of interest (j, i) as follows: ).

Next, the S filter circuit 311 calculates the color difference signal Diff_s by the following equation (14) based on the HV correlation value Ratio_HV and outputs it.
Diff_s _{i, j} = Ratio_HV _{i, j} × Fil_s _{i, j}
+ (1.0-Ratio_HV _{i, j} ) × Diff_v _{i, j} (14)

  Next, in step S <b> 412, the W filter circuit 312 performs leftward filtering on the color difference signal output from the H color difference calculation circuit 306 based on the HV correlation value output from the HV correlation value calculation circuit 309. Do. Specifically, first, the W filter circuit 312 uses the color difference signal Diff_h output from the H color difference calculation circuit 306 to calculate the result Fil_w of the leftward filter processing for the pixel of interest (j, i) as follows: ).

Next, the W filter circuit 312 calculates and outputs the color difference signal Diff_W by the following equation (16) based on the HV correlation value Ratio_HV.
Diff_w _{i, j} = Ratio_HV _{i, j} × Fil_w _{i, j}
+ (1.0-Ratio_HV _{i, j} ) × Diff_h _{i, j} (16)

  Next, in step S413, the E filter circuit 313 performs rightward filter processing on the color difference signal output from the H color difference calculation circuit 306 based on the HV correlation value output from the HV correlation value calculation circuit 309. Do. Specifically, first, the E filter circuit 313 uses the color difference signal Diff_h output from the H color difference calculation circuit 306 to obtain the filter processing result Fil_e in the right direction for the pixel of interest (j, i) as follows: ).

Next, the E filter circuit 313 calculates and outputs the color difference signal Diff_e by the following equation (18) based on the HV correlation value Ratio_HV.
Diff_e _{i, j} = Ratio_HV _{i, j} × Fil_e _{i, j}
+ (1.0-Ratio_HV _{i, j} ) × Diff_h _{i, j} (18)

  Next, in step S414, the N weight calculation circuit 314 calculates the upward weight Wn by the following equation (19) based on the vertical color difference inclination signal Grad_v output from the V color difference inclination calculation circuit 307. .

  Next, in step S415, the S weight calculation circuit 315 calculates the downward weight Ws by the following equation (20) based on the vertical color difference inclination signal Grad_v output from the V color difference inclination calculation circuit 307. .

  Next, in step S416, the W weight calculation circuit 316 calculates the left weight Ww by the following equation (21) based on the horizontal color difference inclination signal Grad_h output from the H color difference inclination calculation circuit 308. .

  Next, in step S417, the E weight calculation circuit 317 calculates the right weight We by the following equation (22) based on the horizontal color difference gradient signal Grad_h output from the H color difference gradient calculation circuit 308. .

Next, in step S418, the synthesis circuit 318 receives the weight Wn input from the N weight calculation circuit 314, the weight Ws input from the S weight calculation circuit 315, the weight Ww input from the W weight calculation circuit 316, and E The weight We input from the weight calculation circuit 317 is input. Further, the synthesis circuit 318 inputs the color difference signal Diff_n input from the N filter circuit 310 and the color difference signal Diff_s input from the S filter circuit 311. Further, the synthesis circuit 318 receives the color difference signal Diff_w input from the W filter circuit 312 and the color difference signal Diff_e input from the E filter circuit 313. The combining circuit 318 combines the color difference signal Diff_n, the color difference signal Diff_s, the color difference signal Diff_w, and the color difference signal Diff_e based on the weight Wn, the weight Ws, the weight Ww, and the weight We, and the color difference signal Diff_mix is expressed by the following formula ( 23). The synthesis circuit 318 is a synthesis unit.
Diff_mix _{i, j} = (Wn _{i, j} × Diff_n _{i, j} + Ws _{i, j} × Diff_s _{i, j}
+ Ww _{i, j} × Diff_w _{i, j} + We _{i, j} × Diff_e _{i, j} ) / Wt _{i, j}
Wt _{i, j} = Wn _{i, j} + Ws _{i, j} + Ww _{i, j} + We _{i, j}
(23)

  Next, in step S419, when the target pixel of the input image input to the G interpolation circuit 202 is an R pixel or a B pixel, the adder circuit 319 is combined with the signal of the target pixel by the combining circuit 318. The color difference signal Diff_mix is added to calculate and output a G signal. In addition, when the target pixel is a G pixel, the addition circuit 319 outputs the image signal input to the G interpolation circuit 202 as it is. The adder circuit 319 is a first adder. An output signal of the adder circuit 319 is an output signal of the G interpolation circuit 202.

  As described above, the N filter circuit 310 performs an upward filter process, the S filter circuit 311 performs a downward filter process, the W filter circuit 312 performs a left filter process, and the E filter circuit 313 performs a right filter process. Perform direction filtering. The N filter circuit 310 and the S filter circuit 311 are first filter processing means, and perform filter processing on the color difference signal in the vertical direction. The W filter circuit 312 and the E filter circuit 313 are second filter processing means and perform filter processing on the color difference signal in the horizontal direction. Here, when there is an oblique color edge, there is no correlation between the horizontal direction and the vertical direction. Therefore, if there is a color edge in an oblique direction and these filter circuits 310 to 313 perform the filter process without using the HV correlation value Ratio_HV calculated by the HV correlation value calculation circuit 309, the filter process Wrapping occurs. As a result, a block-like false pattern is generated with respect to an oblique striped subject, and the image quality is degraded.

  In the present embodiment, an HV correlation value calculation circuit 309 is provided. The HV correlation value calculation circuit 309 can calculate the absolute value signal Diff_Grad of the difference and the HV correlation value Ratio_HV and detect the presence of the color edge in the oblique direction. When there is a color edge in the horizontal direction and when there is a color edge in the vertical direction, the difference absolute value signal Diff_Grad and the HV correlation value Ratio_HV become large values, and the filter circuits 310 to 313 perform the filtering process with a strong intensity. Do. The filter circuits 310 to 313 change the strength of the filter processing based on the HV correlation value Ratio_HV.

  As shown in FIG. 5, when the absolute value signal Diff_Grad of the difference is smaller than the first threshold value Th1, the HV correlation value Ratio_HV is 0.0, and the filter circuits 310 to 313 do not perform the filtering process. When the difference absolute value signal Diff_Grad is equal to or greater than the first threshold (first threshold Th1 or greater), the HV correlation value Ratio_HV is greater than 0.0, and the filter circuits 310 to 313 perform filter processing.

  When there is an oblique color edge and when there is no color edge, the difference absolute value signal Diff_Grad and the HV correlation value Ratio_HV are small values, and the filter circuits 310 to 313 do not perform the filter processing (filter processing is performed). Weaken). As described above, when there is an oblique color edge, the filter circuits 310 to 313 do not perform the filter process or reduce the strength of the filter process, thereby preventing the aliasing due to the filter process. . As a result, it is possible to reduce the occurrence of a block-like false pattern with respect to an oblique striped subject and improve the image quality.

  According to this embodiment, when generating the G signal and the luminance signal from the image signal of the Bayer array, an image with a high resolution feeling is generated regardless of the direction, and the block shape with respect to the oblique striped subject is generated. An image with reduced false patterns can be obtained.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

305 V color difference calculation circuit, 306 H color difference calculation circuit, 310 N filter circuit, 311 S filter circuit, 312 W filter circuit, 313 W filter circuit

Claims (12)

First color difference signal calculating means for calculating a color difference signal in the vertical direction of the input image;
Second color difference signal calculating means for calculating a color difference signal in the horizontal direction of the input image;
First filter processing means for performing filter processing on the color difference signal in the vertical direction;
Second filter processing means for performing filter processing on the horizontal color difference signal;
The image processing apparatus characterized in that the first filter processing means and the second filter processing means change the intensity of filter processing based on the color difference signal in the vertical direction and the color difference signal in the horizontal direction. A first inclination calculating means for calculating the inclination of the color difference in the vertical direction based on the color difference signal in the vertical direction;
Second inclination calculating means for calculating the inclination of the color difference in the horizontal direction based on the color difference signal in the horizontal direction;
The first filter processing means and the second filter processing means change the intensity of filter processing based on a difference between the inclination of the color difference in the vertical direction and the inclination of the color difference in the horizontal direction. The image processing apparatus according to claim 1.   The first filter processing unit and the second filter processing unit do not perform a filter process when the difference is smaller than a first threshold, and perform a filter process when the difference is equal to or larger than the first threshold. The image processing apparatus according to claim 2, wherein: And a correlation value calculating means for calculating a correlation value between the vertical direction and the horizontal direction based on a difference between the inclination of the color difference in the vertical direction and the inclination of the color difference in the horizontal direction,
The image processing apparatus according to claim 2, wherein the first filter processing unit and the second filter processing unit change the strength of the filter processing based on the correlation value. The first filter processing means includes:
Filter processing means for performing upward filter processing on the vertical color difference signal;
Filter processing means for performing a downward filter process on the vertical color difference signal;
The second filter processing means includes
Filter processing means for performing filter processing in the left direction on the horizontal color difference signal;
5. The image processing apparatus according to claim 1, further comprising: a filter processing unit configured to perform a filter process in a right direction with respect to the color difference signal in the horizontal direction.   6. The apparatus according to claim 1, further comprising a combining unit that combines the color difference signal filtered by the first filter processing unit and the color difference signal filtered by the second filter processing unit. The image processing apparatus according to any one of the above.   Furthermore, when the pixel of interest of the input image is a pixel of a red filter or a pixel of a blue filter, a green signal is output by adding the color difference signal synthesized by the synthesis means to the signal of the pixel of interest The image processing apparatus according to claim 6, further comprising a first addition unit configured to perform the first addition. A first interpolation unit that outputs a red signal, a green signal, and a blue signal of all pixels by interpolation processing in a vertical direction with respect to the input image;
Second interpolation means for outputting a red signal, a green signal, and a blue signal of all pixels by a horizontal interpolation process with respect to the input image;
The first color difference calculation means calculates a color difference signal in the vertical direction of the input image based on the output signal of the first interpolation means,
The said 2nd color difference calculation means calculates the color difference signal of the horizontal direction of the said input image based on the output signal of the said 2nd interpolation means, The any one of Claim 1 thru | or 7 characterized by the above-mentioned. The image processing apparatus described. Furthermore, based on the input image and the green signal output by the first addition means, a red interpolation means for interpolating the red signal of the input image;
Blue interpolation means for interpolating a blue signal of the input image based on the input image and the green signal output by the first addition means;
Luminance signal generation means for generating a luminance signal based on the green signal output by the first addition means, the red signal interpolated by the red interpolation means, and the blue signal interpolated by the blue interpolation means. The image processing apparatus according to claim 7.   The image processing apparatus according to claim 1, wherein the input image is a Bayer array image. A first color difference signal calculating step of calculating a color difference signal in the vertical direction of the input image by the first color difference signal calculating means;
A second color difference signal calculating step of calculating a horizontal color difference signal of the input image by a second color difference signal calculating means;
A first filter processing step of performing a filter process on the color difference signal in the vertical direction by a first filter processing unit;
A second filter processing step of performing a filter process on the horizontal color difference signal by a second filter processing means;
In the first filter processing step and the second filter processing step, an intensity of filter processing is changed based on the vertical color difference signal and the horizontal color difference signal.   A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 10.

Images