JP2019097060A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

To generate an image in which a block-like false pattern to an oblique linear photographic subject is reduced.SOLUTION: An image processing apparatus includes: means (305) calculating a vertical color difference signal of an input image signal; means (306) calculating the horizontal color difference signal of the input image signal; first filter processing means (310 and 311) performing a filter processing on the color vertical difference signal; second filter processing means (312 and 313) performing the filter processing on the color horizontal difference signal; and means (309) calculating a first degree indicating whether a pixel of interest of the input image signal is in a region where the pixel of interest receives influence of a prescribed color aberration or an optical folding. The first filter processing means and second filter processing means change characteristics of the filter processing on the basis of a calculation result of the means (309).SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  According to Patent Document 1, when performing G signal interpolation, which is the main component of a luminance signal, vertical or horizontal correlation is determined, and if vertical correlation is present, LPF processing is performed in the vertical direction. A technique is disclosed that performs LPF processing in the lateral direction when there is a correlation in the lateral direction. When there is no correlation in the vertical direction or in the horizontal direction, two-dimensional LPF processing is performed to prevent the edges in the vertical direction and the horizontal direction from being blurred.

  However, in the technique of Patent Document 1, two-dimensional LPF processing is performed when there is no correlation in the vertical or horizontal direction, so that, for example, the edge in the oblique direction is blurred compared to the edge in the vertical direction or the horizontal direction.

  Therefore, Non-Patent Document 1 discloses a method of calculating a G signal at a position of an R or B pixel. In Non-Patent Document 1, color difference signals are calculated by respectively calculating color difference signals of G pixels and pixels of the same color as the target pixel in the upper, lower, left, and right directions including the target pixel, and combining color difference signals in each direction. A G signal is calculated by adding to the target pixel. At this time, in Non-Patent Document 1, a color difference signal in each direction is synthesized according to the inclination of the color difference signal in the corresponding direction, thereby generating an image with high resolution regardless of the direction.

Patent No. 3862506

I. Pekkucuksen, Y. Altunbasak, "Gradient based threshold free color filter array interpolation", ICIP 2010

  However, in Non-Patent Document 1, since the G signal is generated using the R and B signals, when the R and B signals deviate from the G signal due to the influence of optical aliasing and chromatic aberration. In some cases, a block-like false pattern may occur on a diagonal linear object.

  An object of the present invention is, in view of the above-mentioned problems, to be able to generate an image in which a block-shaped false pattern for a diagonal linear object is reduced.

  The image processing apparatus according to the present invention comprises: first color difference signal calculation means for calculating a color difference signal in the vertical direction of an input image signal; and second color difference signal calculation means for calculating a color difference signal in the horizontal direction of the input image signal. A first filter processing unit that performs filter processing on the color difference signal in the vertical direction, a second filter processing unit that performs filter processing on the color difference signal in the horizontal direction, and focusing on the input image signal First degree calculating means for calculating a first degree indicating whether the pixel is an area affected by predetermined chromatic aberration or optical aliasing, and the first filtering means and the second degree The filter processing means is characterized in that the characteristics of the filter processing are changed based on the calculation result of the first degree calculation means.

  According to the present invention, it is possible to generate an image in which a block-like false pattern for an oblique linear object is reduced.

It is a figure which shows 1 unit of a primary color Bayer arrangement. It is a block diagram showing an example of composition of a luminosity signal generation part concerning an embodiment. It is a block diagram showing an example of composition of G interpolation circuit of a 1st embodiment. It is a flowchart which shows an example of the flow of a process of G interpolation circuit in 1st Embodiment. It is a block diagram showing an example of composition of a texture detection circuit. It is a flowchart which shows an example of the flow of a process of a texture detection circuit. It is a figure for demonstrating an example of the signal pattern of a to-be-photographed object. It is a figure for demonstrating the relationship between an edge intensity signal and the texture degree (alpha). It is a block diagram which shows the structural example of G interpolation circuit of 2nd Embodiment. It is a block diagram which shows the structural example of the 1st G interpolation circuit of 2nd Embodiment. It is a block diagram which shows the structural example of the 2nd G interpolation circuit of 2nd Embodiment. It is a flowchart which shows an example of the flow of a process of the 2nd G interpolation circuit in 2nd Embodiment. It is a block diagram which shows the structural example of G interpolation circuit of 3rd Embodiment. It is a flowchart which shows an example of the flow of a process of G interpolation circuit in 3rd Embodiment. It is a block diagram showing an example of composition of HV edge detection circuit. It is a figure for demonstrating the relationship between HV edge correlation value and HV edge degree (beta).

First Embodiment
FIG. 1 is a view showing an imaging device having a Bayer-arranged color filter according to a first embodiment of the present invention. The imaging device has an imaging element such as a CMOS image sensor. The imaging device is applicable to a digital camera, a video camera, a smartphone, a tablet, an industrial camera, a medical camera, and the like. The imaging device has a plurality of pixels arranged in a two-dimensional matrix, and each of the plurality of pixels is a color filter (for example, red (R), green (G), blue (B) of three colors) One color filter). FIG. 1 shows a 1-unit color filter of a primary color Bayer arrangement. The image sensor performs photoelectric conversion for each pixel, performs analog-digital conversion, and outputs digital R signal (red signal), G1 signal (green signal), G2 signal (green signal), or B signal (blue signal). Do. The G1 signal and the G2 signal are G signals (green signals). Since an imaging device having a Bayer array color filter can obtain only one of R, G, and B color signals in each pixel, when obtaining color signals of all RGB in each pixel, the image processing apparatus shown in FIG. The interpolation processing needs to be performed by the luminance signal generation unit 200 of

  FIG. 2 is a diagram showing a configuration example of the luminance signal generation unit 200 according to the present embodiment. The luminance signal generation unit 200 is an image processing apparatus, and receives the digital R signal, G signal and B signal of Bayer arrangement from the imaging device of FIG. The luminance signal generation unit 200 includes a WB circuit (white balance circuit) 201, a G interpolation circuit 202, an R interpolation circuit 203, a B interpolation circuit 204, an APC circuit 205, a luminance signal generation circuit 206, and an addition circuit 207. And.

  The WB circuit 201 receives digital image signals (R signal, G signal and B signal) of Bayer arrangement from the imaging device and corrects the white balance of the image signal. The G interpolation circuit 202 receives the image signal of Bayer array output from the WB circuit 201, calculates G signals at the positions of R pixels and B pixels in FIG. 1 by interpolation, and outputs G signals of all the pixels. The R pixel is a pixel provided with a red filter, the B pixel is a pixel provided with a blue filter, and the G pixel is a pixel provided with a green filter. Details of the processing of the G interpolation circuit 202 will be described later.

  The R interpolation circuit 203 interpolates the R signals of the positions of the G pixel and the B pixel in the input image of FIG. 1 by interpolation with respect to the input image signal of the Bayer array output by the WB circuit 201, and calculates R signals of all pixels. Output. For example, the R interpolation circuit 203 calculates an R signal by performing two-dimensional LPF (low-pass filter) processing after setting the signal level other than the R pixel to 0 with respect to the input image signal of Bayer array output by the WB circuit 201 Do.

  The B interpolation circuit 204 interpolates the B signals of the positions of the R pixel and the G pixel in the input image of FIG. 1 with respect to the input image signal of the Bayer array output by the WB circuit 201, and calculates B signals of all pixels. Output. For example, the B interpolation circuit 204 sets the signal level of the pixels other than the B pixel to 0 with respect to the input image signal of the Bayer array output from the WB circuit 201, and then calculates the B signal by two-dimensional LPF processing.

The APC circuit 205 generates an aperture correction signal by applying an HPF (high pass filter) or the like to the G signal output from the G interpolation circuit 202. The addition circuit 207 adds the G signal output from the G interpolation circuit 202 and the output signal of the APC circuit 205, and outputs G signals of all pixels. Based on the G signal output from the addition circuit 207, the R signal output from the R interpolation circuit 203, and the B signal output from the B interpolation circuit 204, the luminance signal generation circuit 206 performs all operations according to the following equation (1). A luminance signal Y of the pixel is generated.
Y = 0.3R + 0.59G + 0.11B (1)

  FIG. 3 is a block diagram showing a configuration example of the G interpolation circuit 202 of FIG. G interpolation circuit 202 includes G pixel V interpolation circuit 301, G pixel H interpolation circuit 302, R, B pixel V interpolation circuit 303, R, B pixel H interpolation circuit 304, V color difference calculation circuit 305, H And a color difference calculation circuit 306. Furthermore, the G interpolation circuit 202 includes a V color difference inclination calculation circuit 307, an H color difference inclination calculation circuit 308, a texture detection circuit 309, an N filter circuit 310, an S filter circuit 311, a W filter circuit 312, and an E filter. And a circuit 313. Furthermore, the G interpolation circuit 202 includes an N weight calculation circuit 314, an S weight calculation circuit 315, a W weight calculation circuit 316, an E weight calculation circuit 317, a synthesis circuit 318, and an addition circuit 319.

FIG. 4 is a flowchart showing the flow of the image processing method of the G interpolation circuit 202. In step S401, the G pixel V interpolation circuit 301 performs G interpolation processing in the vertical direction on the input image signal of the Bayer array output by the WB circuit 201, and outputs G signals of all pixels. Do. Specifically, the G pixel V interpolation circuit 301 outputs the G signal as it is when the pixel of interest is a G pixel, and vertically interpolates the G signal when the pixel of interest is an R pixel or a B pixel. The G signal is calculated by performing the processing. For example, when the G pixel V interpolation circuit 301 sets the X and Y coordinates of the pixel of interest in the image to (j, i), and the pixel of interest is an R pixel, the G signal Gv _{i, j} is expressed by the following equation (2) Calculated by Further, even when the pixel of interest is a B pixel, the G pixel V interpolation circuit 301 calculates the G signal in the same manner as in the case where the pixel of interest is an R pixel.
Gvi _{, j} = (Gi _{-1, j} + Gi _{+ 1, j} ) / 2 (2)

Next, in step S402, the G pixel H interpolation circuit 302 calculates the G signal by performing interpolation processing in the horizontal direction on the input image signal output from the WB circuit 201, and outputs the G signals of all the pixels. Do. Specifically, the G pixel H interpolation circuit 302 outputs the G signal as it is when the pixel of interest is a G pixel, and interpolates the G signal in the horizontal direction when the pixel of interest is an R pixel or B pixel. The G signal is calculated by performing the processing. For example, when the target pixel is an R pixel, the G pixel H interpolation circuit 302 calculates the G signal Ghi _{, j according} to the following expression (3). The G pixel H interpolation circuit 302 also calculates a G signal in the same manner as in the case where the target pixel is an R pixel, even when the target pixel is a B pixel.
Gh _{i, j} = (G _{i, j-1} + G _{i, j + 1} ) / 2 (3)

Next, in step S403, the R, B pixel V interpolation circuit 303 performs interpolation processing in the vertical direction on the input image signal output from the WB circuit 201 to calculate the R signal and the B signal. Output an R signal and a B signal. Specifically, when the target pixel is an R pixel or a B pixel, the R, B pixel V interpolation circuit 303 outputs the R signal or the B signal as it is. The R, B pixel V interpolation circuit 303 calculates an R signal by performing interpolation processing in the vertical direction when the target pixel is a G pixel and the vertical direction of the target pixel is an R pixel, and When the vertical direction is B pixels, the B signal is calculated by performing interpolation processing in the vertical direction. For example, when the target pixel is a G pixel and the vertical direction of the target pixel is an R pixel, the R, B pixel V interpolation circuit 303 calculates an R signal Rv _{i, j according} to the following expression (4). The R, B pixel V interpolation circuit 303 also calculates the B signal by the same method as in the case where the vertical direction of the target pixel is R pixel, even when the vertical direction of the target pixel is B pixel.
Rv _{i, j} = (R _{i -1, j} + R _{i +1, j} ) / 2 (4)

Next, in step S404, the R, B pixel H interpolation circuit 304 performs interpolation processing in the horizontal direction on the input image signal output from the WB circuit 201 to calculate the R signal and the B signal. Output an R signal and a B signal. Specifically, when the target pixel is an R pixel or a B pixel, the R, B pixel H interpolation circuit 304 outputs the R signal or the B signal as it is. The R and B pixels H interpolation circuit 304 calculates an R signal by performing interpolation processing in the horizontal direction when the target pixel is a G pixel and the horizontal direction of the target pixel is an R pixel, and When the horizontal direction is B pixels, the B signal is calculated by performing interpolation processing in the horizontal direction. For example, when the target pixel is a G pixel and the horizontal direction of the target pixel is an R pixel, the R and B pixels H interpolation circuit 304 calculates the R signal Rh _{i, j according} to the following expression (5). The R, B pixel H interpolation circuit 304 also calculates the B signal in the same manner as in the case where the horizontal direction of the target pixel is R pixel even when the horizontal direction of the target pixel is B pixel.
Rh _{i, j} = (R _{i, j-1} + R _{i, j + 1} ) / 2 (5)

  In steps S401 to S404, as interpolation methods for G pixels and R and B pixels in the vertical and horizontal directions, interpolation is performed from the adjacent pixels of the pixel of interest using Equations (2) to (5). There is no limitation, and interpolation may be performed using pixels other than adjacent pixels.

Next, in step S405, the V color difference calculation circuit 305 subtracts the R signal or B signal output from the R, B pixel V interpolation circuit 303 from the G signal output from the G pixel V interpolation circuit 301. The color difference signal in the vertical direction of the input image signal is calculated. The V color difference calculation circuit 305 is a first color difference signal calculation unit. In the V color difference calculation circuit 305, in the pixel of interest (j, i), the signal output from the G pixel V interpolation circuit 301 is the G signal Gvi _{, j} , and the signal output from the R and B pixels V interpolation circuit 303 is R In the case of the signal Rv _{i, j} , the color difference signal Diff_v in the vertical direction is calculated by the following equation (6). Also, the V color difference calculation circuit 305 uses the same method as in the case where the signal output from the R, B pixel V interpolation circuit 303 is the R signal, even when the signal output from the R, B pixel V interpolation circuit 303 is the B signal. , G signals and B signals are calculated.
Diff_v _{i, j} = Gv _{i, j} -Rv _{i, j} (6)

Next, in step S406, the H color difference calculation circuit 306 subtracts the R signal or B signal output from the R, B pixel H interpolation circuit 304 from the G signal output from the G pixel H interpolation circuit 302. The color difference signal in the horizontal direction of the input image signal is calculated. The H color difference calculation circuit 306 is a second color difference signal calculation unit. In the H color difference calculation circuit 306, in the pixel of interest (j, i), the signal output from the G pixel H interpolation circuit 302 is the G signal Ghi _{, j} , and the signal output from the R and B pixels H interpolation circuit 304 is R In the case of the signal Rh _{i, j} , the color difference signal Diff_h in the horizontal direction is calculated by the following equation (7). Also, the H color difference calculation circuit 306 uses the same method as in the case where the signal output by the R, B pixel H interpolation circuit 304 is the R signal, even when the signal output by the R, B pixel H interpolation circuit 304 is the B signal. , G signals and B signals are calculated.
_{Diff_h i, j = Gh i,} j -Rh i, j ··· (7)

Next, in step S407, the V color difference inclination calculation circuit 307 calculates the inclination of the color difference in the vertical direction based on the color difference signal output from the V color difference calculation circuit 305. Specifically, the V color difference inclination calculation circuit 307 calculates the inclination signal Grad_v of the color difference in the vertical direction based on the color difference signal Diff_v output from the V color difference calculation circuit 305 in the pixel of interest (j, i) Calculated by 8).
Grad_v _{i, j} = | Diff_v _{i−1, j} −Diff_v _{i + 1, j} | (8)

Next, in step S408, the H color difference inclination calculation circuit 308 calculates the inclination of the color difference in the horizontal direction based on the color difference signal output from the H color difference calculation circuit 306. Specifically, based on the color difference signal Diff_h output from the H color difference calculation circuit 306, the H color difference inclination calculation circuit 308 calculates the color difference inclination signal Grad_h in the horizontal direction based on the color difference signal Diff_h in the pixel of interest (j, i) 9) Calculated.
Grad_hi _{, j} = | Diff_hi _{, j-1-} Diff_hi _{, j + 1} (9)

  Next, in step S409, the texture detection circuit 309 generates a block-shaped false pattern on an oblique linear object due to the influence of optical aliasing or chromatic aberration on the input image signal output from the WB circuit 201. To detect whether or not The texture detection circuit 309 is a first degree calculating unit. Although the details of the processing of the texture detection circuit 309 will be described later, the texture detection circuit 309 outputs the texture degree α between 0.0 and 1.0 according to the degree of the texture. The texture degree α is output as 0.0, for example, when it is determined that a block-like fake pattern does not occur in a diagonal linear subject, and a block-like fake pattern is output in the diagonal linear subject If it is determined that is occurring, 1.0 is output.

  Next, in step S410, the N filter circuit 310 performs upward filter processing on the color difference signal output from the V color difference calculation circuit 305 based on the texture degree α output from the texture detection circuit 309. Specifically, first, using the color difference signal Diff_v output from the V color difference calculation circuit 305, the N filter circuit 310 uses the color difference signal Diff_v to calculate the result Fil_n of the upward filter processing of the pixel of interest (j, i) Calculated by).

Next, the N filter circuit 310 calculates and outputs the color difference signal Diff_n according to the following equation (12) based on the texture degree α.
Diff_n _{i, j} = α _{i, j} × Diff_v <sub>i, j
+ (1.0-α i, j</sub> ) × Fil_n i, j ··· (12)

  Next, in step S411, the S filter circuit 311 performs downward filter processing on the color difference signal output from the V color difference calculation circuit 305 based on the texture degree α output from the texture detection circuit 309. Specifically, first, using the color difference signal Diff_v output from the V color difference calculation circuit 305, the S filter circuit 311 uses the color difference signal Diff_v to calculate the result Fil_s of the downward filter processing at the pixel of interest (j, i) according to Calculated by).

Next, the S filter circuit 311 calculates and outputs the color difference signal Diff_s according to the following equation (14) based on the texture degree α.
Diff_s _{i, j} = α _{i, j} × Diff_v _{i, j}
+ (1.0-alpha _{i, j} ) x Fil s _{i, j} (14)

  Next, in step S412, the W filter circuit 312 performs leftward filtering processing on the color difference signal output from the H color difference calculation circuit 306 based on the texture degree α output from the texture detection circuit 309. Specifically, first, using the color difference signal Diff_h output from the H color difference calculation circuit 306, the W filter circuit 312 uses the color difference signal Diff_h to filter the result Fil_w in the left direction of the pixel of interest (j, i) by Calculated by).

Next, the W filter circuit 312 calculates and outputs the color difference signal Diff_W by the following equation (16) based on the texture degree α.
Diff_wi _{, j} = αi _{, j} × Diff_hi _{, j}
+ (1.0-α _{i, j} ) × Fil_w _{i, j} (16)

  Next, in step S413, the E filter circuit 313 performs rightward filtering processing on the color difference signal output from the H color difference calculation circuit 306 based on the texture degree α output from the texture detection circuit 309. Specifically, first, the E filter circuit 313 uses the color difference signal Diff_h output from the H color difference calculation circuit 306 to calculate the result Fil_e of the filter processing in the right direction at the pixel of interest (j, i) according to the following expression (17 Calculated by).

Next, the E filter circuit 313 calculates and outputs the color difference signal Diff_e by the following equation (18) based on the texture degree α.
Diff_e _{i, j} = α _{i, j} × Diff_h _{i, j}
+ (1.0-alpha _{i, j} ) x Fil_ei _{, j} (18)

  Next, in step S414, the N weight calculation circuit 314 calculates the weight Wn in the upward direction according to the following equation (19) based on the color difference inclination signal Grad_v in the vertical direction output from the V color difference inclination calculation circuit 307. .

  Next, in step S415, the S weight calculation circuit 315 calculates the downward weight Ws by the following equation (20) based on the color difference inclination signal Grad_v in the vertical direction output from the V color difference inclination calculation circuit 307. .

  Next, in step S416, the W weight calculation circuit 316 calculates the weight Ww in the left direction according to the following equation (21) based on the horizontal color difference inclination signal Grad_h output from the H color difference inclination calculation circuit 308. .

  Next, in step S417, the E weight calculation circuit 317 calculates the weight We in the right direction according to the following equation (22) based on the horizontal color difference inclination signal Grad_h output from the H color difference inclination calculation circuit 308. .

Next, in step S418, the combining circuit 318 sets the weight Wn input from the N weight calculation circuit 314, the weight Ws input from the S weight calculation circuit 315, the weight Ww input from the W weight calculation circuit 316, and E The weight We input from the weight calculation circuit 317 is input. The synthesis circuit 318 also receives the color difference signal Diff_n input from the N filter circuit 310 and the color difference signal Diff_s input from the S filter circuit 311. The synthesis circuit 318 also receives the color difference signal Diff_w input from the W filter circuit 312 and the color difference signal Diff_e input from the E filter circuit 313. The combining circuit 318 combines the color difference signal Diff_n, the color difference signal Diff_s, the color difference signal Diff_w, and the color difference signal Diff_e based on the weight Wn, the weight Ws, the weight Ww, and the weight We, and sets the color difference signal Diff_mix Calculated by 23).
Diff_mix _{i, j} = (Wn _{i, j} × Diff_n _{i, j} + Ws _{i, j} × Diff_s _{i, j}
+ Ww _{i, j} × Diff_w _{i, j} + We _{i, j} × Diff_e _{i, j} ) / Wt _{i, j}
Wt _{i, j} = Wn _{i, j} + Ws _{i, j} + Ww _{i, j} + We _{i, j}
... (23)

  Next, in step S419, when the target pixel of the input image signal input to the G interpolation circuit 202 is an R pixel or a B pixel, the addition circuit 319 combines the signal of the target pixel in the synthesis circuit 318. The color difference signal Diff_mix is added to calculate and output a G signal. When the pixel of interest is a G pixel, the addition circuit 319 outputs the image signal input to the G interpolation circuit 202 as it is. The output signal of the addition circuit 319 is the output signal of the G interpolation circuit 202.

  As described above, the N filter circuit 310 performs the upward filter processing, the S filter circuit 311 performs the downward filter processing, the W filter circuit 312 performs the left filter processing, and the E filter circuit 313 the right. Filter direction. The N filter circuit 310 and the S filter circuit 311 are first filter processing means, and perform filter processing on the color difference signal in the vertical direction. The W filter circuit 312 and the E filter circuit 313 are second filter processing means and perform filter processing on the color difference signal in the horizontal direction. Then, if the pixel of interest is an R pixel or a B pixel, the color difference signal after filter processing is synthesized by the weight for each direction in the synthesis circuit 318, and then added to the pixel of interest in the addition circuit 319 to obtain a G interpolation circuit. A G signal to be output of 202 is generated.

  FIG. 5 is a block diagram showing a configuration example of the texture detection circuit 309 of FIG. The texture detection circuit 309 has a G pixel 0 insertion circuit 501, an HV interpolation circuit 502, a D45 degree edge detection circuit 503, and a D135 degree edge detection circuit 504. Furthermore, the texture detection circuit 309 has a zero insertion circuit 505 other than G pixels, an HV interpolation circuit 506, an edge direction determination circuit 507, a selector 508, and a coefficient calculation circuit 509.

FIG. 6 is a flowchart showing the flow of the image processing method of the texture detection circuit 309.
Here, in FIG. 7A, when the R and B signals are shifted with respect to the G signal due to the influence of the optical aliasing and the chromatic aberration, a block-shaped false pattern is formed on the diagonal linear object. It is a figure for demonstrating an example of the image signal in case the has generate | occur | produced. On the other hand, FIG. 7D is a view for explaining an example of a diagonal line-shaped subject in which no block-shaped false pattern is generated. In step S601, the G pixel 0 insertion circuit 501 sets the signal level of the G pixel of the image signal of the Bayer array output by the WB circuit 201 to 0.

  Next, in step S602, the HV interpolation circuit 502 calculates R and B signals by performing interpolation processing in the horizontal direction and vertical direction on the image signal output from the G pixel 0 insertion circuit 501. At this time, the HV interpolation circuit 502 uses, for example, a filter with a coefficient of (1, 2, 1) / 2 for interpolation processing in the horizontal direction and the vertical direction. When a block-shaped false pattern is generated on a diagonal linear object, the image signal output from the HV interpolation circuit 502 is a pattern as shown in FIG. 7C. On the other hand, when a block-shaped false pattern is not generated in the diagonal linear object, the image signal output from the HV interpolation circuit 502 becomes a pattern as shown in FIG. 7 (f).

  In step S603, the D45 degree edge detection circuit 503 performs edge detection in the 45 ° oblique direction (upper right direction) on the image signal output from the HV interpolation circuit 502, and outputs an edge intensity signal. Specifically, for example, the filter processing of the coefficient (−1, 2, −1) is performed in the 45 ° diagonal direction, and the absolute value of the output signal of the filter is output as an edge strength signal. Next, in step S604, the D135 degree edge detection circuit 504 performs edge detection in the diagonal 135 degree direction (downward right direction) on the image signal output from the HV interpolation circuit 502, and outputs an edge intensity signal. Specifically, for example, the filter processing of the coefficient (−1, 2, −1) is performed in the diagonal 135 degree direction, and the absolute value of the output signal of the filter is calculated as the edge strength signal.

  In step S605, the non-G pixel zero insertion circuit 505 sets the signal levels of the R pixel and the B pixel of the image signal of the Bayer array output by the WB circuit 201 to zero. Next, in step S606, the HV interpolation circuit 506 calculates the G signal by performing interpolation processing in the horizontal direction and the vertical direction on the image signal output from the zero insertion circuit 505 other than G pixels. At this time, the HV interpolation circuit 506 uses, for example, a filter with a coefficient of (1, 2, 1) / 2 for interpolation processing in the horizontal direction and the vertical direction. When a block-shaped false pattern is generated on a diagonal linear object, the image signal output from the HV interpolation circuit 506 is a pattern as shown in FIG. 7B. On the other hand, when a block-shaped false pattern is not generated in the diagonal linear object, the image signal output from the HV interpolation circuit 506 is a pattern as shown in FIG. 7 (e). That is, the same interpolation result is obtained regardless of the presence or absence of the block-shaped false pattern.

  Next, in step S 607, the edge direction determination circuit 507 determines whether the edge is in the 45 ° oblique direction or the 135 ° oblique direction for each pixel with respect to the image signal output by the HV interpolation circuit 506. . The edge direction determination circuit 507 is an edge direction determination unit. Specifically, for the pixel of interest, for example, a filter process of (-1, 2, -1) is performed in a 45-degree oblique direction and a 135-degree oblique direction, and the magnitudes of the output signal are compared To determine the direction of the edge. If the edge direction determination circuit 507 determines that the edge is in the 45-degree oblique direction, the process proceeds to step S608. If it is determined that the edge is in the 135-degree oblique direction, the process proceeds to step S609.

  For example, when the image signal output from the HV interpolation circuit 506 has a pattern shown in FIG. 7B or FIG. 7E and the target pixel is the central pixel, there is a linear subject in the 135 ° diagonal direction. . Therefore, the absolute value of the image signal subjected to the filtering process in the 45-degree oblique direction becomes a large value according to the amplitude of the edge. Further, the absolute value of the image signal subjected to the filter processing in the diagonal 135 degree direction is a small value because there is no edge. Thus, in the case of the pattern shown in FIG. 7B or FIG. 7E by comparing the absolute values of the image signals after filter processing, there is an edge in the 45 ° oblique direction with respect to the pixel of interest Can be determined.

  In step S608, the selector 508 selects the edge strength signal output from the D135 degree edge detection circuit 504 in the diagonal direction from the determination result of the edge direction determination circuit 507. On the other hand, in step S609, the selector 508 selects the edge strength signal output from the D45 degree edge detection circuit 503 in the diagonal direction from the determination result of the edge direction determination circuit 507. Thus, the selector 508 is an edge strength calculation unit. For example, when the image signal output from the HV interpolation circuit 506 is the pattern shown in FIG. 7B or 7E and the pixel of interest is the central pixel, the pixel of interest has an edge in the 45 ° oblique direction It is determined that In this case, the selector 508 selects the output signal of the D135 degree edge detection circuit 504.

  In the pattern shown in FIG. 7C, when a block-shaped false pattern is generated on a diagonal linear object, the edge strength signal output from the D135 degree edge detection circuit 504 is 135 degrees oblique. There are edges in the direction. For this reason, the edge strength signal output from the D135 degree edge detection circuit 504 has a large value according to the amplitude of the edge. Further, in the pattern shown in FIG. 7F, when no block-shaped false pattern is generated in the diagonal linear object, the edge strength signal output from the D135 degree edge detection circuit 504 is diagonal. There is no edge in the 135 degree direction. Therefore, the edge strength signal output from the D135 degree edge detection circuit 504 has a small value. As described above, when a block-shaped false pattern is generated in the diagonal linear object, the edge strength signal output from the selector 508 has a large value. On the other hand, when the block-shaped false pattern is not generated in the diagonal linear object, the edge strength signal output from the selector 508 has a small value.

  In step S610, the coefficient calculation circuit 509 generates a block-shaped false pattern on the diagonal linear object due to the influence of optical aliasing or chromatic aberration based on the edge intensity signal output from the selector 508. The degree of texture .alpha.

  FIG. 8 is a diagram showing an example of a conversion table used when the coefficient calculation circuit 509 calculates the texture degree α from the edge intensity signal output from the selector 508. In FIG. 8, the horizontal axis indicates the edge strength signal, and the vertical axis indicates the texture degree α. The coefficient calculation circuit 509 considers that a block-shaped false pattern is not generated in the diagonal linear object when the edge strength signal which is the calculation result is equal to or less than the first threshold Th1 set in advance. , Output 0.0 as the texture degree α. In addition, when the edge strength signal which is the calculation result is equal to or larger than the second threshold value Th2, the coefficient calculation circuit 509 considers that a block-shaped false pattern is generated on the diagonal linear object, and the texture Output 1.0 as the degree α. Here, the second threshold Th2 is larger than the first threshold Th1. When the edge strength signal which is the calculation result is larger than the first threshold Th1 and the edge strength signal is smaller than the second threshold Th2, the coefficient calculation circuit 509 is linear according to the magnitude of the edge strength signal. The texture degree α between 0.0 and 1.0 is calculated.

  Here, when the R and B signals are shifted with respect to the G signal due to the influence of optical aliasing and chromatic aberration, the input image output from the WB circuit 201 has a block shape on the oblique linear object. False patterns occur. Then, since the G signal generated by the G interpolation circuit 202 is also generated using the R or B signal of the R pixel or the B pixel, a block-shaped false pattern is generated on the diagonal linear object as in the input image. Image quality is degraded.

  In the present embodiment, the texture detection circuit 309 is provided in order to prevent the occurrence of a block-shaped false pattern and a reduction in image quality. The texture detection circuit 309 can calculate an edge strength signal to calculate the texture degree α, and can detect a block-shaped false pattern on a diagonal linear object. When a block-shaped false pattern is detected in a diagonal linear object, the texture degree α has a large value, and the filter circuits 310 to 313 have a high ratio at which the color difference signal not subjected to the filtering process is output. Do.

  As shown in FIG. 8, when the edge strength signal is smaller than the second threshold Th2, the texture degree α becomes smaller than 1.0, and the filter circuits 310 to 313 perform the filtering process. When the edge strength signal is equal to or greater than the second threshold value Th2, the texture degree α is 1.0, and the filter circuits 310 to 313 do not perform the filtering process.

  That is, when the block-shaped false pattern is generated on the diagonal linear object (when the edge strength signal is equal to or higher than the second threshold), the texture degree α becomes 1.0, and the filter circuit 310 to 313 does not perform filter processing (change the characteristics of the filter processing). When a block-shaped false pattern is generated on a diagonal linear object and the pixel of interest is an R pixel or a B pixel, the color difference signals output from the filter circuits 310 to 313 have the equations (2) to ( From 7), it becomes the difference between the G pixel on the right and left or above and below the pixel of interest and the pixel of interest. Then, when it is added to the pixel of interest by the addition circuit 319, the result is a value calculated from the left and right or upper and lower G signals. As a result, since the G signal can be generated without using the R signal and the B signal, even if the R and B signals deviate with respect to the G signal due to the influence of the optical aliasing and the chromatic aberration, it is oblique. It is possible to prevent the occurrence of block-shaped false patterns for linear objects. Further, when the texture degree α is 0.0 to 1.0, block-like false with respect to a diagonal line-like subject is obtained by changing the intensity of the color difference signal not subjected to the filtering process according to the intensity. It is possible to reduce the occurrence of patterns and improve the image quality.

  As described above, according to the present embodiment, when generating the G signal and the luminance signal from the image signal of the Bayer array, an image having a high sense of resolution is generated regardless of the direction, and an oblique linear It is possible to obtain an image in which a blocky false pattern for an object is reduced.

Second Embodiment
Hereinafter, a second embodiment of the present invention will be described. The configuration of the luminance signal generation unit 200 according to the present embodiment is the same as that shown in FIG. In the present embodiment, only portions different from the first embodiment will be described.

  FIG. 9 is a block diagram showing a configuration example of the G interpolation circuit 202 according to the second embodiment of the present invention. The G interpolation circuit 202 includes a first G interpolation circuit 901, a second G interpolation circuit 902, a texture detection circuit 903 and a synthesis circuit 904. The first G interpolation circuit 901 is a first interpolation unit, which interpolates the G signal without using a color difference signal with respect to the image signal, and outputs a first G signal. The second G interpolation circuit 902 is a second interpolation unit, which interpolates the G signal with respect to the image signal using the color difference signal, and outputs a second G signal. The synthesizing circuit 904 is a synthesizing means, and based on the texture degree α outputted from the texture detecting circuit 903, the first G signal interpolated by the first G interpolating circuit 901 and the second G interpolating circuit 902 interpolate. And the synthesized second G signal. The details will be described later. The texture detection circuit 903 is a circuit similar to the texture detection circuit 309 of FIG. 5 of the first embodiment.

  FIG. 10 is a block diagram showing a configuration example of the first G interpolation circuit 901. As shown in FIG. The first G interpolation circuit 901 has a 0 insertion circuit 1001 and an HV interpolation circuit 1002. The zero insertion circuit 1001 sets the signal levels of the R pixel and the B pixel in the input image signal to zero. Next, the HV interpolation circuit 1002 calculates the G signal by performing interpolation processing in the horizontal direction and the vertical direction on the image signal output from the zero insertion circuit 1001. At this time, the HV interpolation circuit 1002 uses, for example, a filter with a coefficient of (1, 2, 1) / 2 for interpolation processing in the horizontal direction and the vertical direction.

  FIG. 11 is a block diagram showing a configuration example of the second G interpolation circuit 902. The second G interpolation circuit 902 includes a G pixel V interpolation circuit 1101, a G pixel H interpolation circuit 1102, an R, B pixel V interpolation circuit 1103, an R, B pixel H interpolation circuit 1104, and a V color difference calculation circuit 1105. And an H color difference calculation circuit 1106. Furthermore, the second G interpolation circuit 902 includes a V color difference inclination calculation circuit 1107, an H color difference inclination calculation circuit 1108, an N filter circuit 1109, an S filter circuit 1110, a W filter circuit 1111, and an E filter circuit 1112. Have. Furthermore, the second G interpolation circuit 902 includes an N weight calculation circuit 1113, an S weight calculation circuit 1114, a W weight calculation circuit 1115, an E weight calculation circuit 1116, a synthesis circuit 1117, and an addition circuit 1118. .

FIG. 12 is a flowchart showing the process flow of the second G interpolation circuit 902. In step S1201, the G pixel V interpolation circuit 1101 generates a G signal by performing interpolation processing on the input image signal in the vertical direction. Specifically, when the pixel of interest is a G pixel, the G pixel V interpolation circuit 1101 outputs the signal of the G pixel as it is as a G signal, and when the pixel of interest is an R pixel or a B pixel, the vertical direction The G signal is generated by performing interpolation processing in For example, assuming that the X and Y coordinates of the pixel of interest in the image are (j, i), the G pixel V interpolation circuit 1101 generates the G signal Gv _{i, j} by the following equation (24) when the pixel of interest is R pixels. Generate by
Gvi _{, j} = (Gi _{-1, j} + Gi _{+ 1, j} ) / 2 + (2 x _{Ri, j-} Ri _{-2, j} + _{Ri + 2, j} ) / 4
... (24)

  The G pixel V interpolation circuit 1101 generates a G signal in the same manner as described above even when the pixel of interest is a B pixel.

Next, in step S1202, the G pixel H interpolation circuit 1102 performs interpolation processing on the input image signal in the horizontal direction to generate a G signal. Specifically, when the pixel of interest is a G pixel, the G pixel H interpolation circuit 1102 outputs the signal of the G pixel as it is as a G signal, and when the pixel of interest is an R pixel or B pixel, the horizontal direction The G signal is generated by performing interpolation processing in For example, when the target pixel is an R pixel, the G pixel H interpolation circuit 1102 generates a G signal Ghi _{, j according} to the following equation (25).
Gh _{i, j} = (G _{i, j-1} + G _{i, j + 1} ) / 2 + (2 × R _{i, j} -R _{i, j} -2 + R _{i, j +2} ) / 4
... (25)

  The G pixel H interpolation circuit 1102 generates a G signal in the same manner as described above even when the pixel of interest is a B pixel.

Next, in step S1203, the R, B pixel V interpolation circuit 1103 performs an interpolation process on the input image signal in the vertical direction to generate an R signal or a B signal. Specifically, when the target pixel is an R pixel or a B pixel, the R, B pixel V interpolation circuit 1103 outputs the signal of the R pixel or the B pixel as it is as an R signal or a B signal. The R, B pixel V interpolation circuit 1103 generates an R signal by performing interpolation processing in the vertical direction when the pixel of interest is the R pixel when the pixel of interest is the G pixel, and similarly, When the vertical direction of the target pixel is a B pixel, the B signal is generated by performing interpolation processing in the vertical direction. For example, when the target pixel is a G pixel and the vertical direction of the target pixel is an R pixel, the R, B pixel V interpolation circuit 1103 generates an R signal Rv _{i, j according} to the following equation (26).
Rv _{i, j} = (R _{i -1, j} + R _{i + 1, j} ) / 2 + (2 × G _{i, j} -G _{i-2, j} + G _{i + 2, j} ) / 4
... (26)

  The R, B pixel V interpolation circuit 1103 generates a B signal in the same manner as described above when the vertical direction of the target pixel is a B pixel.

Next, in step S1204, the R, B pixel H interpolation circuit 1104 performs interpolation processing on the input image signal in the horizontal direction to generate an R signal or B signal. Specifically, when the target pixel is an R pixel or a B pixel, the R, B pixel H interpolation circuit 1104 outputs the signal of the R pixel or the B pixel as it is as an R signal or a B signal. The R, B pixel H interpolation circuit 1104 generates an R signal by performing interpolation processing in the horizontal direction when the target pixel is a G pixel and the horizontal direction of the target pixel is an R pixel, and similarly, When the horizontal direction of the pixel of interest is B pixels, the B signal is generated by performing interpolation processing in the horizontal direction. For example, when the target pixel is a G pixel and the horizontal direction of the target pixel is an R pixel, the R, B pixel H interpolation circuit 1104 generates an R signal Rh _{i, j according} to the following expression (27).
Rh _{i, j} = (R _{i, j-1} + R _{i, j + 1} ) / 2 + (2 × G _{i, j} -G _{i, j} -2 + G _{i, j +2} ) / 4
... (27)

  The R and B pixels H interpolation circuit 1104 generates a B signal in the same manner as described above when the horizontal direction of the target pixel is a B pixel. Moreover, although Formula (24)-(27) was used as an interpolation method of step S1201-S1204, it is not limited to this, For example, it uses Formula (2)-(5) like 1st Embodiment. The same color pixels may be linearly interpolated in each direction.

  Next, in step S1205, the V color difference calculation circuit 1105 subtracts the R signal or B signal output from the R, B pixel V interpolation circuit 1103 from the G signal output from the G pixel V interpolation circuit 1101, and Generate a color difference signal. For example, the V color difference calculation circuit 1105 calculates the color difference signal Diff_v in the vertical direction by the equation (6), as in the V color difference calculation circuit 305 of FIG.

  Next, in step S1206, the H color difference calculation circuit 1106 subtracts the R signal or B signal output from the R, B pixel H interpolation circuit 1104 from the G signal output from the G pixel H interpolation circuit 1102, Generate a color difference signal. For example, the H color difference calculation circuit 1106 calculates the color difference signal Diff_h in the horizontal direction according to equation (7), as in the H color difference calculation circuit 306 of FIG.

  Next, in step S1207, the V color difference inclination calculation circuit 1107 calculates the inclination of the color difference in the vertical direction based on the color difference signal output from the V color difference calculation circuit 1105. Specifically, the V color difference inclination calculation circuit 1107 calculates an inclination signal Grad_v of the color difference in the vertical direction according to equation (8), as in the V color difference inclination calculation circuit 307 of FIG.

  The equation for calculating the slope signal Grad_v is not limited to the equation (8). For example, the V color difference inclination calculation circuit 1107 may calculate the inclination signal Grad_v based on the difference between the pixel of interest and the upper and lower adjacent pixels. The V color difference inclination calculation circuit 1107 outputs the calculated color difference inclination signal Grad_v in the vertical direction to the N weight calculation circuit 1113 and the S weight calculation circuit 1114, respectively.

  Next, in step S1208, the H color difference inclination calculation circuit 1108 calculates the inclination of the color difference in the horizontal direction based on the color difference signal output from the H color difference calculation circuit 1106. Specifically, the H color difference inclination calculating circuit 1108 calculates the color difference inclination signal Grad_h in the horizontal direction by the equation (9), as in the H color difference inclination calculating circuit 308 of FIG.

  The equation for calculating the inclination signal Grad_h is not limited to the equation (9). The H color difference inclination calculation circuit 1108 may calculate the inclination signal Grad_h, for example, based on the difference between the pixel of interest and the adjacent pixels on the left and right. The H color difference inclination calculation circuit 1108 outputs the calculated color difference inclination signal Grad_h in the horizontal direction to the W weight calculation circuit 1115 and the E weight calculation circuit 1116, respectively.

  Next, in step S1209, the N filter circuit 1109 performs upward filter processing on the color difference signal output from the V color difference calculation circuit 1105. Specifically, the N filter circuit 1109 uses the color difference signal Diff_v output from the V color difference calculation circuit 1105 to filter the result Diff_n of the target pixel (j, i) in the upward direction according to the following equation (28) calculate.

  In step S1210, the S filter circuit 1110 performs downward filter processing on the color difference signal output from the V color difference calculation circuit 1105. Specifically, the S filter circuit 1110 uses the color difference signal Diff_v output from the V color difference calculation circuit 1105 to filter the result Diff_s of the target pixel (j, i) in the downward direction according to the following expression (29) calculate.

  In step S1211, the W filter circuit 1111 performs leftward filtering processing on the color difference signal output from the H color difference calculation circuit 1106. Specifically, the W filter circuit 1111 uses the color difference signal Diff_h output from the H color difference calculation circuit 1106 to filter the result Diff_w in the left direction of the pixel of interest (j, i) by the following expression (30) calculate.

  In step S1212, the E filter circuit 1112 performs rightward filtering on the color difference signal output from the H color difference calculation circuit 1106. Specifically, the E filter circuit 1112 uses the color difference signal Diff_h output from the H color difference calculation circuit 1106 to filter the result Diff_e in the right direction in the pixel of interest (j, i) according to the following expression (31) calculate.

  In step S1213, the N weight calculation circuit 1113 calculates the upward weight Wn by the equation (19) as in the N weight calculation circuit 314 of FIG.

  In step S1214, the S weight calculation circuit 1114 calculates the downward weight Ws by equation (20) as in the S weight calculation circuit 315 of FIG.

  In step S1215, the W weight calculation circuit 1115 calculates the weight Ww in the left direction according to equation (21), as in the W weight calculation circuit 316 in FIG.

  In step S1216, the E weight calculation circuit 1116 calculates the weight We in the right direction according to equation (22), as with the E weight calculation circuit 317 in FIG.

  In step S1217, the combining circuit 1117 combines the color difference signals Diff_n, Diff_s, Diff_w, and Diff_e based on the weights Wn, Ws, Ww, and We, as in the combining circuit 318 in FIG. Then, the combining circuit 1117 calculates the color difference signal Diff_mix according to equation (23).

  In step S 1218, the addition circuit 1118 adds the color difference signal Diff_mix synthesized by the synthesis circuit 1117 to the image signal output from the WB circuit 201 if the pixel of interest is an R pixel or a B pixel. Output G signal. The output signal of the addition circuit 1118 is the output signal of the second G interpolation circuit 902. As a result of the above, the second G interpolation circuit 902 can calculate a G signal having a high sense of resolution regardless of the direction with respect to the pixel position of the R pixel or the B pixel.

Next, the combining process of the combining circuit 904 of FIG. 9 will be described. The synthesis circuit 904 synthesizes the first G signal from the first G interpolation circuit 901 and the second G signal from the second G interpolation circuit 902 based on the texture degree α from the texture detection circuit 903. Output the final G signal. The G signal output from the synthesis circuit 904 is an output signal of the G interpolation circuit 202. The synthesis circuit 904 outputs a final G signal G_sig _{i, j} when the X coordinate and the Y coordinate of the target pixel in the image are (j, i). Specifically, the synthesis circuit 904 uses the first G signal G1_sig _{i, j} , the second G signal G2_sig _{i, j} , and the synthesis coefficient α _{i, j} in the target pixel to generate the final G signal G_sig _{i. , j} are calculated by the following equation (32).
G_sig _{i, j} = α _{i, j} × G1_sig _{i, j} + (1.0-α _{i, j} ) × G2_sig _{i, j}
... (32)

  The synthesizing circuit 904 has a high sense of resolution regardless of the direction when the texture degree α is 0.0, that is, when the block-like fake pattern is not generated in the diagonal linear object. The second G signal G2_sig capable of generating an image is output as a final G signal G_sig. Further, the combining circuit 904 generates the first signal only from the G signal when the combining coefficient α is 1.0, that is, when the block-shaped false pattern is generated in the diagonal linear object. G signal G1_sig is output as a final G signal G_sig.

  As described above, according to the present embodiment, even if the R and B signals are shifted with respect to the G signal due to the influence of the optical aliasing and the chromatic aberration, the block shape for the diagonal linear object is obtained. It is possible to prevent the occurrence of false patterns.

Third Embodiment
Hereinafter, a third embodiment of the present invention will be described. In the first and second embodiments described above, aliasing may occur due to sampling of G pixels, and deflection may occur in a region where vertical lines and horizontal lines are switched to oblique lines. Therefore, in the present embodiment, an example will be described in which an image is generated in which not only a block-shaped false pattern for a diagonal linear object but also a deflection is reduced. The luminance signal generation unit 200, which is an image processing apparatus according to the present embodiment, is the same as that shown in FIG. Hereinafter, only differences from the first embodiment will be described.

  FIG. 13 is a block diagram showing a configuration example of the G interpolation circuit 202 of FIG. G interpolation circuit 202 includes G pixel V interpolation circuit 301, G pixel H interpolation circuit 302, R, B pixel V interpolation circuit 303, R, B pixel H interpolation circuit 304, V color difference calculation circuit 305, H And a color difference calculation circuit 306. Further, the G interpolation circuit 202 includes a V color difference inclination calculation circuit 307, an H color difference inclination calculation circuit 308, a texture detection circuit 309, an HV edge detection circuit 1301, and a filter gain calculation circuit 1302. Furthermore, the G interpolation circuit 202 includes an N filter circuit 310, an S filter circuit 311, a W filter circuit 312, and an E filter circuit 313. Furthermore, the G interpolation circuit 202 includes an N weight calculation circuit 314, an S weight calculation circuit 315, a W weight calculation circuit 316, an E weight calculation circuit 317, a synthesis circuit 318, and an addition circuit 319. Compared to the configuration shown in FIG. 3, the G interpolation circuit 202 further includes an HV edge detection circuit 1301 and a filter gain calculation circuit 1302, and the other configuration is the same as that of FIG.

  FIG. 14 is a flowchart showing the flow of the image processing method of the G interpolation circuit 202. Steps S401 to S409 in FIG. 14 are the same as steps S401 to S409 in FIG. Hereinafter, only differences from the procedure of FIG. 4 will be described.

  In step S1401, the HV edge detection circuit 1301 detects whether the input image signal output from the WB circuit 201 is in the horizontal and vertical edge regions. The HV edge detection circuit 1301 is a second degree calculating unit. Although the details of the process of the HV edge detection circuit 1301 will be described later, the HV edge detection circuit 1301 outputs the HV edge degree β between 0.0 and 1.0 according to the degree of the horizontal and vertical edges. The HV edge degree β outputs, for example, 0.0 when it is judged to be a horizontal or vertical edge area, and 1.0 if it is judged not to be a horizontal or vertical edge area .

Next, in step S1402, the filter gain calculation circuit 1302 calculates the filter gain γ based on the texture degree α output from the texture detection circuit 309 and the HV edge degree β output from the HV edge detection circuit 1301. Do. Specifically, the filter gain calculation circuit 1302 calculates the filter gain γ in the pixel of interest (j, i) by the following equation (33).
γ _{i, j} = α _{i, j} × β _{i, j} (33)

  In this way, by calculating the filter gain γ, for example, when it is judged that a block-shaped false pattern does not occur in a diagonal linear object, or when it is judged to be a horizontal or vertical edge region The filter gain γ has a value close to 0.0. In addition, when a block-shaped false pattern is generated on a diagonal linear object, and when it is determined that the horizontal and vertical edge regions are not included, the filter gain γ has a value close to 1.0. It becomes.

  Next, in step S1403, the N filter circuit 310 performs upward filter processing on the color difference signal output from the V color difference calculation circuit 305 based on the filter gain γ output from the filter gain calculation circuit 1302. . Specifically, first, using the color difference signal Diff_v output from the V color difference calculation circuit 305, the N filter circuit 310 uses the color difference signal Diff_v to calculate the result Fil_n of the upward filter processing of the pixel of interest (j, i) as Calculated by

Next, the N filter circuit 310 calculates the color difference signal Diff_n according to the following equation (34) based on the filter gain γ and outputs it.
Diff_n _{i, j} = γ _{i, j} × Diff_v _{i, j}
+ (1.0-? _{I, j} ) x Fil_n _{i, j} (34)

  Next, in step S1404, the S filter circuit 311 performs downward filter processing on the color difference signal output from the V color difference calculation circuit 305 based on the filter gain γ output from the filter gain calculation circuit 1302. . Specifically, first, the S filter circuit 311 uses the color difference signal Diff_v output from the V color difference calculation circuit 305 to apply the result Fil_s of the downward filter processing at the pixel of interest (j, i) to the formula (13) Calculated by

Next, the S filter circuit 311 calculates and outputs the color difference signal Diff_s by the following equation (35) based on the filter gain γ.
Diff_s _{i, j} = γ _{i, j} × Diff_v _{i, j}
+ (1.0-γ _{i, j} ) × Fil_s _{i, j} (35)

  Next, in step S1405, the W filter circuit 312 performs leftward filtering on the color difference signal output from the H color difference calculation circuit 306 based on the filter gain γ output from the filter gain calculation circuit 1302. . Specifically, first, using the color difference signal Diff_h output from the H color difference calculation circuit 306, the W filter circuit 312 uses the color difference signal Diff_h of the target pixel (j, i) to filter the result Fil_w in the left direction into Calculated by

Next, the W filter circuit 312 calculates and outputs the color difference signal Diff_W according to the following equation (36) based on the filter gain γ.
Diff_wi _{, j} = γi _{, j} × Diff_hi _{, j}
+ (1.0-? _{I, j} ) x Fil_wi _{, j} (36)

  Next, in step S1406, the E filter circuit 313 performs rightward filtering on the color difference signal output from the H color difference calculation circuit 306 based on the filter gain γ output from the filter gain calculation circuit 1302. . Specifically, first, the E filter circuit 313 uses the color difference signal Diff_h output from the H color difference calculation circuit 306 to calculate the result Fil_e of the filter processing in the right direction of the pixel of interest (j, i) Calculated by

Next, the E filter circuit 313 calculates and outputs the color difference signal Diff_e by the following equation (37) based on the filter gain γ.
Diff_e _{i, j} = γ _{i, j} × Diff_h _{i, j}
+ (1.0-? _{I, j} ) x Fil_e _{i, j} (37)

  Steps S414 to S419 in FIG. 14 are the same as steps S414 to S419 in FIG. As described above, when the target pixel is an R pixel or a B pixel, the color difference signal after filter processing is synthesized by the combining circuit 318 with the weight for each direction, and then added to the target pixel by the addition circuit 319. A G signal to be output of the interpolation circuit 202 is generated.

  FIG. 15 is a block diagram showing a configuration example of the HV edge detection circuit 1301 of FIG. The HV edge detection circuit 1301 includes an HV correlation determination circuit 1501 and an edge degree calculation circuit 1502.

The HV correlation determination circuit 1501 calculates correlation values of horizontal and vertical edges from the magnitude of the horizontal correlation and the magnitude of the vertical correlation. Specifically, the HV correlation determination circuit 1501 determines the horizontal difference from the difference between the signal level difference of the pixel in the horizontal direction with respect to the pixel of interest and the signal level of the pixel in the vertical direction with respect to the pixel of interest. Calculate the correlation value of the vertical edge. For example, when the pixel of interest (j, i) is an R pixel, the HV edge correlation value corrHV is calculated by the following equation (38).
corrHV _{i, j} = | diffH _{i, j-} diffV _{i, j} |
diff H _{i, j} = | G _{i, j-1} -G _{i, j + 1} | + | 2 × R _{i, j} -R _{i, j-2} -R _{i, j +2} |
diff V _{i, j} = | G _{i -1, j} -G _{i + 1, j} | + | 2 x R _{i, j} -R _{i-2, j} -R _{i + 2, j} |
... (38)

  The HV correlation determination circuit 1501 calculates the HV edge correlation value corrHV in the same manner as described above even when the target pixel is a B pixel.

  The edge degree calculation circuit 1502 receives the HV edge correlation value corrHV from the HV correlation determination circuit 1501, and calculates an HV edge degree β indicating whether it is a horizontal or vertical edge region.

  FIG. 16 is a diagram showing an example of input / output characteristics of the edge degree calculation circuit 1502, and shows an example of a conversion table from the HV edge correlation value corrHV to the HV edge degree β. In FIG. 16, the horizontal axis indicates the HV edge correlation value corrHV, and the vertical axis indicates the HV edge degree β. The edge degree calculation circuit 1502 determines that the HV edge degree β is 1.0 if the HV edge correlation value corrHV is equal to or less than a predetermined threshold (third threshold Th3 or less) and is not a horizontal or vertical edge region. Make it If the edge degree calculation circuit 1502 determines that the HV edge degree correlation value corrHV is equal to or more than a predetermined threshold (fourth threshold Th4 or more) and is a horizontal or vertical edge region, the HV edge degree β Set to 0.0. Further, when the HV edge degree correlation value corrHV is larger than the third threshold Th3 and smaller than the fourth threshold Th4, the edge degree calculation circuit 1502 linearly sets 0 according to the HV edge degree correlation value corrHV. Output an HV edge degree β between 0 and 1.0.

  In this embodiment, a texture detection circuit 309, an HV edge detection circuit 1301, and a filter gain calculation circuit 1302 are provided in order to reduce block-shaped false patterns and deviations with respect to a diagonal linear object. The texture detection circuit 309 can calculate the edge strength signal to calculate the texture degree α, and can detect a block-shaped false pattern in the diagonal linear object. On the other hand, the HV edge detection circuit 1301 calculates the HV edge correlation value to calculate the HV edge degree β, and can detect whether or not the pixel of interest is a horizontal or vertical edge region. Further, the filter gain calculation circuit 1302 calculates the filter gain γ from the texture degree α output from the texture detection circuit 309 and the HV edge degree β output from the HV edge detection circuit 1301. Then, the rate at which the color difference signal not subjected to the filtering process of the filter circuits 310 to 313 is output is adjusted.

  When a block-shaped false pattern occurs in a diagonal linear object, and it is determined that it is not in the horizontal and vertical edge regions, the filter gain γ is 1.0, and the filter circuits 310 to 313 perform filter processing. Do not change (change the filtering characteristics). Color difference signals output from the filter circuits 310 to 313 when the target pixel is an R pixel or a B pixel when a block-shaped false pattern occurs in a diagonal linear object and the horizontal and vertical edge regions are not included Is the difference between the left and right or upper and lower G pixels of the target pixel and the target pixel. Then, when it is added to the pixel of interest by the addition circuit 319, the result is a value calculated from the left and right or upper and lower G signals. As a result, since the G signal can be generated without using the R signal and the B signal, even if the R and B signals deviate with respect to the G signal due to the influence of the optical aliasing and the chromatic aberration, it is oblique. It is possible to prevent the occurrence of block-shaped false patterns for linear objects.

  On the other hand, when a block-shaped false pattern does not occur in the diagonal line-like subject, or when it is determined to be a horizontal or vertical edge region, the filter gain γ is 0.0, and the filter circuit 310 to 313 performs filter processing. As described above, when the block-shaped false pattern does not occur in the diagonal linear object, the G signal can be generated using not only the G signal but also the R and B signals by performing the filtering process. Also, even if it is determined that the area is a horizontal or vertical edge area, even if it is an area where a block-shaped false pattern is likely to occur in a diagonally linear object, that is, an area where vertical and horizontal lines are switched to oblique lines. The same is true for Also in this case, the G signal can be generated using not only the G signal but also the R and B signals by performing the filtering process. Therefore, aliasing caused by sampling of G pixels can be reduced. Furthermore, when the filter gain γ is 0.0 to 1.0, the block-like false with respect to the diagonal line-like subject is obtained by changing the intensity of the color difference signal not to be subjected to the filtering process according to the intensity. It is possible to reduce the occurrence of patterns and deflections and improve the image quality.

  As described above, according to the present embodiment, when generating the G signal and the luminance signal from the image signal of the Bayer array, an image having a high sense of resolution is generated regardless of the direction, and an oblique linear It is possible to obtain an image in which a block-like false pattern and a slip on the subject are reduced.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. Can also be realized. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

  In addition, the said embodiment only shows the example of embodiment in the case of implementing this invention, and the technical scope of this invention should not be limitedly interpreted by these. That is, the present invention can be implemented in various forms without departing from the technical concept or the main features thereof.

305 V color difference calculation circuit, 306 H color difference calculation circuit, 309 texture detection circuit, 310 N filter circuit, 311 S filter circuit, 312 W filter circuit, 313 E filter circuit

Claims (13)

First color difference signal calculation means for calculating a color difference signal in the vertical direction of the input image signal;
Second color difference signal calculation means for calculating a color difference signal in the horizontal direction of the input image signal;
First filter processing means for filtering the color difference signal in the vertical direction;
Second filter processing means for performing filter processing on the color difference signal in the horizontal direction;
First degree calculating means for calculating a first degree indicating whether the pixel of interest of the input image signal is an area affected by predetermined chromatic aberration or optical aliasing;
An image processing apparatus characterized in that the first filter processing means and the second filter processing means change the characteristics of the filter processing based on the calculation result of the first degree calculation means.   The first filter processing unit and the second filter processing unit execute the filter processing when the first degree calculated by the first degree calculation unit is equal to or more than a first threshold set in advance. The image processing apparatus according to claim 1, wherein the image processing is not performed.   When the first degree calculated by the first degree calculating means is smaller than the first threshold, the first filtering means and the second filtering means set the first degree to the first degree. The image processing apparatus according to claim 2, wherein filtering processing is performed according to the request. First interpolation means for interpolating a color signal without using a color difference signal for an input image signal;
Second interpolation means for interpolating a color signal using the color difference signal with respect to the input image signal;
A first degree calculating unit that calculates a first degree indicating whether the pixel of interest of the input image signal is an area affected by predetermined chromatic aberration or optical aliasing;
And combining means for combining the color signal interpolated by the first interpolation means and the color signal interpolated by the second interpolation means based on the calculation result of the first degree calculation means. Image processing device characterized by   The combining means performs combining processing so that the ratio of the interpolation result of the first interpolation means becomes higher as the first degree calculated by the first degree calculation means becomes higher. The image processing device according to Item 4. The second interpolation means is
First color difference signal calculation means for calculating a color difference signal in the vertical direction of the input image signal;
Second color difference signal calculation means for calculating a color difference signal in the horizontal direction of the input image signal;
First filter processing means for filtering the color difference signal in the vertical direction;
And second filtering means for filtering the horizontal color difference signal,
6. The image processing apparatus according to claim 4, wherein the color signal is interpolated based on the signals processed by the first filter processing unit and the second filter processing unit. The apparatus further comprises second degree calculating means for calculating a second degree indicating the degree of the horizontal or vertical edge with respect to the input image signal,
The first filter processing means and the second filter processing means change the characteristics of the filter processing based on the calculation results of the first degree calculation means and the second degree calculation means. The image processing apparatus according to claim 1.   In the first filter processing means and the second filter processing means, the first degree calculated by the first degree calculation means is equal to or greater than a preset first threshold, and the second degree 8. The image processing apparatus according to claim 7, wherein the filter processing is not performed when the second degree calculated by the calculation means is equal to or less than a second threshold set in advance.   The first filter processing means and the second filter processing means are configured such that the first degree calculated by the first degree calculation means is smaller than the first threshold, or by the second degree calculation means 9. The image processing apparatus according to claim 8, wherein, when the calculated second degree is larger than the second threshold value, filtering processing is performed according to the first degree and the second degree. . The degree calculating means
Edge direction determining means for determining the direction of the edge of the first color signal of the input image signal;
Edge strength calculating means for calculating edge strength in a direction opposite to the direction determined by the edge direction determining means as the first degree with respect to color signals of colors different from the first color signal The image processing apparatus according to any one of claims 1 to 9, characterized in that: A first color difference signal calculating step of calculating a color difference signal in the vertical direction of the input image signal;
A second color difference signal calculating step of calculating a horizontal color difference signal of the input image signal;
A first filtering step of filtering the color difference signal in the vertical direction;
A second filtering step of filtering the color difference signal in the horizontal direction;
A first degree calculating step of calculating a first degree indicating whether the target pixel of the input image signal is an area affected by predetermined chromatic aberration or optical aliasing;
An image processing method characterized in that in the first filter processing step and the second filter processing step, the characteristic of the filter processing is changed based on the calculation result of the first degree calculation step. A first interpolation step of interpolating color signals without using color difference signals with respect to an input image signal;
A second interpolation step of interpolating color signals from the input image signal using color difference signals;
A first degree calculating step of calculating a first degree indicating whether the target pixel of the input image signal is an area affected by predetermined chromatic aberration or optical aliasing;
Combining the color signal interpolated in the first interpolation step and the color signal interpolated in the second interpolation step based on the calculation result of the first degree calculation step; Characteristic image processing method.   The program for functioning a computer as each means of the image processing apparatus in any one of Claims 1-10.