JP6987622B2 - Image processing equipment, image processing methods and programs - Google Patents

Description

The present invention relates to an image processing apparatus, an image processing method and a program.

In Patent Document 1, when the G signal, which is the main component of the luminance signal, is interpolated, the correlation in the vertical direction or the horizontal direction is discriminated, and if there is a correlation in the vertical direction, LPF processing is performed in the vertical direction. , A technique for performing LPF processing in the lateral direction when there is a correlation in the lateral direction is disclosed. Then, when there is no correlation in the vertical direction and the horizontal direction, a two-dimensional LPF process is performed to prevent the vertical and horizontal edges from being blurred.

However, in the technique of Patent Document 1, since the two-dimensional LPF processing is performed when there is no vertical or horizontal correlation, for example, the diagonal edge is blurred as compared with the vertical and horizontal edges.

Therefore, Non-Patent Document 1 discloses a method for calculating the G signal at the position of the R or B pixel. In Non-Patent Document 1, the color difference signals of the G pixel and the pixel of the same color as the pixel of interest are calculated in the upper, lower, left, and right directions including the pixel of interest, and the color difference signal obtained by synthesizing the color difference signals in each direction is obtained. The G signal is calculated by adding it to the pixel of interest. At this time, in Non-Patent Document 1, by synthesizing the color difference signal in each direction according to the inclination of the color difference signal in the corresponding direction, an image having a high resolution is generated regardless of the direction.

Japanese Patent No. 3862506

I. Pekkucuksen, Y. Altunbasak, "Gradient based threshold free color filter array interpolation", ICIP 2010

However, in Non-Patent Document 1, since the G signal is generated using the R and B signals, the subject whose R and B signal values are higher than the G signal, such as a red subject and a blue subject, is used. The following problems occur. That is, the folding back of the R and B signals due to the small sampling of the R and B signals with respect to the G signal is added to the G signal as an edge having a higher amplitude than the edge existing in the G signal. False patterns due to wrapping may be noticeable.

In view of the above-mentioned problems, it is an object of the present invention to be able to generate an image in which false patterns generated in a subject such as red or blue, which have higher signal values of R and B signals than G signals, are reduced. ..

The image processing apparatus according to the present invention is an image processing apparatus that calculates a G signal for an input image signal having an RGB signal based on a bayer arrangement in the horizontal and vertical directions, and interpolates the input image signal in the vertical direction. The first color difference signal calculating means for calculating the color difference signal, the second color difference signal calculating means for calculating the color difference signal by horizontally interpolating the input image signal, and the first color difference signal calculating means. A first filter processing means for filtering the calculated color difference signal, a second filter processing means for filtering the color difference signal calculated by the second color difference signal calculation means, and R. A color signal intensity calculating means for calculating at least one intensity of a signal and a B signal, a result of filtering by the first filtering means, a result of filtering by the second filtering means, and the color. It has a color signal calculation means for calculating the G signal based on at least one intensity of the R signal and the B signal calculated by the signal strength calculation means, and has the first filter processing means and the second filter processing means. The filter processing means is characterized in that the filter processing is not performed when at least one intensity of the R signal and the B signal calculated by the color signal intensity calculation means is equal to or higher than a preset threshold value. do.

According to the present invention, it is possible to generate an image in which false patterns generated in a subject such as red or blue, in which the signal values of the R and B signals are higher than those of the G signal, are reduced.

It is a figure which shows 1 unit of the primary color Bayer array. It is a block diagram which shows the structural example of the luminance signal generation part which concerns on embodiment. It is a block diagram which shows the structural example of the G interpolation circuit of 1st Embodiment. It is a flowchart which shows an example of the processing flow of the G interpolation circuit in 1st Embodiment. It is a block diagram which shows the structural example of a red, blue subject detection circuit. It is a flowchart which shows an example of the processing flow of a red, blue subject detection circuit. It is a figure for demonstrating the relationship between a color signal C and a color signal intensity α. It is a block diagram which shows the structural example of the G interpolation circuit of 2nd Embodiment. It is a block diagram which shows the structural example of the 1st G interpolation circuit of 2nd Embodiment. It is a block diagram which shows the structural example of the 2nd G interpolation circuit of 2nd Embodiment. It is a flowchart which shows an example of the processing flow of the 2nd G interpolation circuit.

(First Embodiment)
FIG. 1 is a diagram showing an image pickup device having a Bayer array color filter according to the first embodiment of the present invention. The image pickup device has an image pickup element such as a CMOS image sensor. The image pickup device can be applied to smartphones, tablets, industrial cameras, medical cameras, etc., in addition to digital cameras and video cameras. The image pickup element has a plurality of pixels arranged in a two-dimensional matrix, and each of the plurality of pixels has a color filter (for example, three colors of red (R), green (G), and blue (B). Has one color filter). FIG. 1 shows a unit color filter of a primary color Bayer array. The image pickup element performs photoelectric conversion for each pixel, performs analog-to-digital conversion, and outputs a digital R signal (red signal), G1 signal (green signal), G2 signal (green signal), or B signal (green signal). do. The G1 signal and the G2 signal are G signals (green signals). As described above, the G signal has a larger number of samplings than the R signal and the B signal. An image pickup device having a Bayer array color filter can obtain only one color signal of R, G, and B in each pixel. Therefore, when obtaining a color signal of all RGB in each pixel, FIG. It is necessary to perform interpolation processing in the brightness signal generation unit 200 of.

FIG. 2 is a diagram showing a configuration example of the luminance signal generation unit 200 according to the present embodiment. The luminance signal generation unit 200 is an image processing device, and inputs digital R signals, G signals, and B signals of the Bayer array from the image pickup device of FIG. The luminance signal generation unit 200 includes a WB circuit (white balance circuit) 201, a G interpolation circuit 202, an R interpolation circuit 203, a B interpolation circuit 204, an APC circuit 205, a luminance signal generation circuit 206, and an addition circuit 207. And have.

The WB circuit 201 inputs digital image signals (R signal, G signal, and B signal) of the bayer arrangement from the image pickup device, and corrects the white balance of the image signal. The G interpolation circuit 202 inputs the image signal of the Bayer array output from the WB circuit 201, calculates the G signal at the positions of the R pixel and the B pixel in FIG. 1 by interpolation, and outputs the G signal of all the pixels. The R pixel is a pixel provided with a red filter, the B pixel is a pixel provided with a blue filter, and the G pixel is a pixel provided with a green filter. Details of the processing of the G interpolation circuit 202 will be described later.

The R interpolation circuit 203 calculates the R signal at the positions of the G pixel and the B pixel of the input image of FIG. 1 by interpolation with respect to the input image signal of the bayer array output by the WB circuit 201, and obtains the R signal of all the pixels. Output. For example, the R interpolation circuit 203 calculates the R signal by two-dimensional LPF (low-pass filter) processing after setting the signal level other than the R pixel to 0 with respect to the input image signal of the bayer array output by the WB circuit 201. do.

The B interpolation circuit 204 calculates the B signal at the positions of the R pixel and the G pixel of the input image of FIG. 1 by interpolation with respect to the input image signal of the bayer array output by the WB circuit 201, and obtains the B signal of all the pixels. Output. For example, the B interpolation circuit 204 calculates the B signal by two-dimensional LPF processing after setting the signal level other than the B pixel to 0 with respect to the input image signal of the Bayer array output by the WB circuit 201.

The APC circuit 205 generates an aperture correction signal by applying an HPF (high-pass filter) or the like to the G signal output from the G interpolation circuit 202. The adder circuit 207 adds the G signal output by the G interpolation circuit 202 and the output signal of the APC circuit 205, and outputs the G signal of all pixels. The luminance signal generation circuit 206 is based on the G signal output by the addition circuit 207, the R signal output by the R interpolation circuit 203, and the B signal output by the B interpolation circuit 204, according to the following equation (1). The brightness signal Y of the pixel is generated.
Y = 0.3R + 0.59G + 0.11B ... (1)

FIG. 3 is a block diagram showing a configuration example of the G interpolation circuit 202 of FIG. The G interpolation circuit 202 includes a G pixel V interpolation circuit 301, a G pixel H interpolation circuit 302, an R / B pixel V interpolation circuit 303, an R / B pixel H interpolation circuit 304, a V color difference calculation circuit 305, and H. It has a color difference calculation circuit 306. Further, the G interpolation circuit 202 includes a V color difference inclination calculation circuit 307, an H color difference inclination calculation circuit 308, a red / blue subject detection circuit 309, an N filter circuit 310, an S filter circuit 311 and a W filter circuit 312. , E-filter circuit 313. Further, the G interpolation circuit 202 includes an N weight calculation circuit 314, an S weight calculation circuit 315, a W weight calculation circuit 316, an E weight calculation circuit 317, a synthesis circuit 318, and an addition circuit 319.

FIG. 4 is a flowchart showing the flow of the image processing method of the G interpolation circuit 202. In step S401, the G pixel V interpolation circuit 301 calculates the G signal by vertically interpolating the input image signal of the Bayer array output by the WB circuit 201, and outputs the G signal of all the pixels. do. Specifically, the G pixel V interpolation circuit 301 outputs as a G signal as it is when the pixel of interest is a G pixel, and when the pixel of interest is an R pixel or a B pixel, the interpolation of the G signal in the vertical direction. The G signal is calculated by performing the processing. For example, in the G pixel V interpolation circuit 301, when the X coordinate and the Y coordinate of the pixel of interest in the image are (j, i) and the pixel of interest is an R pixel, the G signal Gv _{i, j} is given by the following equation (2). Calculated by. Further, the G pixel V interpolation circuit 301 calculates the G signal by the same method as when the pixel of interest is the R pixel even when the pixel of interest is the B pixel.
Gv _{i, j} = (G _{i-1, j} + G _{i + 1, j} ) / 2 ... (2)

Next, in step S402, the G pixel H interpolation circuit 302 calculates the G signal by horizontally interpolating the input image signal output by the WB circuit 201, and outputs the G signal of all the pixels. do. Specifically, the G pixel H interpolation circuit 302 outputs as a G signal as it is when the pixel of interest is a G pixel, and when the pixel of interest is an R pixel or a B pixel, the interpolation of the G signal in the horizontal direction. The G signal is calculated by performing the processing. For example, the G pixel H interpolation circuit 302 _{calculates the G signals Gh i, j} by the following equation (3) when the pixel of interest is the R pixel. Further, the G pixel H interpolation circuit 302 calculates the G signal by the same method as when the pixel of interest is the R pixel even when the pixel of interest is the B pixel.
Gh _{i, j} = (G _{i, j-1} + G _{i, j + 1} ) / 2 ... (3)

Next, in step S403, the R and B pixel V interpolation circuit 303 calculates the R signal and the B signal by performing interpolation processing in the vertical direction with respect to the input image signal output by the WB circuit 201, and all the pixels. R signal and B signal are output. Specifically, when the pixel of interest is an R pixel or a B pixel, the R, B pixel V interpolation circuit 303 outputs it as an R signal or a B signal as it is. The R and B pixel V interpolation circuit 303 calculates an R signal by performing interpolation processing in the vertical direction when the pixel of interest is a G pixel and the vertical direction of the pixel of interest is an R pixel, and the pixel of interest When the vertical direction is a B pixel, the B signal is calculated by performing interpolation processing in the vertical direction. For example, in the R, B pixel V interpolation circuit 303, when the pixel of interest is a G pixel and the vertical direction of the pixel of interest is an R pixel, the R signals Rv _{i, j} are calculated by the following equation (4). Further, the R and B pixel V interpolation circuit 303 calculates the B signal by the same method as when the vertical direction of the pixel of interest is the B pixel even when the vertical direction of the pixel of interest is the R pixel.
Rv _{i, j} = (R _{i-1, j} + R _{i + 1, j} ) / 2 ... (4)

Next, in step S404, the R and B pixel H interpolation circuit 304 calculates the R signal and the B signal by horizontally interpolating the input image signal output by the WB circuit 201, and all the pixels. R signal and B signal are output. Specifically, when the pixel of interest is an R pixel or a B pixel, the R and B pixel H interpolation circuit 304 outputs the R signal or the B signal as it is. The R and B pixel H interpolation circuit 304 calculates the R signal by performing the horizontal interpolation processing when the pixel of interest is the G pixel and the pixel of interest is the R pixel in the horizontal direction, and the pixel of interest is calculated. When the horizontal direction is a B pixel, the B signal is calculated by performing an interpolation process in the horizontal direction. For example, in the R and B pixel H interpolation circuit 304, when the pixel of interest is a G pixel and the horizontal direction of the pixel of interest is an R pixel, the R signals Rh _{i and j} are calculated by the following equation (5). Further, the R and B pixel H interpolation circuit 304 calculates the B signal by the same method as when the horizontal direction of the pixel of interest is the B pixel even when the horizontal direction of the pixel of interest is the R pixel.
Rh _{i, j} = (R _{i, j-1} + R _{i, j + 1} ) / 2 ... (5)

In steps S401 to S404, as a method of vertically and horizontally interpolating the G pixel and the R and B pixels, interpolation was performed from the adjacent pixels of the pixel of interest using the equations (2) to (5). The interpolation may be performed using pixels other than adjacent pixels without limitation.

Next, in step S405, the V color difference calculation circuit 305 subtracts the R signal or the B signal output from the R and B pixel V interpolation circuit 303 from the G signal output from the G pixel V interpolation circuit 301. , Calculates the vertical color difference signal of the input image signal. The V color difference calculation circuit 305 is a first color difference signal calculation means. In the V color difference calculation circuit 305, in the pixel of interest (j, i), the signal output by the G pixel V interpolation circuit 301 is the G signal Gv _{i, j} , and the signal output by the R, B pixel V interpolation circuit 303 is R. In the case of signals Rv _{i, j} , the color difference signal Diff_v in the vertical direction is calculated by the following equation (6). Further, in the V color difference calculation circuit 305, even when the signal output by the R and B pixel V interpolation circuit 303 is a B signal, the same method as when the signal output by the R and B pixel V interpolation circuit 303 is an R signal is used. , The vertical color difference signal of the G signal and the B signal is calculated.
Diff_v _{i, j} = Gv _{i, j} −Rv _{i, j}・ ・ ・ (6)

Next, in step S406, the H color difference calculation circuit 306 subtracts the R signal or the B signal output from the R and B pixel H interpolation circuits 304 from the G signal output from the G pixel H interpolation circuit 302. , Calculates the horizontal color difference signal of the input image signal. The H color difference calculation circuit 306 is a second color difference signal calculation means. In the H color difference calculation circuit 306, in the pixel of interest (j, i), the signal output by the G pixel H interpolation circuit 302 is the G signal Gh _{i, j} , and the signal output by the R, B pixel H interpolation circuit 304 is R. In the case of the signals Rh _{i and j} , the horizontal color difference signal Diff_h is calculated by the following equation (7). Further, in the H color difference calculation circuit 306, even when the signal output by the R and B pixel H interpolation circuit 304 is a B signal, the same method as when the signal output by the R and B pixel H interpolation circuit 304 is an R signal is used. , The horizontal color difference signal of the G signal and the B signal is calculated.
_{Diff_h i, j = Gh i,} j -Rh i, j ··· (7)

Next, in step S407, the V color difference inclination calculation circuit 307 calculates the inclination of the color difference in the vertical direction based on the color difference signal output from the V color difference calculation circuit 305. Specifically, the V color difference inclination calculation circuit 307 uses the following equation (the vertical color difference inclination signal Grad_v) based on the color difference signal Diff_v output from the V color difference calculation circuit 305 in the pixel (j, i) of interest. Calculated according to 8).
Grad_v _{i, j} = | Diff_v _{i-1, j −} Diff_v _{i + 1, j} ｜ ・ ・ ・ (8)

Next, in step S408, the H color difference inclination calculation circuit 308 calculates the inclination of the color difference in the horizontal direction based on the color difference signal output from the H color difference calculation circuit 306. Specifically, the H color difference inclination calculation circuit 308 uses the following equation (the horizontal color difference inclination signal Grad_h) based on the color difference signal Diff_h output from the H color difference calculation circuit 306 in the pixel (j, i) of interest. Calculated according to 9).
Grad_h _{i, j} = | Diff_h _{i, j-1 −} Diff_h _{i, j + 1} ｜ ・ ・ ・ (9)

Next, in step S409, the red and blue subject detection circuit 309 is a red subject having a high R signal intensity, a blue subject having a high B signal intensity, or a magenta color in which both the R signal and the B signal are high. Detects the subject of. The red and blue subject detection circuit 309 is a color signal intensity calculation means. The details of the processing of the red / blue subject detection circuit 309 will be described later, but in the red / blue subject detection circuit 309, the subject is red, blue, or magenta, and the color is between 0.0 and 1.0 depending on the degree. The signal strength α is output. The color signal intensity α is output as, for example, 0.0 when it is determined that the subject is not a red, blue, or magenta subject, and the subject is determined to be a red, blue, or magenta subject. If so, 1.0 is output.

Next, in step S410, the N filter circuit 310 filters the color difference signal output from the V color difference calculation circuit 305 in the upward direction based on the color signal intensity α output from the red and blue subject detection circuits 309. Perform processing. Specifically, first, the N filter circuit 310 uses the color difference signal Diff_v output from the V color difference calculation circuit 305 to obtain the result of the upward filter processing in the pixel of interest (j, i) by the following equation (11). ).

Next, the N filter circuit 310 calculates and outputs the color difference signal Diff_n by the following equation (12) based on the color signal intensity α.
Diff_n _{i, j} = α _{i, j} × Diff_v _{i, j}
+ (1.0-α _{i, j} ) × Fil_ni _{i, j} ... (12)

Next, in step S411, the S filter circuit 311 filters the color difference signal output from the V color difference calculation circuit 305 in the downward direction based on the color signal intensity α output from the red and blue subject detection circuits 309. Perform processing. Specifically, first, the S filter circuit 311 uses the color difference signal Diff_v output from the V color difference calculation circuit 305 to obtain the result of the downward filter processing in the pixel of interest (j, i) by the following equation (13). ).

Next, the S filter circuit 311 calculates and outputs the color difference signal Diff_s by the following equation (14) based on the color signal intensity α.
Diff_s _{i, j} = α _{i, j} × Diff_v _{i, j}
+ (1.0-α _{i, j} ) × Fil_s _{i, j} ... (14)

Next, in step S412, the W filter circuit 312 filters the color difference signal output from the H color difference calculation circuit 306 in the left direction based on the color signal intensity α output from the red and blue subject detection circuits 309. Perform processing. Specifically, first, the W filter circuit 312 uses the color difference signal Diff_h output from the H color difference calculation circuit 306 to obtain the left-hand filter processing result Fil_w in the pixel of interest (j, i) by the following equation (15). ).

Next, the W filter circuit 312 calculates and outputs the color difference signal Diff_W by the following equation (16) based on the color signal intensity α.
Diff_w _{i, j} = α _{i, j} × Diff_h _{i, j}
+ (1.0-α _{i, j} ) × Fil_w _{i, j} ... (16)

Next, in step S413, the E filter circuit 313 filters the color difference signal output from the H color difference calculation circuit 306 in the right direction based on the color signal intensity α output from the red and blue subject detection circuits 309. Perform processing. Specifically, first, the E filter circuit 313 uses the color difference signal Diff_h output from the H color difference calculation circuit 306 to obtain the result of the filter processing in the right direction in the pixel of interest (j, i) by the following equation (17). ).

Next, the E filter circuit 313 calculates and outputs the color difference signal Diff_e by the following equation (18) based on the color signal intensity α.
Diff_e _{i, j} = α _{i, j} × Diff_h _{i, j}
+ (1.0-α _{i, j} ) × Fil_e _{i, j} ... (18)

Next, in step S414, the N weight calculation circuit 314 calculates the upward weight Wn by the following equation (19) based on the vertical color difference slope signal Grad_v output from the V color difference slope calculation circuit 307. ..

Next, in step S415, the S weight calculation circuit 315 calculates the downward weight Ws by the following equation (20) based on the vertical color difference slope signal Grad_v output from the V color difference slope calculation circuit 307. ..

Next, in step S416, the W weight calculation circuit 316 calculates the left weight Ww by the following equation (21) based on the horizontal color difference slope signal Grad_h output from the H color difference slope calculation circuit 308. ..

Next, in step S417, the E weight calculation circuit 317 calculates the right weight We by the following equation (22) based on the horizontal color difference slope signal Grad_h output from the H color difference slope calculation circuit 308. ..

Next, in step S418, the synthesis circuit 318 has a weight Wn input from the N weight calculation circuit 314, a weight Ws input from the S weight calculation circuit 315, a weight Ww input from the W weight calculation circuit 316, and E. The weight We input from the weight calculation circuit 317 is input. Further, the synthesis circuit 318 inputs the color difference signal Diff_n input from the N filter circuit 310 and the color difference signal Diff_s input from the S filter circuit 311. Further, the synthesis circuit 318 inputs the color difference signal Diff_w input from the W filter circuit 312 and the color difference signal Diff_e input from the E filter circuit 313. The synthesis circuit 318 synthesizes the color difference signal Diff_n, the color difference signal Diff_s, the color difference signal Diff_w, and the color difference signal Diff_e based on the weights Wn, the weight Ws, the weight Ww, and the weight We, and formulates the color difference signal Diff_mix by the following equation ( Calculated according to 23).
Diff_mix _{i, j} = (Wn _{i, j} x Diff_n _{i, j} + Ws _{i, j} x Diff_s _{i, j}
＋ Ww _{i, j} × Diff_w _{i, j} ＋ We _{i, j} × Diff_e _{i, j} ) / Wt _{i, j}
Wt _{i, j} = Wn _{i, j} + Ws _{i, j} + Ww _{i, j} + We _{i, j}
... (23)

Next, in step S419, when the pixel of interest of the input image signal input to the G interpolation circuit 202 is an R pixel or a B pixel, the addition circuit 319 is synthesized by the synthesis circuit 318 with respect to the signal of the pixel of interest. The G signal is calculated and output by adding the color difference signal Diff_mix. Further, when the pixel of interest is a G pixel, the addition circuit 319 outputs the image signal input to the G interpolation circuit 202 as it is. The output signal of the adder circuit 319 is an output signal of the G interpolation circuit 202.

As described above, the N filter circuit 310 performs an upward filter process, the S filter circuit 311 performs a downward filter process, the W filter circuit 312 performs a left filter process, and the E filter circuit 313 performs a right filter process. Filter the direction. The N filter circuit 310 and the S filter circuit 311 are first filter processing means, and perform filter processing on the color difference signal in the vertical direction. The W filter circuit 312 and the E filter circuit 313 are second filter processing means, and perform filter processing on the color difference signal in the horizontal direction. When the pixel of interest is an R pixel or a B pixel, the color difference signal after the filter processing is combined with the weight for each direction by the synthesis circuit 318, and then added to the pixel of interest by the addition circuit 319, and the G interpolation circuit is used. A G signal that is the output of 202 is generated.

FIG. 5 is a block diagram showing a configuration example of the red and blue subject detection circuits 309 of FIG. The red and blue subject detection circuits 309 include an R interpolation circuit 501, a G interpolation circuit 502, a B interpolation circuit 503, a subtractor 504, a subtractor 505, a limiter 506, a limiter 507, and an adder 508. It has a coefficient calculation circuit 509.

FIG. 6 is a flowchart showing the flow of the image processing method of the red / blue subject detection circuit 309. In step S601, the R interpolation circuit 501 calculates the R signals at the positions of the G pixels and the B pixels by interpolation with respect to the input image signal of the Bayer array output by the WB circuit 201, and outputs the R signals of all the pixels. .. For example, the R interpolation circuit 501 calculates the R signal by two-dimensional LPF processing after setting the signal level other than the R pixel to 0 with respect to the input image signal of the Bayer array output by the WB circuit 201.

In step S602, the G interpolation circuit 502 calculates the G signals at the positions of the R pixels and the B pixels by interpolation with respect to the input image signal of the Bayer array output by the WB circuit 201, and outputs the G signals of all the pixels. .. For example, the G interpolation circuit 502 calculates the G signal by two-dimensional LPF processing after setting the signal level other than the G pixel to 0 with respect to the input image signal of the Bayer array output by the WB circuit 201.

In step S603, the B interpolation circuit 503 calculates the B signal at the positions of the R pixel and the G pixel by interpolation with respect to the input image signal of the Bayer array output by the WB circuit 201, and outputs the B signal of all the pixels. .. For example, the B interpolation circuit 503 calculates the B signal by two-dimensional LPF processing after setting the signal level other than the B pixel to 0 with respect to the input image signal of the Bayer array output by the WB circuit 201.

Next, in step S604, the subtractor 504 subtracts the G signal output from the G interpolation circuit 502 from the R signal output from the R interpolation circuit 501. In the subtractor 504, in the pixel of interest (j, i), the signal output by the R interpolation circuit 501 is the R signal R _{i, j} , and the signal output by the G interpolation circuit 502 is the G signal G _{i, j. , The color difference signals V i and j,} which are the subtraction results, are calculated by the following equation (24).
V _{i, j} = R _{i, j} −G _{i, j}・ ・ ・ (24)

In step S605, the subtractor 505 subtracts the G signal output from the G interpolation circuit 502 from the B signal output from the B interpolation circuit 503. In the subtractor 505, in the pixel of interest (j, i), the signal output by the B interpolation circuit 503 is the B signal B _{i, j} , and the signal output by the G interpolation circuit 502 is the G signal G _{i, j. , The color difference signals U i and j,} which are the subtraction results, are calculated by the following equation (25).
U _{i, j} = B _{i, j} −G _{i, j}・ ・ ・ (25)

In step S606, the limiter 506 clips the negative signal to 0 with respect to the color difference signal V output from the subtractor 504. By performing the clip processing in this way, when the G signal is larger than the R signal, that is, when there is no possibility that the subject is a red subject, the color difference signal V outputs 0.

In step S607, the limiter 507 clips the negative signal to 0 with respect to the color difference signal U output from the subtractor 505. By performing the clip processing in this way, the color difference signal U outputs 0 when the G signal is larger than the B signal, that is, when there is no possibility that the subject is a blue subject.

In step S608, the adder 508 adds the color difference signal V output from the limiter 506 and the color difference signal U output from the limiter 507. The adder 508 adds when the color difference signal V output from the limiter 506 is V _{i, j} and the color difference signal U output from the limiter 507 is U _{i, j} in the pixel of interest (j, i). The resulting color signals C _{i and j} are calculated by the following equation (26).
C _{i, j} = V _{i, j} + U _{i, j} ... (26)

In step S609, the coefficient calculation circuit 509 calculates the color signal intensity α indicating a certain degree in red, blue, or magenta color based on the color signal C output from the adder 508.

FIG. 7 is a diagram showing an example of a conversion table when the coefficient calculation circuit 509 calculates the color signal intensity α from the color signal C output from the adder 508. In FIG. 7, the horizontal axis represents the color signal C, and the vertical axis represents the color signal intensity α. When the color signal C, which is the calculation result, is equal to or less than the preset first threshold value Th1, the coefficient calculation circuit 509 can consider that the subject is not red, blue, or magenta, and thus sets the color signal intensity α. Output 0.0. Further, the coefficient calculation circuit 509 can consider that the subject is red, blue, or magenta when the color signal C as the calculation result is equal to or higher than the second threshold value Th2. Therefore, the color signal intensity α is 1. Output 0. Here, the second threshold value Th2 is larger than the first threshold value Th1. When the color signal C as the calculation result is larger than the first threshold value Th1 and the color signal C is less than the second threshold value Th2, the coefficient calculation circuit 509 is linear according to the magnitude of the color signal C. The color signal intensity α between 0.0 and 1.0 is calculated.

In the present embodiment, a red / blue subject detection circuit 309 is provided in order to reduce false patterns generated in a subject such as red or blue. The red and blue subject detection circuit 309 calculates the color signal C, calculates the color signal intensity α, and adjusts the ratio of the color difference signals that have not been filtered by the filter circuits 310 to 313.

When the subject is red, blue, or magenta (when the color signal C is equal to or higher than the second threshold value), the color signal intensity α is 1.0, and the filter circuits 310 to 313 do not perform filter processing (when the color signal C is equal to or higher than the second threshold value). Change the characteristics of filtering). In this way, when the subject has a red, blue, or magenta color and the pixel of interest is an R pixel or a B pixel, the color difference signals output from the filter circuits 310 to 313 are of interest from equations (2) to (7). It is the difference between the left and right or upper and lower G pixels of the pixel and the pixel of interest. Then, when it is added to the pixel of interest by the addition circuit 319, it becomes a value calculated from the left and right or upper and lower G signals as a result. As a result, since the G signal can be generated without using the R signal and the B signal, the R and B signals are folded back due to the small sampling of the R and B signals with respect to the G signal. Can be prevented.

On the other hand, when the subject is neither red nor blue nor magenta, the color signal intensity α is 0.0, and the filter circuits 310 to 313 perform filter processing. In this way, when the subject is neither red nor blue nor magenta, the G signal can be generated using not only the G signal but also the R and B signals by performing the filtering process, and the solution can be performed regardless of the direction. It is possible to generate an image with a high image quality. Further, when the color signal intensity α is 0.0 to 1.0, the intensity of the color difference signal that is not filtered is changed according to the intensity, so that a false image generated in a subject such as red or blue is generated. The pattern can be reduced and the image quality can be improved.

As described above, according to the present embodiment, when the G signal and the luminance signal are generated from the image signal of the bayer array, the false patterns generated in the subject such as red and blue are reduced, and then the direction is used. It is possible to obtain an image with a high resolution.

(Second embodiment)
Hereinafter, a second embodiment of the present invention will be described. The configuration of the luminance signal generation unit 200 according to the present embodiment is the same as that in FIG. In this embodiment, only the parts different from the first embodiment will be described.

FIG. 8 is a block diagram showing a configuration example of the G interpolation circuit 202 according to the second embodiment of the present invention. The G interpolation circuit 202 includes a first G interpolation circuit 801 and a second G interpolation circuit 802, a red / blue subject detection circuit 803, and a synthesis circuit 804. The first G interpolation circuit 801 is the first interpolation means, interpolates the G signal with respect to the image signal without using the color difference signal, and outputs the first G signal. The second G interpolation circuit 802 is a second interpolation means, interpolates the G signal using the color difference signal with respect to the image signal, and outputs the second G signal. The synthesis circuit 804 is a synthesis means, and the first G signal and the second G interpolation interpolated by the first G interpolation circuit 801 based on the color signal intensity α output from the red and blue subject detection circuits 803. It is combined with the second G signal interpolated by the circuit 802. The details will be described later. The red / blue subject detection circuit 803 is the same circuit as the red / blue subject detection circuit 309 of FIG. 5 of the first embodiment.

FIG. 9 is a block diagram showing a configuration example of the first G interpolation circuit 801. The first G interpolation circuit 801 has a 0 insertion circuit 901 and an HV interpolation circuit 902. The 0 insertion circuit 901 sets the signal levels of the R pixel and the B pixel of the input image signals to 0. Next, the HV interpolation circuit 902 calculates the G signal by performing interpolation processing in the horizontal direction and the vertical direction with respect to the image signal output by the 0 insertion circuit 901. At this time, the HV interpolation circuit 902 uses, for example, a filter having a coefficient of (1, 2, 1) / 2 for the horizontal and vertical interpolation processing.

FIG. 10 is a block showing a configuration example of the second G interpolation circuit 802. The second G interpolation circuit 802 includes a G pixel V interpolation circuit 1001, a G pixel H interpolation circuit 1002, an R / B pixel V interpolation circuit 1003, an R / B pixel H interpolation circuit 1004, and a V color difference calculation circuit 1005. And the H color difference calculation circuit 1006. Further, the second G interpolation circuit 802 includes a V color difference inclination calculation circuit 1007, an H color difference inclination calculation circuit 1008, an N filter circuit 1009, an S filter circuit 1010, a W filter circuit 1011 and an E filter circuit 1012. Has. Further, the second G interpolation circuit 802 includes an N weight calculation circuit 1013, an S weight calculation circuit 1014, a W weight calculation circuit 1015, an E weight calculation circuit 1016, a synthesis circuit 1017, and an addition circuit 1018. ..

FIG. 11 is a flowchart showing the processing flow of the second G interpolation circuit 802. In step S1101, the G pixel V interpolation circuit 1001 generates a G signal by performing interpolation processing in the vertical direction with respect to the input image signal. Specifically, the G pixel V interpolation circuit 1001 outputs the signal of the G pixel as it is as a G signal when the pixel of interest is a G pixel, and when the pixel of interest is an R pixel or a B pixel, the vertical direction. A G signal is generated by performing interpolation processing. For example, when the X and Y coordinates of the pixel of interest in the image are (j, i), the G pixel V interpolation circuit 1001 uses the G signal Gv _{i, j} as the following equation (27) when the pixel of interest is an R pixel. Generated by.
Gv _{i, j} = (G _{i-1, j} + G _{i + 1, j} ) / 2 + (2 x R _{i, j} −R _{i-2, j} + R _{i + 2, j} ) / 4
... (27)

The G pixel V interpolation circuit 1001 generates a G signal by the same method as described above even when the pixel of interest is the B pixel.

Next, in step S1102, the G pixel H interpolation circuit 1002 generates a G signal by performing horizontal interpolation processing on the input image signal. Specifically, the G pixel H interpolation circuit 1002 outputs the signal of the G pixel as it is as a G signal when the pixel of interest is a G pixel, and when the pixel of interest is an R pixel or a B pixel, the horizontal direction. A G signal is generated by performing interpolation processing. For example, the G pixel H interpolation circuit 1002 _{generates G signals Gh i, j} by the following equation (28) when the pixel of interest is an R pixel.
Gh _{i, j} = (G _{i, j-1} + G _{i, j + 1} ) / 2 + (2 x R _{i, j} −R _{i, j-2} + R _{i, j + 2} ) / 4
... (28)

The G pixel H interpolation circuit 1002 generates a G signal by the same method as described above even when the pixel of interest is the B pixel.

Next, in step S1103, the R and B pixel V interpolation circuit 1003 generates an R signal or a B signal by performing interpolation processing in the vertical direction with respect to the input image signal. Specifically, the R and B pixel V interpolation circuit 1003 outputs the signal of the R pixel or the B pixel as it is as the R signal or the B signal when the pixel of interest is the R pixel or the B pixel. The R and B pixel V interpolation circuit 1003 generates an R signal by performing interpolation processing in the vertical direction when the pixel of interest is a G pixel and the vertical direction of the pixel of interest is an R pixel. When the vertical direction of the pixel of interest is the B pixel, the B signal is generated by performing interpolation processing in the vertical direction. For example, when the pixel of interest is a G pixel and the pixel of interest is an R pixel in the vertical direction, the R, B pixel V interpolation circuit 1003 _{generates R signals Rv i, j} by the following equation (29).
Rv _{i, j} = (R _{i-1, j} + R _{i + 1, j} ) / 2 + (2 x G _{i, j −} G _{i-2, j} + G _{i + 2, j} ) / 4
... (29)

The R and B pixel V interpolation circuit 1003 generates a B signal by the same method as described above when the vertical direction of the pixel of interest is the B pixel.

Next, in step S1104, the R and B pixel H interpolation circuit 1004 generates an R signal or a B signal by performing horizontal interpolation processing on the input image signal. Specifically, the R and B pixel H interpolation circuit 1004 outputs the signal of the R pixel or the B pixel as it is as the R signal or the B signal when the pixel of interest is the R pixel or the B pixel. The R and B pixel H interpolation circuit 1004 generates an R signal by performing horizontal interpolation processing when the pixel of interest is a G pixel and the pixel of interest is an R pixel in the horizontal direction. When the horizontal direction of the pixel of interest is the B pixel, the B signal is generated by performing the horizontal interpolation processing. For example, when the pixel of interest is a G pixel and the pixel of interest is an R pixel in the horizontal direction, the R, B pixel H interpolation circuit 1004 _{generates R signals Rh i, j} by the following equation (30).
Rh _{i, j} = (R _{i, j-1} + R _{i, j + 1} ) / 2 + (2 x G _{i, j −} G _{i, j-2} + G _{i, j + 2} ) / 4
... (30)

The R and B pixel H interpolation circuit 1004 generates a B signal by the same method as described above when the horizontal direction of the pixel of interest is the B pixel. Further, the equations (27) to (30) are used as the interpolation method of steps S1101 to S1104, but the present invention is not limited to this, and the equations (2) to (5) are used, for example, as in the first embodiment. Pixels of the same color may be linearly interpolated in each direction.

Next, in step S1105, the V color difference calculation circuit 1005 subtracts the R signal or the B signal output by the R and B pixel V interpolation circuit 1003 from the G signal output by the G pixel V interpolation circuit 1001 in the vertical direction. Generate a color difference signal. For example, the V color difference calculation circuit 1005 calculates the color difference signal Diff_v in the vertical direction by the equation (6) in the same manner as the V color difference calculation circuit 305 of FIG.

Next, in step S1106, the H color difference calculation circuit 1006 subtracts the R signal or the B signal output by the R and B pixel H interpolation circuits 1004 from the G signal output by the G pixel H interpolation circuit 1002, and subtracts the R signal or the B signal in the horizontal direction. Generate a color difference signal. For example, the H color difference calculation circuit 1006 calculates the horizontal color difference signal Diff_h by the equation (7) in the same manner as the H color difference calculation circuit 306 of FIG.

Next, in step S1107, the V color difference inclination calculation circuit 1007 calculates the inclination of the color difference in the vertical direction based on the color difference signal output from the V color difference calculation circuit 1005. Specifically, the V color difference inclination calculation circuit 1007 calculates the vertical color difference inclination signal Grad_v by the equation (8), similarly to the V color difference inclination calculation circuit 307 of FIG.

The formula for calculating the slope signal Grad_v is not limited to the formula (8). The V color difference inclination calculation circuit 1107 may calculate the inclination signal Grad_v based on the difference between the pixel of interest and the adjacent pixels above and below, for example. The V color difference inclination calculation circuit 1007 outputs the calculated vertical color difference inclination signal Grad_v to the N weight calculation circuit 1013 and the S weight calculation circuit 1014, respectively.

Next, in step S1108, the H color difference inclination calculation circuit 1008 calculates the inclination of the color difference in the horizontal direction based on the color difference signal output from the H color difference calculation circuit 1006. Specifically, the H color difference inclination calculation circuit 1008 calculates the horizontal color difference inclination signal Grad_h by the equation (9) in the same manner as the H color difference inclination calculation circuit 308 of FIG.

The formula for calculating the slope signal Grad_h is not limited to the formula (9). The H color difference inclination calculation circuit 1008 may calculate the inclination signal Grad_h based on the difference between the pixel of interest and the adjacent pixels on the left and right, for example. The H color difference inclination calculation circuit 1008 outputs the calculated horizontal color difference inclination signal Grad_h to the W weight calculation circuit 1015 and the E weight calculation circuit 1016, respectively.

Next, in step S1109, the N filter circuit 1009 performs an upward filter process on the color difference signal output from the V color difference calculation circuit 1005. Specifically, the N filter circuit 1009 uses the color difference signal Diff_v output from the V color difference calculation circuit 1005 to obtain the Diff_n result of the upward filter processing on the pixel of interest (j, i) by the following equation (31). calculate.

In step S1110, the S filter circuit 1010 performs a downward filter process on the color difference signal output from the V color difference calculation circuit 1005. Specifically, the S filter circuit 1010 uses the color difference signal Diff_v output from the V color difference calculation circuit 1005 to obtain Diff_s as a result of downward filtering in the pixel of interest (j, i) by the following equation (32). calculate.

In step S1111, the W filter circuit 1011 filters the color difference signal output from the H color difference calculation circuit 1006 in the left direction. Specifically, the W filter circuit 1011 uses the color difference signal Diff_h output from the H color difference calculation circuit 1006 to obtain the result Diff_w of the left-hand filter processing in the pixel of interest (j, i) by the following equation (33). calculate.

In step S1112, the E filter circuit 1012 filters the color difference signal output from the H color difference calculation circuit 1006 in the right direction. Specifically, the E filter circuit 1012 uses the color difference signal Diff_h output from the H color difference calculation circuit 1006 to obtain the result Diff_e of the right-hand filter processing in the pixel of interest (j, i) by the following equation (34). calculate.

In step S1113, the N weight calculation circuit 1013 calculates the upward weight Wn by the equation (19) in the same manner as the N weight calculation circuit 314 of FIG.

In step S1114, the S weight calculation circuit 1014 calculates the downward weight Ws by the equation (20) in the same manner as the S weight calculation circuit 315 of FIG.

In step S1115, the W weight calculation circuit 1015 calculates the weight Ww in the left direction by the equation (21) in the same manner as the W weight calculation circuit 316 of FIG.

In step S1116, the E weight calculation circuit 1016 calculates the weight We in the right direction by the equation (22) in the same manner as the E weight calculation circuit 317 of FIG.

In step S1117, the synthesis circuit 1017 synthesizes the color difference signals Diff_n, Diff_s, Diff_w and Diff_e based on the weights Wn, Ws, Ww and We, similarly to the synthesis circuit 318 of FIG. Then, the synthesis circuit 1017 calculates the color difference signal Diff_mix by the equation (23).

In step S1118, the addition circuit 1018 adds the color difference signal Diff_mix synthesized by the synthesis circuit 1017 to the image signal output by the WB circuit 201 when the pixel of interest is an R pixel or a B pixel, and the second Outputs a G signal. The output signal of the adder circuit 1018 is the output signal of the second G interpolation circuit 802. As a result of the above, the second G interpolation circuit 802 can calculate a G signal having a high resolution regardless of the direction with respect to the pixel positions of the R pixel or the B pixel.

Next, the synthesis process of the synthesis circuit 804 of FIG. 8 will be described. The synthesis circuit 804 has a first G signal from the first G interpolation circuit 801 and a second G from the second G interpolation circuit 802 based on the color signal intensity α from the red and blue subject detection circuits 803. The signal is combined and the final G signal is output. The G signal output from the synthesis circuit 804 is an output signal of the G interpolation circuit 202. _{The synthesis circuit 804 outputs the final G signal G_sig i, j} when the X coordinate and the Y coordinate of the pixel of interest in the image are (j, i). Specifically, the synthesis circuit 804 uses the first G signal G1_sig _{i, j} , the second G signal G2_sig _{i, j} , and the synthesis coefficient α _{i, j} in the pixel of interest, and the final G signal G_sig _{i , J} is calculated by the following equation (35). That is, the _{larger the synthesis coefficients α i and j} , the higher the ratio of the interpolation result of the first G signal.
G_sig _{i, j} = α _{i, j} × G1_sig _{i, j} + (1.0−α _{i, j} ) × G2_sig _{i, j}
... (35)

When the color signal intensity α is 0.0, that is, when the subject is neither red nor blue nor magenta, the synthesis circuit 804 can generate a second image having a high resolution regardless of the direction. The G signal G2_sig is output as the final G signal G_sig. Further, when the synthesis coefficient α is 1.0, that is, when the subject is red, blue, or magenta, the synthesis circuit 804 finally uses the first G signal G1_sig generated only from the G signal. It is output as a G signal G_sig. This makes it possible to prevent the R and B signals from being folded back due to the small amount of sampling of the R and B signals with respect to the G signal.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

It should be noted that the above embodiments are merely examples of embodiment of the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

305 V color difference calculation circuit, 306 H color difference calculation circuit, 309 red, blue subject detection circuit, 310 N filter circuit, 311 S filter circuit, 312 W filter circuit, 313 E filter circuit

Claims (9)

An image processing device that calculates a G signal for an input image signal that has RGB signals based on the Bayer arrangement in the horizontal and vertical directions.
The first color difference signal calculation means for calculating the color difference signal by vertically interpolating the input image signal, and
A second color difference signal calculating means for calculating the color difference signal by interpolating the input image signal in the horizontal direction,
A first filter processing means that performs a filter process on the color difference signal calculated by the first color difference signal calculation means, and a first filter processing means.
A second filter processing means that performs a filter process on the color difference signal calculated by the second color difference signal calculation means, and a second filter processing means.
And a color signal intensity calculating means for calculating at least one of the intensity of the R and B signals,
The result of the filter processing by the first filter processing means, the result of the filter processing by the second filter processing means, and at least one intensity of the R signal and the B signal calculated by the color signal intensity calculation means. It has a color signal calculation means for calculating the G signal based on the above .
When the first filter processing means and the second filter processing means have at least one intensity of the R signal and the B signal calculated by the color signal intensity calculation means equal to or higher than a preset threshold value, the first filter processing means and the second filter processing means have a preset threshold value or more. An image processing device characterized in that filtering processing is not performed. An image processing device that calculates a G signal for an input image signal that has RGB signals based on the Bayer arrangement in the horizontal and vertical directions.
The first color difference signal calculation means for calculating the color difference signal by vertically interpolating the input image signal, and
A second color difference signal calculating means for calculating the color difference signal by interpolating the input image signal in the horizontal direction,
A first filter processing means that performs a filter process on the color difference signal calculated by the first color difference signal calculation means, and a first filter processing means.
A second filter processing means that performs a filter process on the color difference signal calculated by the second color difference signal calculation means, and a second filter processing means.
And a color signal intensity calculating means for calculating at least one of the intensity of the R and B signals,
The result of the filter processing by the first filter processing means, the result of the filter processing by the second filter processing means, and at least one intensity of the R signal and the B signal calculated by the color signal intensity calculation means. It has a color signal calculation means for calculating the G signal based on the above .
When at least one intensity of the R signal and the B signal calculated by the color signal intensity calculation means is smaller than a preset threshold value, the first filter processing means and the second filter processing means are described. An image processing apparatus characterized by performing filter processing according to at least one intensity of an R signal and a B signal. An image processing device that calculates a G signal by performing interpolation processing on an input image signal that has RGB signals based on the Bayer arrangement in the horizontal and vertical directions.
A first interpolation means for interpolating the G signal without using a color difference signal with respect to the input image signal,
A second interpolation means for interpolating the G signal using the color difference signal with respect to the input image signal,
A color signal intensity calculating means for calculating at least one intensity of an R signal and a B signal, and
Based on the calculation result of the color signal intensity calculating means, and combining means for combining the G signal interpolated by said first G signal interpolated by interpolation means second interpolation means,
Based on the synthesis result by said synthesizing means, have a color signal calculating means for calculating a final the G signal,
The compositing means performs a compositing process so that the higher the intensity of at least one of the R signal and the B signal calculated by the color signal intensity calculating means, the higher the proportion of the interpolation result of the first interpolating means. An image processing device characterized by this. The second interpolation means is
The first color difference signal calculation means for calculating the color difference signal by vertically interpolating the input image signal, and
A second color difference signal calculating means for calculating the color difference signal by interpolating the input image signal in the horizontal direction,
A first filter processing means that performs a filter process on the color difference signal calculated by the first color difference signal calculation means, and a first filter processing means.
It has a second filter processing means for filtering the color difference signal calculated by the second color difference signal calculation means.
The image processing apparatus according to claim 3 , wherein the G signal is interpolated based on the signals processed by the first filter processing means and the second filter processing means. The first filter processing means performs upward and downward filter processing on the color difference signal calculated by the first color difference signal calculation means.
The image according to claim 1 or 2 , wherein the second filter processing means performs left-right and right-direction filtering on the color difference signal calculated by the second color difference signal calculation means. Processing device. It is an image processing method that calculates a G signal for an input image signal that has RGB signals based on the Bayer arrangement in the horizontal and vertical directions.
The first color difference signal calculation step of vertically interpolating the input image signal to calculate the color difference signal, and
A second color difference signal calculation step of interpolating the input image signal in the horizontal direction to calculate the color difference signal, and
A first filter processing step for filtering the color difference signal calculated by the first color difference signal calculation step, and a first filter processing step.
A second filter processing step that filters the color difference signal calculated by the second color difference signal calculation step, and a second filter processing step.
And a color signal strength calculation step of calculating at least one of the intensity of the R and B signals,
The result of the filter processing by the first filter processing step, the result of the filter processing by the second filter processing step, and at least one intensity of the R signal and the B signal calculated by the color signal intensity calculation step. It has a color signal calculation step for calculating the G signal based on the above .
In the first filter processing step and the second filter processing step, when at least one intensity of the R signal and the B signal calculated by the color signal intensity calculation step is equal to or higher than a preset threshold value, An image processing method characterized by not performing filter processing. It is an image processing method that calculates a G signal for an input image signal that has RGB signals based on the Bayer arrangement in the horizontal and vertical directions.
The first color difference signal calculation step of vertically interpolating the input image signal to calculate the color difference signal, and
A second color difference signal calculation step of interpolating the input image signal in the horizontal direction to calculate the color difference signal, and
A first filter processing step that filters the color difference signal calculated by the first color difference signal calculation step, and a first filter processing step.
A second filter processing step that filters the color difference signal calculated by the second color difference signal calculation step, and a second filter processing step.
And a color signal strength calculation step of calculating at least one of the intensity of the R and B signals,
The result of the filter processing by the first filter processing step, the result of the filter processing by the second filter processing step, and at least one intensity of the R signal and the B signal calculated by the color signal intensity calculation step. It has a color signal calculation step for calculating the G signal based on the above .
In the first filter processing step and the second filter processing step, when at least one intensity of the R signal and the B signal calculated by the color signal intensity calculation step is smaller than a preset threshold value, the said. An image processing method comprising performing filter processing according to at least one intensity of an R signal and a B signal. An image processing method for calculating a G signal by performing interpolation processing on an input image signal having an RGB signal based on a Bayer arrangement in the horizontal and vertical directions.
A first interpolation step of interpolating the G signal without using a color difference signal with respect to the input image signal,
A second interpolation step of interpolating the G signal using the color difference signal with respect to the input image signal, and
A color signal intensity calculation step for calculating at least one intensity of the R signal and the B signal, and
A synthesis step of synthesizing the G signal interpolated in the first interpolation step and the G signal interpolated in the second interpolation step based on the calculation result of the color signal intensity calculation step .
Based on said synthesis results by the synthetic steps, have a color signal calculating step of calculating the final the G signal,
In the synthesis step, the synthesis process is performed so that the higher the intensity of at least one of the R signal and the B signal calculated by the color signal intensity calculation step, the higher the ratio of the interpolation result of the first interpolation step. An image processing method characterized by that. A program for making a computer function as each means of the image processing apparatus according to any one of claims 1 to 5.

Images