JPH0823541A - Color image pickup device - Google Patents

Abstract

PURPOSE:To reproduce a correct color tone with less moire and to suppress the generation of a pseudo contour in a color image pickup device such as a video camera or the like. CONSTITUTION:An image pickup sensor 101 where an object image is formed is provided with a color filter array 100 provided with the color filter array of Bayer system, the chrominance signals of R (red), G1 (green), G2 (green) and B (blue) from the image pickup sensor 101 are passed through thinning filters 106-109 and made simultaneous, the difference signals of R signals and G1 signals and G2 signals and B signals are generated by adders 129 and 130 and the difference signals of the R signals and G2 signals and the G1 signals and the B signals are generated by the adders 139 and 140. Then, the respective difference signals for which a difference with the R signals is obtained are inputted to a mixing circuit 150, the respective difference signals for which the difference with the B signals is obtained are inputted to the mixing circuit 151 and R-G and B-G signals are generated by multiplying a coefficiet corresponding to the size of the difference signals and the respective difference signals. After white balance adjustment is performed in a white balance part 145, color difference signals R-Y and B-Y are generated by a color difference matrix processing part 113.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention has an image sensor in which a plurality of light receiving elements (pixels) are arranged in a two-dimensional lattice, and a subject image formed on the light receiving element is provided with brightness information and color information. The present invention relates to a color image pickup device such as a video camera for converting into an electric signal, and more particularly to a color image pickup device capable of outputting an image with less moire and less pseudo contour.

[0002]

2. Description of the Related Art FIG. 10 is a block diagram showing a circuit configuration of a conventionally known color image pickup device for suppressing moire. Imaging light from a subject enters an imaging sensor 101, which is an imaging element, by an imaging optical system (not shown). The image sensor 101 is provided with a color filter array 100 having R (red), G1 (green), G2 (green), and B (blue) color filters in the Bayer array shown in FIG. Then, the image signal read from the image sensor 101 for each pixel is subjected to R, G1,
It is separated into G2 and B color signals. Here, the G1 and G2 signals are the G signals of pixels in the same column as the R and B pixels, respectively, as shown in FIG.

The respective R, G1, G2 and B signals described above are A /
After the A / D conversion is performed by the D (analog-digital) converter 103, each gain of the white balance unit (WB) 111 is white balance based on the color temperature information obtained from the white balance sensor (AWB) 120. Adjusted,
Further, the gamma conversion unit 112 performs gamma conversion.

Further, the luminance signal is a switch circuit (SWY).
The high frequency component Y _H of the luminance signal that has been arranged in the order of reading by being switched by the 126 and passed through the band pass filter (BPF) 116 is added to the low frequency component Y _{L of the} luminance signal obtained by a method described later and the adder 117. In the D / A (digital-analog) converter 118, and is added by the D / A converter 118.
A-converted and output.

On the other hand, the color signal is passed through the A / D converter 103, the white balance section 111 and the γ conversion section 112 to obtain R (γ
Power) signal, G1 (γ power) signal, G2 (γ power) signal, B
(Γ power) signals are interpolated filters 106, 107, 108,
The signals R, G1, and G, which are input to 109 and are synchronized with each other, respectively.
2, B.

The synchronized R and G1 signals become the first difference signal R-G1 in the adder 129, and the R and G2 signals are obtained.
The signal becomes the second difference signal R-G2 at the adder 139, and similarly, the B signal and the G2 signal become the third difference signal B at the adder 130.
-G2, and the B signal and G1 signal are added to the fourth signal by the adder 140.
Difference signal B-G1. And these first, second,
The difference signals of 3 and 4 are input to the determination circuit 141.

The determination circuit 141 compares the magnitudes of the four input difference signals and outputs two difference signals R-GBG (Equation 1) as a result. This determination method is, for example, as follows.

That is, K1 and K2 are given constants, and when R-G is | R-G1 | <| R-G2 | + K1, R-G = R-G1 | R-G1 |> | R-G2 | + K1 When RG = R-G2 (Equation 2) Regarding BG | B-G1 | <| B-G2 | + K2 When BG = B-G1 | B-G1 |> | B-G2 | + K2 At this time, BG = BG2 (formula 3).

Here, it is assumed that a monochrome object having a spatial frequency (1 / 2P _H , 0) is sampled by the image pickup device 101. This subject is vertical stripes having a period of 2P _{H. Since} R signal = G1 signal and B signal = G2 signal for such a subject, R output from the adders 129 and 130
Both -G1 signal and B-G2 signal become zero. Therefore, the determination circuit 141 outputs R−G = R−G1 B−G = B−G2 (Equation 4). Therefore, the color difference signals RY and BY output from the color difference matrix processing unit 113 are also zero. At this time, the carrier of the R signal and the carrier of the G1 signal on the frequency (1 / 2P _H , 0) have the same phase, and the carrier of the B signal and the carrier of the G2 signal have the same phase. Therefore,
Since the carriers of the R-G1 signal and the B-G2 signal at this frequency can be eliminated, the carrier of the color difference signal is not generated, and moire can be suppressed.

Next, it is assumed that a black and white subject having a spatial frequency (0,1 / 2P _V ) is sampled by the image pickup device 101.
This subject is a horizontal stripe with a period of 2P _V. For such a subject, the R signal = G2 signal and the B signal = G1 signal. Therefore, the R−G output from the adders 139 and 140.
Both the 2 signal and the B-G1 signal are zero. Therefore, the determination circuit 141 outputs R-G = R-G2 B-G = B-G1 (Equation 5).

Therefore, the color difference signals RY and BY output from the color difference matrix processing unit 113 are also zero. At this time, the carrier of the R signal and the carrier of the G2 signal have the same phase on the frequency (0,1 / 2P _V ) and the carrier of the B signal and the carrier of the G1 signal have the same phase. Therefore,
Since the carriers of the R-G2 signal and the B-G1 signal at this frequency can be eliminated, the carrier of the color difference signal is not generated and moire is suppressed.

The two color difference signals RG and BG output from the above-mentioned determination circuit 141 are the low-pass filter 13
After a predetermined band limitation is made at 7, 138 and the white balance is adjusted at the white balance unit (WB) 145,
It is input to the color difference matrix processing unit 113. In this color difference matrix processing unit 113,

[0013]

[Equation 1]

The conversion process is performed, and the color difference signal R-
Y, BY are generated. Then, these color difference signals R
-Y and BY are continuously D / A converters 114 and 115.
Is output after being D / A converted.

The interpolation filters 106, 107,
The output signals of 108 and 109 are low-pass filters (LP
F) 135 is limited to a predetermined band and input to the luminance signal generation circuit 127. Then, the low frequency component Y _L of the luminance signal
Is generated by the luminance signal generation circuit 127 by Y _L = 0.30R (αG1 + βG2) + 0.11B (where α + β = 0.59), and as described above, the high frequency component Y of the luminance signal is generated.
_H is added by the adder 117 to obtain a luminance signal Y. The brightness signal Y is D / A converted by the D / A converter 118 and output.

[0016]

However, in the conventional color image pickup apparatus as described above, there are problems in that the correct color tone may not be reproduced and pseudo contours are likely to occur.

That is, with respect to the pixel arrangement shown in FIG. 11, considering the case where the input signal S changes only in the X direction and changes in the X direction as shown in FIG. When the circuit 141 is not present, the G signal is obtained by the arithmetic mean of the G1 and G2 pixels, and therefore the difference signals R-G and B-G.
Becomes RG = R- (G1 + G2) / 2 BG = B- (G1 + G2) / 2 (Equation 8), and the output signal is as shown in FIG.
On the other hand, when the determination circuit 141 is present, the G signal is obtained only by the signal value of the G1 or G2 pixel.
The difference signals R-G and B-G are R-G = R-G1 B-G = B-G2 (Formula 9) from the determination formulas (Formula 2) and (Formula 3), and the difference signals of the G1 signal and the G2 signal are Since the switching process is performed, the difference signals R-G and B-G are outputs in which a part of the signal information for G is missing, and an output having an abrupt image change as compared with the original image as shown in FIG. 12B. There is a problem in that it becomes a signal, and a false contour is likely to be generated, in which the color tone is different or a contour which should not exist is reproduced.

The present invention has been made in view of the above problems, and it is an object of the present invention to provide a color image pickup apparatus having less moire, capable of reproducing a correct color tone, and having less false contour. .

[0019]

The color image pickup device of the present invention is configured as follows.

(1) In a color image pickup device for converting an object image into an electric signal having luminance information and color information, two-dimensional lattice-like pixels having a horizontal pitch Ph and a vertical pitch Pv are formed. And the first color filter, which is provided corresponding to the image sensor in which the light receiving element is arranged and the light receiving element forming the pixel, has a horizontal pitch of 2Ph and a vertical pitch of Pv offset by Pv. The second with a vertical pitch of 2Ph and a vertical pitch of 2Pv
A color filter array having a color filter and a third color filter, the pixel corresponding to the second color filter among the first color signals obtained from the output of the pixel corresponding to the first filter. A first difference signal obtained by simultaneously synchronizing only the signals of the pixels of the same column as the pixels of the second color signal obtained from the output of the first color signal, and the first color signal. Of the pixels of the second color signal, only the signals of the pixels in the same row are simultaneously synchronized to obtain a second difference signal which is the difference from the second color signal, or the luminance obtained from the output of the light receiving element. First color information generating means for generating the color information from the first difference signal and the second difference signal according to the magnitude of the signal;
Of the color signals of the third color signal obtained from the output of the pixel corresponding to the third color filter, only the signals of the pixels in the same column are synchronized, and the difference from the third color signal is calculated. Only the taken third color signal and the signal of the pixel in the same row as the pixel of the third color signal of the first color signal are synchronized and the third color signal is output.
Second color information generation for generating the color information from the third difference signal and the fourth difference signal in accordance with the magnitude of the fourth difference signal or the luminance signal. And means.

(2) In the color image pickup apparatus of (1) above, the first color information generating means uses a table that defines a mixing ratio corresponding to the magnitudes of the first difference signal and the second difference signal. Determining the proportion of mixing the one difference signal with the second difference signal,
The second color information generating means uses the table that defines the mixing ratio corresponding to the magnitudes of the third difference signal and the fourth difference signal to determine the third color information.
The mixing ratio of the difference signal and the fourth difference signal is determined.

(3) In the color image pickup apparatus of (1) above, the first color information generating means determines the difference between the first difference signal and the second difference signal according to the magnitude of the difference signal. The ratio of mixing the 1st difference signal and the 2nd difference signal is determined, and the 2nd color information generating means determines the difference between the 3rd difference signal and the 4th difference signal according to the magnitude of the difference signal. Thus, the ratio of mixing the third difference signal and the fourth difference signal is determined.

(4) In the color image pickup apparatus according to (1), the first color information generating means has the first difference signal and the second difference signal according to the frequency component signal in the horizontal or vertical direction of the subject. And the second color information generating means determines the proportion of mixing the third difference signal and the fourth difference signal in accordance with the horizontal or vertical frequency component signal of the subject. I chose

(5) In the color image pickup device according to any one of (1) to (4), the first color difference signal obtained by the first color information generating means and the first color difference signal obtained by the first color information generating means. A signal obtained by multiplying the second color difference signal by a constant is added to the first
The luminance information generating means for generating the luminance information by adding at least one signal of the color signal, the second color signal, and the third color signal.

(6) In the color image pickup device according to any one of (1) to (5) above, the first color filter has a green light transmissive property, and the second color filter has a red light transmissive property.
The third color filter has blue light transmittance.

(7) In the color image pickup device according to any one of (1) to (5) above, the first color filter has yellow light transmissivity, and the second color filter has cyan light transmissivity. The third color filter was made to have magenta color light transmittance.

(8) In the color image pickup device according to any one of (1) to (7), the first color information generating means and the second color information generating means are provided.
A common circuit is used for the color information generating means.

[0028]

According to the color image pickup apparatus of the present invention, (1 / 2Ph, 0) and (0,1 / 2Pv) in the two-dimensional frequency space are obtained.
The carrier of the color difference signal can be suppressed to reduce moire, and the color tone and the pseudo contour, which have been problems in the past, can be suppressed.

[0029]

1 is a block diagram showing a circuit configuration of a color image pickup apparatus according to a first embodiment of the present invention, showing a configuration of a signal processing system for a luminance signal and a color signal, and the same reference numerals as those in FIG. Indicates an element.

In the image sensor 101, a plurality of light receiving elements are arranged so as to form a two-dimensional lattice-shaped pixel having a horizontal pitch Ph and a vertical pitch Pv.
A first color filter having an offset sampling structure in which the horizontal pitch is 2Ph and the vertical pitch is offset by Pv in correspondence with the light receiving elements that form the pixel, and the horizontal pitch is 2Ph and the vertical pitch is 2Ph. A color filter array 100 having a second color filter and a third color filter having a rectangular grid-like sampling structure with a pitch of 2Pv is provided.

The interpolation filters 106 to 109 are provided as means for synchronizing the color signals, and the adders 129, 130 and 139, 140 are provided as means for obtaining the difference between the color signals. Then, in the first color signal obtained from the output of the pixel corresponding to the first filter, the same column as the pixel of the second color signal obtained from the output of the pixel corresponding to the second color filter Difference signal obtained by synchronizing only the signals of the pixels of the second color signal and the signal of the pixel in the same row as the pixel of the second color signal among the first color signals. Only the first difference signal and the second difference signal according to the magnitude of the luminance signal obtained from the output of the light receiving element. The first color information generating means for generating the color information of the subject from the difference signal is composed of the mixing circuit (1) 150. From the output of the pixel corresponding to the third color filter in the first color signal, Only the signals of the pixels of the same column as the obtained pixels of the third color signal are synchronized with the third color signal. The difference between the third color signal and the third color signal is obtained by synchronizing only the signals of the pixels in the same row as the pixels of the third color signal among the first color signals. The second color information generating means for generating color information of the object from the third difference signal and the fourth difference signal according to the magnitude of the fourth difference signal or the luminance signal is the mixing circuit (2) 151. It is composed by.

The storage unit 155 stores a mixing ratio corresponding to the magnitudes of the first and second difference signals and a mixing ratio corresponding to the magnitudes of the third and fourth difference signals. Is stored, and the mixing circuits 150 and 151 are configured to determine the mixing ratio of the difference signals by the mixing table and generate the color difference signals (color information) RG and BG. Has been done.

Image sensor 101 on which the subject image is formed
The color filter array 100 is composed of Bayer array color filters shown in FIG. 11, the first color filter has a green (G1, G2) light transmissivity, and the second color filter has a red color ( R) is light transmissive, and the third color filter is blue (B) light transmissive.

Then, as in the conventional case, the image signal read out from the image sensor 101 for each pixel is processed by the color separation unit 102.
Are separated into R, G1, G2 and B color signals. Here, the G1 and G2 signals are the G signals of the pixels in the same column as the R and B pixels, respectively.
After the B signal is A / D converted by the A / D converter 103,
Each gain is white-balance adjusted by the white balance unit 111 based on the color temperature information obtained from the white balance sensor 120, and is further γ-converted by the γ conversion unit 112.

The luminance signal is also used in the switch circuit 12 as in the conventional case.
The high frequency component Y _H of the luminance signal that has been arranged in the order of reading by being switched by 6 and passed through the band pass filter 116 is added to the low frequency component Y _L of the luminance signal to be described later by the adder 117, and the enhancement circuit 142 is added. After being enhanced by, the signal is D / A converted by the D / A converter 118 and output.

On the other hand, the color signal is passed through the A / D converter 103, the white balance section 111 and the γ conversion section 112 to obtain R (γ
Power) signal, G1 (γ power) signal, G2 (γ power) signal, B
(Γ power) signals are interpolated filters 106, 107, 108,
The signals R, G1, and G, which are input to 109 and are synchronized with each other, respectively.
2, B. The interpolation filters 106 to 109 preferably perform pre-interpolation in which the signal values of pixels that are adjacent in the horizontal and vertical directions are kept as they are, or one that does not drop the horizontal and vertical bands so much. Methods or a combination thereof may be performed.

The synchronized R signal and G1 signal become the above-mentioned first difference signal R-G1 in the adder 129, and the R signal and G2 signal become the second difference signal R-G2 in the adder 139. Similarly, the B signal and the G2 signal become the third difference signal B-G2 at the adder 130, and the B signal and the G1 signal become the adder 14 respectively.
At 0, the fourth difference signal B-G1 is obtained. Then, these first and second difference signals are input to the mixing circuit 150, and the third and third
The fourth difference signal is input to the mixing circuit 151.

In the mixing circuit 150, based on the values obtained from the mixing table stored in the storage unit 155 from the magnitudes of the values of the input first difference signal R-G1 and second input difference signal R-G2. , And the coefficients a and b to be multiplied respectively, and from these coefficients and the difference signals R-G1 and R-G2, RG = a (R-G1) + b (R-G2) (a + b = 1) (Equation 10) The RG signal is output as.

In the mixing circuit 151, a coefficient to be applied to each of the input third difference signal B-G2 and fourth difference signal B-G1 based on the value obtained from the above-mentioned mixing table is calculated. c and d are determined, and from these coefficients and the difference signals B-G2 and B-G1, B-G = c (B-G1) + d (B-G2) (c + d = 1) (Equation 11) Output G signal.

At this time, by appropriately setting the values of the mixing table, the switching processing is hardly performed for an image with a small change in color, and the mixing circuits 150 and 151 respectively have RG≈R- (G1 + G2) /. 2B−G≈B− (G1 + G2) / 2 (Equation 12) By performing the output, it is possible to suppress abrupt changes in color tone and prevent pseudo contours from occurring, and also abrupt color changes. In some cases, by switching the difference signals R-G1, R-G2 or B-G1, B-G2 according to the magnitude of the difference, moire can be suppressed and correct color tone can be reproduced. it can.

The two color difference signals RG and BG output from the mixing circuits 150 and 151 are subjected to predetermined band limitation by the low pass filters 137 and 138, respectively, and white balance adjusted by the white balance section 145. After that, it is input to the color difference matrix processing unit 113. In this color difference matrix processing unit 113,

[0042]

[Equation 2]

Then, the color difference signal R-
Y, BY are generated. Then, these color difference signals R
-Y and BY are subsequently D / A converted by the D / A converters 114 and 115 and output.

The interpolation filters 106, 107,
The output signals of 108 and 109 are low-pass filter 135.
Is limited to a predetermined band and is input to the luminance signal generation circuit 127. Then, the low-frequency component Y _L of the luminance signal is generated by the luminance color difference signal generation circuit 127 as follows: Y _L = 0.30R (αG1 + βG2) + 0.11B (where α + β = 0.59) (Equation 14) As described above, the high frequency component Y of the luminance signal
_H is added by the adder 117 to obtain a luminance signal Y. The brightness signal Y is D / A converted by the D / A converter 118 and output.

Since the color difference signals R-Y, B-Y and the low-frequency component Y _L of luminance are generally narrower in band than the luminance signal Y, the synchronized R signal, G1 signal, G2 signal, and B signal are mixed. In the processing by the color difference matrix processing unit 113, the luminance color difference signal generation circuit 127, etc., the luminance signal Y
It may be performed with a clock slower than the processing of. The white balance 145 may be omitted.

Next, a second embodiment of the present invention will be described. The second embodiment differs from the above-described first embodiment in the method of generating the two difference signals R-G and B-G, and the magnitude of the difference signal obtained by taking the difference between the two difference signals of the color signals. The mixing ratio of the two difference signals is determined according to FIG. 2 shows the circuit configuration of the signal processing system. The same components as those in FIG. 1 are designated by the same reference numerals and detailed description thereof will be omitted.

The first difference signal R-G1 and the second difference signal R-G2 respectively generated by the adders 129 and 139 are added by the adder 160 as K1 = (R-G2)-(R-G1) ( It becomes the difference signal of Expression 15) and is input to the determination circuit (1) 165.
In the determination circuit 165, the input difference signal K1 = (R−G
2) The coefficients a and b to be multiplied are determined from the size of (R-G1). Then, this determination circuit 165
For example, if K1 <C1, a = a1 b = b1 If C1 ≦ K1 <C2, a = a2 b = b2 If C2 ≦ K1 <C3, a = a3 b = b3 If C3 ≦ K1 <C4, a = a4 b = b4 (Equation 16) The following determination is performed. Where C1, C2, C3
C4, a1, a2, a3, a4, b1, b2, b3, b
4 is an arbitrary constant.

The mixing circuit 150 includes a first difference signal R-G.
From the first and second difference signals R-G2 and the coefficients a and b obtained from the determination circuit 165, R-G = a (R-G1) + B (R-G2) (a + b = 1) (Equation 17) To output the R-G signal.

Further, the third difference signal B-G2 and the fourth difference signal B-G1 generated by the adders 130 and 140, respectively.
Is a difference signal of K2 = (B-G1)-(B-G2) (Equation 18) in the adder 161, and is input to the determination circuit (2) 166.
In the determination circuit 166, the input difference signal K2 = (B−G
1)-(B-G2), the coefficients c and d to be multiplied are determined. Then, this determination circuit 165
For example, if K2 <1, c = c1 d = d1 D1 ≦ K2 <D2 c = c2 d = d2 D2 ≦ K2 <D3 c = c3 d = d3 D3 ≦ K2 <D4 c = c4 d = d4 (Formula 19) A determination such as the following is performed. Where D1, D2, D3, D
4, c1, c2, c3, c4, d1, d2, d3, d4
Is an arbitrary constant.

The mixing circuit 151 has a third difference signal B-G.
2, BG = c (B-G1) + d (B-G2) (c + d = 1) (Equation 20) from the second and fourth difference signals B-G1 and the coefficients c and d obtained from the determination circuit 166. To output a B-G signal.

By using the decision circuits 165 and 166 and the mixing circuits 150 and 151 in this manner, as in the first embodiment, the switching processing is hardly performed for an image with a small change in color, and the two difference signals are used. To suppress abrupt changes in color tone, and prevent pseudo contours from occurring,
When there is a rapid color change, the difference signal can be switched according to the magnitude of the color change to suppress moire.

Next, a third embodiment of the present invention will be described. The third embodiment has substantially the same configuration as the above-described second embodiment, but is different in the determination method associated with the generation of the two difference signals RG and BG. FIG. 3 shows the circuit configuration of the signal processing system, and the same components as those in FIG. 1 are designated by the same reference numerals and detailed description thereof will be omitted.

The judging circuit 159 judges the ratio of mixing the above two difference signals based on the luminance signal information. That is, the luminance signal obtained from the switch circuit 126 is subjected to a bandpass filter to extract a band component, and the magnitudes thereof are used to multiply the first difference signal R-G1 and the second difference signal R-G2 by the coefficients a, b ( a + b = 1) and the third difference signal B-
Coefficients c and d (c +) for multiplying G2 and the fourth difference signal B-G1
Determine d = 1).

The mixing circuit 150 includes a first difference signal R-G.
From the first and second difference signals R-G2 and the coefficients a and b obtained from the determination circuit 159, R-G = a (R-G1) + b (R-G2) (a + b = 1) (Equation 21) Output RG signal.

Further, the mixing circuit 151 has a third difference signal B-
G2, the fourth difference signal B-G1, and the coefficients c and d obtained from the determination circuit 159, BG = c (B-G1) + d (B-G2) (c + d = 1) (Equation 22) To output a B-G signal.

Next, a fourth embodiment of the present invention will be described. FIG. 4 shows the circuit configuration of the signal processing system of the present embodiment. The same components as those in FIG. 1 are designated by the same reference numerals and detailed description thereof will be omitted.

The image pickup light from the subject enters the image pickup sensor 101 by the image forming optical system.
Is a complementary color filter such as a cyan light transmission filter Cy, a yellow (yellow) light transmission filter Ye, and a magenta light transmission filter Mg arranged in a Bayer array.
A color filter array 100 including e1, Ye2, and Mg color filters is provided. Then, the image sensor 10
The image signal read out for each pixel from 1 is separated into color signals of Cy, Ye1, Ye2, and Mg by the color separation unit 102. Here, the Ye1 and Ye2 signals are Ye signals of pixels in the same column as the Mg and Cy pixels, respectively, as shown in FIG. 5, and these Cy, Ye1, Ye2, and Mg signals are A / D converters, respectively. A / D conversion is performed at 103.

The high-frequency components Y _H of the luminance signal are arranged in the order of reading by being switched by the switch circuit 126, pass through the band pass filter 116, and then white balance is adjusted by the white balance (WB) 122. Is converted to γ by the γ conversion unit 121. Then, the high frequency component Y _H of the luminance signal is added to the low frequency component Y _L of the luminance signal by the adder 117 as described later, enhanced by the enhancement circuit 142, and then D / A converter 118. / A converted and output.

On the other hand, as the color signal, the Cy signal, the Ye1 signal, the Ye2 signal, and the Mg signal from the A / D converter 103 are input to the interpolation filters 106, 107, 108, and 109, and the synchronized signals Cy and Ye1 are input. , Ye2, Mg, and the synchronized signals Cy, Ye1, Ye2, Mg
Are input to the difference signal generation circuit 170 shown in FIG. Here, the Cy signal and the Ye1 signal become the first difference signal Cy-Ye1 in the adder 129, the Cy signal and the Ye2 signal become the second difference signal Cy-Ye2 in the adder 139, and the Mg signal and the Ye2 signal become The third difference signal Mg-Y is added by the adder 130.
e2, and the Mg signal and the Ye1 signal become the fourth difference signal Mg-Ye1 in the adder 140. And these first
Difference signal, the second difference signal, the third difference signal, and the fourth difference signal are input to the determination circuit 172.

In the judgment circuit 172, the first difference signal C
Based on the magnitudes of the values of y-Ye1, the second difference signal Cy-Ye2, the third difference signal Mg-Ye2, and the fourth difference signal Mg-Ye1, the embodiment of FIG. 1 stored in the storage unit 171. Based on the values obtained from the similar mixing table, Ye1 and Ye
The coefficients a and b (a + b = 1) to be multiplied by 2 are determined. Then, the mixing circuit 173 in which the first color information generating means and the second color information generating means described above are configured by using a common circuit, the coefficient a, b and the values of Ye1 and Ye2 obtained by the determination circuit 172 are used. Then, the Ye signal is output as Ye = aYe1 + bYe2 (a + b = 1) (Equation 23).

Next, in the primary color separation matrix section 180, C
R, G, B signals are obtained from the y, Ye, and Mg signals. For this color conversion, for example,

[0062]

(Equation 3)

The conversion matrix processing as described above is performed.
Here, s, t, and u are arbitrary constants.

The R, G, and B signals separated by the primary colors are
Each gain is subjected to white balance adjustment by a white balance unit (WB) 181, and then subjected to γ conversion by a γ conversion unit 182. This γ-converted signal is passed through the low-pass filter 1
After a predetermined band limitation is made at 85, it is inputted to the matrix processing unit 186,

[0065]

[Equation 4]

Then, the color difference signal R-
Y, BY are generated. These color difference signals are subsequently D / A converted by the D / A converters 114 and 115 and output.

The output signal of the γ conversion section 182 is
It is limited to a predetermined band by the low pass filter 128 and input to the luminance signal generation circuit 127. Then, the low frequency component Y _L of the luminance signal is generated by the luminance color difference signal generation circuit 127 according to Y _L = 0.30R + 0.59G + 0.11B (Equation 26), and as described above, the high frequency component Y _L of the luminance signal is generated.
_H is added by the adder 117 to obtain a luminance signal Y. The brightness signal Y is D / A converted by the D / A converter 118 and output.

Since the color difference signals RY and BY and the low-frequency component Y _L of luminance are generally narrower in band than the luminance signal Y, the synchronized Cy signal, Ye1 signal, Ye2 signal, and Mg signal are synchronized. The processing in the color difference matrix processing unit 113, the luminance color difference signal generation circuit 127, and the like may be performed by a clock slower than the processing of the luminance signal Y by thinning out. Further, the color filter array shown in FIG. 7 may be used instead of the color filter array shown in FIG.

In this embodiment, since a complementary color filter is used as the color filter, there is an advantage that the sensitivity is better than that in the first embodiment.

Next, a fifth embodiment of the present invention will be described. The fifth embodiment has substantially the same configuration as the above-mentioned fourth embodiment, but the method of generating the Ye signal is different. FIG.
The circuit configuration of the signal processing system is shown in FIG. 4, and the same parts as those in FIG.

The first difference signal Cy-Ye1 and the second difference signal Cy-Y respectively generated by the difference signal generation circuit 170.
e2 and the third difference signal Mg-Ye2 and the fourth difference signal Mg
-Ye1 is (Cy-
Ye2)-(Cy-Ye1) signal and (Mg-Ye1)-
It becomes a (Mg-Ye2) signal and is input to the determination circuit 193. In this determination circuit 193, (Cy-Ye2)-
(Cy-Ye1) signal and (Mg-Ye1)-(Mg-Y
e2) Coefficients a and b to be multiplied by the signal magnitude
Is determined. Then, the mixing circuit 194 configured similarly to the mixing circuit 173 of FIG. 4 uses Ye1 and Ye2 and the coefficients a and b obtained from the determination circuit 193 as follows: Ye = aYe1 + Ye2 (a + b = 1) (Equation 27) Output a signal.

Also in the fifth embodiment, the color filter array shown in FIG. 7 may be used in addition to the color filter array shown in FIG. 5 as in the fourth embodiment. At this time, Ye
If the roles of the filter and the Cy filter are exchanged, the same signal processing system as in FIGS. 4 and 8 can be implemented. Further, in the fifth embodiment, it is possible to make a determination such that the luminance signal is used as the determination condition as in the third embodiment of FIG.

Next, a sixth embodiment of the present invention will be described. The sixth embodiment is different from the first embodiment in FIG. 1 in the method of forming a low-frequency luminance signal. FIG. 9 shows the circuit configuration of the signal processing system. The same components as those in FIG. 1 are designated by the same reference numerals and detailed description thereof will be omitted.

The luminance signals arranged in the order of reading by being switched by the switch circuit 126 are band-limited by the low-pass filter (LPF) 200 and input to the adder 117 as a luminance signal Y _G containing high frequency components.
Further, the two difference signals RG and BG band-limited by the low pass filters 137 and 138 are converted into the coefficient multiplier 201,
The constants S1 and S2 are multiplied by 202, respectively, and further added by the adder 203, and are input to the adder 117 as the luminance signal Y _L by Y _L = S1 (R−G) + S2 (B−G) (Equation 28). It
Then, in the adder 117, the luminance signal Y _L and the luminance signal Y
_G is added to form the luminance signal Y. Such processing has an advantage that the circuit scale for forming the luminance signal can be simplified.

The embodiments of the present invention have been described above. In each of the embodiments, a Bayer type color filter array is adopted, and the mixing ratio of each difference signal is determined by the frequency component in the horizontal or vertical direction of the subject. Since such signal processing is performed, (1 / 2P in the two-dimensional frequency space
The carriers of the color difference signals of (h, 0) and (0, 1/2 Pv) can be suppressed, the moire can be reduced, and the color tone and the pseudo contour which have been problems in the past can be suppressed.

In each of the above-described embodiments, the case where R, G, B color filters are used as the color combination of the Bayer array color filters and the case where Ye, Mg, Cy color filters are used are shown. However, it goes without saying that a Bayer array using color filters of other color combinations may be used.

Needless to say, the present invention can be applied even if the outputs are digital R, G, B outputs.

[0078]

As described above, according to the present invention,
The image sensor is provided with a Bayer array color filter array,
Since the difference signal of each color signal from the pixel of each color filter is formed and the color information of the subject is generated according to the magnitude of the difference signal or the luminance signal, moire can be suppressed and correct. The color tone can be reproduced, and the pseudo contour can be suppressed.

[Brief description of drawings]

FIG. 1 is a block diagram showing a circuit configuration of a first embodiment of the present invention.

FIG. 2 is a block diagram showing a circuit configuration of a second embodiment of the present invention.

FIG. 3 is a block diagram showing a circuit configuration of a third embodiment of the present invention.

FIG. 4 is a block diagram showing a circuit configuration of a fourth embodiment of the present invention.

FIG. 5 is an explanatory diagram showing an example of a color filter array in the fourth embodiment.

FIG. 6 is a configuration diagram showing details of a difference signal generation circuit.

FIG. 7 is an explanatory diagram showing another example of a color filter array in the fourth embodiment.

FIG. 8 is a block diagram showing a circuit configuration of a fifth embodiment of the present invention.

FIG. 9 is a block diagram showing a circuit configuration of a sixth embodiment of the present invention.

FIG. 10 is a block diagram showing a circuit configuration of a conventional example.

FIG. 11 is an explanatory diagram showing an example of a Bayer array color filter array.

FIG. 12 is an explanatory diagram showing the principle of pseudo contour generation.

[Explanation of symbols]

 100 Color Filter Array 101 Image Sensor 102 Color Separation Unit 106 Interpolation Filter 107 Interpolation Filter 108 Interpolation Filter 109 Interpolation Filter 113 Color Difference Matrix Processing Unit 117 Adder 126 Switch Circuit 127 Luminance Signal Generation Circuit 129 Adder 130 Adder 139 Adder 140 Addition Device 150 Mixing Circuit (First Color Information Generating Means) 151 Mixing Circuit (Second Color Information Generating Means) 155 Storage Unit 159 Judgment Circuit 160 Adder 161 Adder 165 Judgment Circuit 166 Judgment Circuit 170 Difference Signal Generation Circuit 171 Memory Part 172 Judgment Circuit 173 Mixing Circuit 191 Adder 192 Adder 193 Judgment Circuit 194 Mixing Circuit 201 Coefficient Multiplier 202 Coefficient Multiplier 203 Adder

Claims (8)

[Claims] 1. A color image pickup device for converting an object image into an electric signal having luminance information and color information so as to form a two-dimensional lattice-like pixel having a horizontal pitch Ph and a vertical pitch Pv. The light-receiving element is arranged in the image sensor and the light-receiving element forming the pixel is provided corresponding to the horizontal pitch of 2Ph and the vertical pitch of P.
a color filter array having a first color filter offset by v, a second color filter having a horizontal pitch of 2Ph and a vertical pitch of 2Pv, and a third color filter, and the first filter Among the first color signals obtained from the output of the pixel corresponding to the above, only the signals of the pixels in the same column as the pixels of the second color signal obtained from the output of the pixel corresponding to the second color filter are simultaneously obtained. Of the first color signal and the signal of the pixel in the same row as the pixel of the second color signal of the first color signal, The color information from the first difference signal and the second difference signal according to the magnitude of the second difference signal which is the difference from the second color signal or the luminance signal obtained from the output of the light receiving element. A first color information generating means for generating the third color signal of the first color signal A third color signal obtained by synchronizing only the signals of the pixels in the same column as the pixels of the third color signal obtained from the output of the pixels corresponding to the color filter, and taking the difference from the third color signal; Of the first color signals, only the signals of the pixels in the same row as the pixels of the third color signal are synchronized, and a fourth difference signal is obtained by taking the difference from the third color signal, or the luminance signal. And a second color information generating means for generating the color information from the third difference signal and the fourth difference signal according to the magnitude of the color difference. 2. The first color information generating means determines the first difference signal and the second difference signal from a table that defines a mixing ratio corresponding to the magnitudes of the first difference signal and the second difference signal. The mixing ratio is determined, and the second color information generating means uses the table that defines the mixing ratio corresponding to the magnitudes of the third difference signal and the fourth difference signal to determine the third difference signal and the fourth difference signal. The color image pickup device according to claim 1, wherein a ratio of mixing the signals is determined. 3. The first color information generating means generates the first difference signal and the second difference signal according to the magnitude of the difference signal which is the difference between the first difference signal and the second difference signal. The mixing ratio is determined, and the second color information generating means determines the third difference signal and the fourth difference signal according to the magnitude of the difference signal which is the difference between the third difference signal and the fourth difference signal.
2. The color image pickup apparatus according to claim 1, wherein a ratio of mixing the difference signals of 1 is determined. 4. The first color information generating means determines a mixing ratio of the first difference signal and the second difference signal in accordance with a frequency component signal in a horizontal direction or a vertical direction of the subject, and a second ratio signal is generated. 2. The color image pickup according to claim 1, wherein the color information generating means determines a mixing ratio of the third difference signal and the fourth difference signal in accordance with a horizontal or vertical frequency component signal of the subject. apparatus. 5. A first color information obtained by the first color information generating means.
Of the first color signal and the second color difference signal obtained by the first color information generating means by a constant multiple.
5. A brightness information generating means for generating brightness information by adding at least one signal of the second color signal and the third color signal.
The color imaging device according to any one of claims. 6. The first color filter has a green light transmissive property, the second color filter has a red light transmissive property, and the third color filter has a blue light transmissive property. The color image pickup apparatus according to claim 1, wherein the color image pickup apparatus is a color image pickup apparatus. 7. The first color filter has a yellow light transmissive property, the second color filter has a cyan light transmissive property, and the third color filter has a third light transmissive property.
6. The color image pickup device according to claim 1, wherein the color filter has a magenta color light transmissive property. 8. The color image pickup apparatus according to claim 1, wherein the first color information generating means and the second color information generating means use a common circuit.