JPH05244611A - Solid-state color image pickup device - Google Patents

Abstract

PURPOSE:To obtain a solid-state color image pickup device which can improve the resolution of a chrominance signal in horizontal and vertical directions with keeping the resolution of a luminance signal high. CONSTITUTION:At least three kinds of light-receiving elements different in a sensitive spectrum characteristic are arranged in the horizontal and vertical directions. Two light-receiving element signals adjacent in the vertical direction are added, the combination of addition is alternately shifted by one element in the vertical direction at every field and respective addition signals are sequentially read so as to form an output signal. A omitted signal component in said output signal is interpolated by a signal obtained by delaying the output signal by one field.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a single-chip color image pickup device for obtaining a color video signal by using one solid-state image pickup device having filters arranged two-dimensionally, and more particularly to a means for increasing resolution in both horizontal and vertical directions. It is a thing.

[0002]

2. Description of the Related Art In order to color a solid-state image sensor with a single plate, a normal color filter is arranged on the light-receiving surface of the image sensor, and each color filter and pixel are made to correspond to each other to perform spatial sampling separately for each color component. Is used. In this case, the resolution of each component signal forming the color video signal greatly changes depending on the color filter array and the signal processing method. In addition,
Since the video signal normally performs interlaced scanning, the following description assumes that the image sensor is interlaced readout.

In order to obtain a high resolution, a well-known color filter array has a green (G) filter, which occupies most of the luminance signal for which a visually high resolution is required, arranged in a checkered pattern, and a red color at the remaining positions. The (R) filter and the blue (B) filter are arranged at a constant cycle. In the Bayer array shown in FIG. 1A, the R filter and the B filter alternate in the scanning line cycle in the vertical direction, and In the interline arrangement shown in (), the R filters and the B filters alternate in the horizontal direction every two vertical columns. In these cases, a high resolution can be obtained for the luminance signal, but the resolution of the color signal is insufficient for both the R signal and the B signal. That is, in the Bayer array, both the R signal and the B signal are alternately missing every horizontal scanning period (1H), and therefore it is necessary to interpolate by the signal delayed by 1H, but the vertical resolution is lowered. In particular, due to interlaced readout, 1/2 of the vertical Nyquist limit frequency
Even at, the response is zero. On the other hand, in the interline array, the resolution of both the R and B signals does not deteriorate in the vertical direction, but the resolution is low in the horizontal direction, and the response becomes 0 at 1/2 of the horizontal Nyquist limit frequency.

The above is the case of using the three primary color filters of G, R and B, but basically the same problem exists even if a complementary color filter is used. For example, FIGS. 1 (a) and 1 (b)
, G → W = G + R + B (transparent), R → Cy = G
Consider the case of + B (cyan color) and B → Ye = G + R (yellow). At this time, a high resolution signal is obtained as the G signal as the base band component, and the R and B signals are obtained as the modulation components. However, the spatial area in which the R and B signals are modulated is the same as the spatial area of the R and B pixels in the three primary color filter. Therefore, the same problem as described above occurs with respect to resolution.

[0005]

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and provides a solid-state color image pickup device capable of increasing the resolution of a color signal in both horizontal and vertical directions while keeping the resolution of a luminance signal high. To do.

[0006]

According to the solid-state color image pickup device of the present invention, a first light receiving element, a second light receiving element and a third light receiving element having different sensitive spectral characteristics, and the same sensitive spectral characteristic as the second light receiving element. And a fourth light receiving element having
In the first horizontal row, the first light receiving elements and the second light receiving elements are alternately arranged, and in the second horizontal row, the third light receiving elements and the fourth light receiving elements are alternately arranged. Further, in the third horizontal row, the same light receiving element set is arranged with a phase shift of 180 ° with respect to the first horizontal row, and in the fourth horizontal row, the same light receiving element set is phased with respect to the second horizontal row. 18
The solid-state image sensor, which is arranged by shifting 0 ° and the four horizontal columns are sequentially repeated, and two light-receiving element signals that are vertically adjacent to each other are added, and the combination of addition is alternated one element in the vertical direction for each field. The addition signals are sequentially read out by shifting the addition signals to form a solid-state imaging device output signal, and the solid-state imaging device output signal output from the reading device is summed to obtain a sum of horizontally adjacent signals,
A real-time luminance signal forming circuit that forms a real-time luminance signal;
A real-time color signal forming circuit that forms a real-time color signal by taking a difference between horizontally adjacent signals from the solid-state image sensor output signal, and a field delay output signal obtained by delaying the solid-state image sensor output signal by one field. Therefore, a field delay color signal forming circuit for forming a field delay color signal by taking a difference between horizontally adjacent signals is provided. In addition, it comprises a first light receiving element, a second light receiving element and a third light receiving element having different sensitive spectrum characteristics, and a fourth light receiving element having the same sensitive spectrum characteristic as the second light receiving element, In the first horizontal row, the first light receiving elements and the second light receiving elements are alternately arranged, and in the second horizontal row, the third light receiving elements and the fourth light receiving elements are alternately arranged. Further, in the third horizontal row, the same light receiving element group has the same phase as that of the first horizontal row.
A solid-state imaging device in which the fourth horizontal row is arranged by shifting by 80 °, the same light receiving element set is arranged by shifting the phase by 180 ° with respect to the second horizontal row, and the four horizontal rows are sequentially repeated. , Add two adjacent photodetector signals in the vertical direction, and set the combination of addition to 1 in the vertical direction for each field.
The elements are alternately shifted, and the addition signals are sequentially read to form a solid-state image sensor output signal, and from the solid-state image sensor output signal output from the read unit,
A real-time luminance signal forming circuit that forms a real-time luminance signal by taking the sum of horizontally adjacent signals and the difference between horizontally adjacent signals from the output signal of the solid-state imaging device is calculated to obtain a real-time color signal. A field-delayed luminance signal for forming a field-delayed luminance signal by taking a sum of horizontally adjacent signals from a real-time color signal forming circuit to be formed and a field-delayed output signal obtained by delaying the output signal of the solid-state imaging device by one field. A forming circuit and a field delay color signal forming circuit for forming a field delay color signal by subtracting a signal adjacent in the horizontal direction from a field delay output signal obtained by delaying the output signal of the solid-state image sensor by one field. The field delayed luminance signal is subtracted from the time luminance signal to form a color correction signal, and the color correction signal is added to the field delayed color signal. By and is characterized by comprising providing a correction field delay chrominance signal forming circuit for forming a corrected field delay chrominance signal.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the case of FIG. 1A, the horizontal resolution of both the R signal and the B signal has a great advantage. Therefore, if there is a method to prevent deterioration of vertical resolution, high resolution can be obtained in both horizontal and vertical directions. As shown in FIG. 2, assuming that the vertical repeating unit of the filter array is one pixel and the R signal and the B signal are interpolated by the signal one field before, the vertical resolution of the R signal and the B signal is doubled. However, in this case, the period for reading out the signal of each pixel is one frame (two fields) as shown in FIG. 3A, which is a signal integrated for one frame period. Therefore, a time difference of up to 3 fields occurs between the real-time signal and the signal delayed by one field period (1F), and when capturing a moving optical image, the interpolated color signal is significantly deviated from the real signal. As a result, the color breakup phenomenon becomes remarkable.

In order to solve the above problem, it is sufficient to perform field reading in which the integration period of the optical image is half of one field and reduce the time difference between the real-time signal and the 1F delay signal. As a result, as shown in FIG. 3B, the time difference between the real-time signal and the 1F delay signal is at most 2 fields, and the integration period is halved, so that the time function is improved. Further, even in the field reading, the false signal associated with the field interpolation which still exists can obtain the correction signal easily, as described later, so that the color splitting phenomenon can be substantially eliminated.

FIGS. 4A and 4B show an example of a color filter array related to the present invention which enables field reading and field interpolation. These both use four types of picture elements, one of which has the spectral characteristic of the other three types added or has a spectral characteristic which is a constant multiple thereof, and arranged in a repeating unit of every four horizontal rows. It is a thing. Ma is a magenta filter composed of R + B. Further, the signal processing and the image quality are the same even if the arrangement is switched between Ye and Cy in FIG. 4A and between B and R in FIG. 4B. 4A and 4B, in the odd field, the upper and lower two pixel signals are added between the even-numbered horizontal column and the odd-numbered horizontal column above it, and in the even-numbered field, the even-numbered horizontal column and the odd-numbered horizontal column below it. The upper and lower two pixel signals are added between the columns. As a result, each pixel signal is taken out in the field cycle and the field is read out. Further, at this time, field interpolation becomes useful as follows. In FIG. 4A, R- (G + B) in the odd field
The signal is modulated at the Nyquist limit frequency (f _N ) and 3
The (R + G + B) signal becomes the base band signal. Band of the base band signal is present to f _N, but slightly narrower bandwidth than f _N for removing a modulation component present in the vicinity f _N.
Even in this case, the band is sufficiently wide and a high resolution signal is obtained.
In the even field, the B- (G + R) signal is modulated with f _N , and the 3 (R + G + B) signal becomes the base band signal. From the above, R + G + B having a wider band than 1/3 of the baseband signal
The luminance signal is always obtained, and by adding the luminance signal to the modulation signal and halving the luminance signal, the R signal and the B signal with high horizontal resolution are alternately obtained in the field cycle. Therefore, by performing the field interpolation by the 1F delay circuit, the R signal and the B signal of high horizontal resolution can always be obtained without deteriorating the vertical resolution.

In FIG. 4B, 2R is used in the odd field.
The signal is modulated with f _N and the 2 (R + G + B) signal becomes the base band signal. Similarly, in the even field, the 2B signal is modulated by f _N and the 2 (R + G + B) signal becomes the base band signal. Therefore, R + G + B from 1/2 of the base band signal
The luminance signal is always obtained. On the other hand, since the R signal and the B signal are alternately obtained from 1/2 of the modulation signal in the field cycle, the R signal and the B signal with high resolution can be always obtained in the horizontal and vertical directions by performing the field interpolation by the 1F delay circuit. become.

Even in the field reading described above,
When field interpolation is performed, which is less than that of frame reading, a false signal is generated for a moving optical image. That is, for example, in FIG.
When the white light image temporally fluctuates as shown in FIG. 5, the R signal is lost in the field number 2 and the white image becomes cyan, and in the field number 5, the B signal becomes redundant and the black image becomes blue. .. This phenomenon is resolved as follows. Since the luminance signal is obtained in real time in each field, the luminance 1F delay signal Y (1F) is subtracted from the luminance real-time signal Y (0F) to obtain the field correction signal Y (0F).
-Y (1F) is formed. When this field correction signal is added to the field-delayed color signal, the false signal is just deleted and a normal color signal is obtained.

FIG. 6 shows a circuit block diagram for realizing the above-mentioned example, which is suitable for the color filter array of FIG. 4 (a). First, a signal from the solid-state image sensor 1 having the color filter shown in FIG. 4A is branched, and one of the signals is directly guided to the sample hold circuits 3, 4 to 5 and 6 through the 1F delay circuit 2. .. In the odd field, the circuit 3 has Ye + Ma, the circuit 4 has Cy + w,
Cy + Ma is sampled and held by the circuit 5 and Ye + W signal is sampled and held by the circuit 6, respectively.
+ Ma, the circuit 4 samples and holds the Ye + W signal, the circuit 5 samples and holds the Ye + Ma signal, and the circuit 6 samples and holds the Cy + W signal. One of the signals from the circuits 3 and 4 is added and multiplied by ⅓ to obtain an R + G + B signal 7, and the other is obtained by subtracting the output of the circuit 4 from the output of the circuit 3 to obtain R−C in the odd field.
A signal 8 of B-Ye is obtained in the y and even fields. Similarly, from the output signals of the circuits 5 and 6, addition and 1 /
Signal 1 which becomes R + G + B signal by 3 times and becomes B-Ye in odd field and R-Cy in even field by subtraction
0 is obtained.

Here, the addition between the sample and hold signals is the same as taking the sum of the pixel signals which are adjacent to each other in the horizontal direction, and is a signal obtained by removing the component near f _N from the base band signal. Further, the subtraction between the sample and hold signals is the same as the difference between the pixel signals that are adjacent to each other in the horizontal direction being sequentially obtained, and is the same as the case where the modulation component in the vicinity of f _N is extracted and the low frequency conversion is performed. Now, the signal 7 has its band limited to f _N / 2 by the low-pass conversion filter 11, and then the signal 8
, And further multiplied by 1/2 to obtain a signal 13 which becomes R in the odd field and B in the even field. Similarly, the band of the signal 9 is set to f _N / by the low-pass filter 12.
After being limited to 2, it is added to the signal 10 and further multiplied by 1/2 to obtain a signal 14 which becomes B in the odd field and R in the even field. On the other hand, the outputs of the branched low-pass filters 11 and 12 are subtracted from the former by the latter, adjusted to an appropriate size, and the field correction signal 15 is obtained.
Is obtained. By adding this signal 15 to the signal 14, a signal 16 in which a false signal due to field interpolation is eliminated is obtained. The signals 13 and 16 are distributed to the R signal only and the B signal only by the changeover switch 17 which alternates for each field, and the gain is adjusted to obtain the R signal output and the B signal output, respectively. The luminance signal Y is obtained by adjusting the gain of the signal 7.

FIG. 7 shows an example of a circuit block diagram related to the present invention, which is compatible with the color filter array of FIG. 4 (b). First, the signal from the solid-state imaging device 1 including the color filter shown in FIG. 4B is branched, and one of them is directly connected to the sample hold circuit 18 via the 1F delay circuit 2.
You will be led to 19 to 20, 21. In the odd field, the circuit 18 has B + G, the circuit 19 has R + W, and the circuit 20 has R + G.
G, the B + W signal is sampled and held in the circuit 21, and R + G in the circuit 18 and B + in the circuit 19 in the even field.
W, the circuit 20 samples and holds the B + G signal, and the circuit 21 samples and holds the R + W signal. One of the signals from the circuits 18 and 19 is added together to obtain a 2 (R + G + B) signal 22, and the other one is obtained by subtracting the output of the circuit 18 from the output of the circuit 19, 2R in the odd field and 2 in the even field.
A signal 23 that becomes B is obtained. Similarly, 2 (R + G + B) signal 2 is obtained by addition from the output signals of the circuits 20 and 21.
4 is subtracted to obtain a signal 25 having 2B in the odd field and 2R in the even field. Next, the bands of the signal 22 and the signal 24 are both limited to f _N / 2 by the low-pass filters 26 and 27, and then the latter is subtracted from the former and adjusted to an appropriate size to obtain the field correction signal 28. Be done. By adding this signal to the signal 25, a signal 29 in which a false signal due to field interpolation is eliminated can be obtained. The signals 23 and 29 are distributed to only the R signal and only the B signal by the changeover switch 30 which alternates for each field, and the gain is adjusted to obtain the R output signal and the B output signal, respectively. The luminance signal Y is obtained by adjusting the gain of the signal 22.

FIGS. 8 (a) and 8 (b) show an example of a color filter array of one embodiment according to the present invention which enables field reading and field interpolation.
Both of them use 3 kinds of picture elements, all picture elements are sensitive to G color, and 2 picture elements sensitive to R color or B color respectively, and R color and B color are sensitive or both It is composed of insensitive pixels. Note that FIG.
In both (a) and (b), the signal processing and the image quality are the same even if the arrangement is switched between Ye and Cy. Further, FIG. 8 (b)
The image quality is the same as that of (a), and the signal processing can be easily considered from the case of (a). Therefore, only the case of (a) will be described below.

In the case of FIG. 8A, the luminance signal is easily obtained from the addition Cy + Ye + 2W = 3R + 4G + 3B between adjacent pixels. The color signal takes the difference between adjacent pixels,
Since R + B signals are obtained in the odd fields and RB signals in the even fields, field interpolation is used, and R = 1/2
{(R + B) + (R−B)}, B = 1/2 {(R + B)
-(R-B)}.

FIG. 9 shows a method for eliminating the false signal by the field interpolation in the case of FIG. 8 (a). First, from the luminance real-time signal Y (0F) to the luminance 1F delay signal Y (1
Field correction signal Y (0F) -Y (1
F) is formed. Next, this correction signal is converted to R for the 1F color signal.
It is added to the 1F color signal only in the + B field. As a result, the false signal due to the field interpolation is erased, and a normal color signal is always obtained. FIG. 10 shows an example of a circuit block diagram for realizing the above-mentioned embodiment in the color filter array of FIG. A signal from the solid-state image sensor 1 including the color filter shown in FIG. 8A is branched, and one of the signals is directly guided to the sample hold circuits 31, 32 to 33, 34 through the 1F delay circuit 2. In the odd field, the circuit 31 has Ye + Cy, the circuit 32 has W + W,
The circuit 33 samples and holds the Cy + W signal, and the circuit 34 samples and holds the Ye + W signal.
W, the circuit 32 samples and holds the signal Ye + W, the circuit 33 samples Ye + Cy, and the circuit 34 samples and holds the W + W signal. Circuit 31
One of the signals from 32 and 32 is 3R + 4
A G + 3B signal 35 is obtained, and the other one subtracts the output of the circuit 33 from the output of the circuit 32 to obtain R + B,
A signal 36 that is RB in the even field is obtained. Similarly, by adding from the output signals of the circuits 33 and 34,
The R + 4G + 3B signal 37 is subtracted to obtain a signal 38 that is R−B in the odd field and R + B in the even field. Next, the bands of the signal 35 and the signal 37 are both limited to f _N / 2 by the low-pass filters 39 and 40, the latter is subtracted from the former, adjusted to an appropriate size, and guided to the switch circuit 41. The switch circuit 41 is turned on in the field where the signal 38 becomes R + B, the field correction signal is added to the signal 38, and the signal 42 is obtained. One of the signals 36 and 42 is the sum of the two
The R signal 43 is obtained, and on the other hand, the signal 42 is subtracted from the signal 36 to obtain a 2B signal whose sign is inverted for each field. The subtraction signal is always a positive sign by the synchronous detection circuit 44. From the above, the gains of the signal 35, the signal 43, and the signal from the synchronous detection circuit 44 are respectively adjusted, and Y,
R and B signals are obtained.

[0018]

As described above, according to the present invention, the luminance signal maintains high resolution in both the horizontal and vertical directions, and
The two color signals have a high horizontal resolution, and the vertical resolution can be kept high by field interpolation. Further, it is possible to eliminate a false signal for a time-varying image by field interpolation, and it is possible to dynamically obtain a high quality image pickup signal.

[Brief description of drawings]

1A and 1B are configuration diagrams of a color filter array in a conventional solid-state color imaging device.

FIG. 2 is a configuration diagram of a color filter array in a conventional solid-state color imaging device.

FIG. 3 is a diagram comparing timings of signal reading in a solid-state color imaging device between a conventional case and a case of the present invention.

4A and 4B are configuration diagrams of a color filter array related to the present invention.

FIG. 5 is a diagram for explaining a method of eliminating a false signal.

FIG. 6 is a circuit block diagram corresponding to FIG.

FIG. 7 is a circuit block diagram corresponding to FIG.

8A and 8B are configuration diagrams of a color filter array according to the present invention.

FIG. 9 is a diagram for explaining a method of eliminating a false signal.

FIG. 10 is a circuit block diagram corresponding to FIG. 8A and showing an embodiment of the present invention.

[Explanation of symbols]

 1 Solid Color Imaging Device 2 1F Delay Circuit 31, 32, 33, 34 Sample Hold Circuit 39, 40 Low Pass Filter 41 Switch Circuit 44 Synchronous Detection Circuit

Claims (2)

[Claims] 1. A first device having different sensitive spectrum characteristics.
A second light receiving element, a third light receiving element, and a fourth light receiving element having the same sensitivity spectrum characteristic as the second light receiving element.
The second horizontal light receiving elements are alternately and repeatedly arranged in the first horizontal row.
In the horizontal row, the third light receiving elements and the fourth light receiving elements are alternately and repeatedly arranged, and the third horizontal row is arranged in the first horizontal row.
The same light receiving element set is arranged 180 degrees out of phase with respect to the horizontal row, and the fourth horizontal row is arranged with the same light receiving element set 180 degrees out of phase in the second horizontal row. A solid-state image sensor consisting of a series of columns arranged in sequence is added to two vertically adjoining light-receiving element signals, and the combination of addition is staggered one element in the vertical direction alternately in each field, and each addition signal is read out sequentially. A read-out means for forming a solid-state image sensor output signal, and a real-time brightness signal for forming a real-time brightness signal by taking the sum of signals adjacent in the horizontal direction from the solid-state image sensor output signal output from the read-out means A forming circuit, a real-time color signal forming circuit that forms a real-time color signal by taking a difference between horizontally adjacent signals from the solid-state image sensor output signal, and the solid-state image sensor output signal as one filter. From the field delayed output signal obtained by field delays, taking the difference of the signals adjacent in the horizontal direction, the solid-state color imaging apparatus comprising: the field delay chrominance signal forming circuit for forming a field delay chrominance signal, characterized by comprising providing a. 2. A first device having different sensitive spectral characteristics.
A second light receiving element, a third light receiving element, and a fourth light receiving element having the same sensitivity spectrum characteristic as the second light receiving element.
The second horizontal light receiving elements are alternately and repeatedly arranged in the first horizontal row.
In the horizontal row, the third light receiving elements and the fourth light receiving elements are alternately and repeatedly arranged, and the third horizontal row is arranged in the first horizontal row.
The same light receiving element set is arranged 180 degrees out of phase with respect to the horizontal row, and the fourth horizontal row is arranged with the same light receiving element set 180 degrees out of phase in the second horizontal row. A solid-state image sensor consisting of a series of columns arranged in sequence is added to two vertically adjoining light-receiving element signals, and the combination of addition is staggered one element in the vertical direction alternately in each field, and each addition signal is read out sequentially. A read-out means for forming a solid-state image sensor output signal, and a real-time brightness signal for forming a real-time brightness signal by taking the sum of signals adjacent in the horizontal direction from the solid-state image sensor output signal output from the read-out means A forming circuit, a real-time color signal forming circuit that forms a real-time color signal by taking a difference between horizontally adjacent signals from the solid-state image sensor output signal, and the solid-state image sensor output signal as one filter. Field delay luminance signal forming circuit for forming a field delay luminance signal by summing horizontally adjacent signals from the field delayed output signal delayed by the field delay, and a field delay obtained by delaying the output signal of the solid-state imaging device by one field. A field delay chrominance signal forming circuit that forms a field delay chrominance signal by taking the difference between horizontally adjacent signals from the output signal, and a color correction signal by subtracting the field delay luminosity signal from the real-time luminance signal Then, a correction field delay color signal forming circuit for forming a correction field delay color signal by adding the color correction signal to the field delay color signal is provided.