JP3733172B2 - Imaging device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an imaging apparatus suitable for use in a video camera or the like provided with a progressive scanning solid-state imaging device in which color filters are arranged in a so-called Bayer array.
[0002]
[Prior art]
In recent years, with the advance of semiconductor technology, solid-state imaging devices (all-pixel reading solid-state imaging devices, hereinafter referred to as all-pixel imaging devices) that can read signals by sequential scanning have been developed. These image sensors have less blur even with respect to a moving subject and can capture a high-resolution image compared to conventional interlaced scan (interlaced scan) image sensors.
[0003]
That is, in the interlace imaging device, since one frame image is composed of two fields having a time difference of one field period, when a moving subject is imaged, a jaggedness is generated due to the time difference between fields. If one field image is taken out so that there is no jaggedness, the vertical resolution will be halved. On the other hand, since the all-pixel imaging device can image all scanning lines for one frame in one field period, the above-described problem does not occur. Utilizing such characteristics, the all-pixel image sensor is expected to be applied to still image capturing cameras, computer input cameras, and the like.
[0004]
As a signal processing circuit of a single-plate all-pixel image sensor, for example, a configuration as shown in FIG. 9 can be considered.
In FIG. 9, incident light from a subject is imaged on a CCD 102 which is an all-pixel imaging element by an imaging optical system 101 and converted into an electrical signal by the CCD 102. A color filter array for color imaging is attached on the CCD 102. The color filter array is, for example, an R, B, G Bayer array as shown in FIG. The converted electric signal is transferred in a vertical register and a horizontal register in accordance with a timing signal from a timing generation circuit (not shown), and signals for two scanning lines are output in parallel from two output terminals of the CCD 102. That is, an odd-numbered scanning line signal is output from one of the two output terminals of the CCD 102, and an even-numbered scanning line signal is output simultaneously from the other. By adopting such an output method, a sequential scanning signal can be output at the same readout speed as that of an interlaced scanning CCD.
[0005]
The respective signals are processed by the noise reduction circuits 103 and 104 and the amplification circuits 105 and 106, converted into digital signals by the AD conversion circuits 107 and 108, and input to the camera process circuit 130 from the input terminals 128 and 129. . The input digital signal is time-axis converted into a line sequential signal by the 1H memories 109 and 110. The read clock of the 1H memories 109 and 110 is twice as fast as the write clock. The timing in this operation is shown in FIG. Thereafter, signals for three lines are further synchronized by the selector 133 and the 1H memories 111 and 112, and are respectively input to the color separation circuit 113 and the contour enhancement circuit 117.
[0006]
The GBR primary color signal demodulated by the color separation circuit 113 is limited to a necessary band by low-pass filters 114, 115, and 116, and interference components such as moire are removed. After the band-limited GBR signals are adjusted in level by the white balance circuit 118, the G signal is added by the adder 119 to the contour signal extracted by the contour emphasizing circuit 117. Further, the high-pass component of the G signal subjected to the contour correction is extracted by the high-pass filter 120 and added to the RB signal by the adders 121 and 122. Thereafter, after γ correction is performed by the γ correction circuits 123, 124, and 125, the matrix circuit 126 converts the luminance signal Y into the color difference signals RY and BY.
[0007]
Further, when a standard video output is required for monitor output or the like, the luminance signal Y and the color difference signals RY and BY are input to the scan conversion circuit 127. The scan conversion circuit 127 is configured as shown in FIG. 11, for example, and operates at the timing shown in FIG. In FIG. 11, the Y input is input to the selector 204 via the 1H memory 202, the RY input and the BY input are switched by the selector 201, and then switched to the Y input by the selector 204 via the 1H memory 203. Thus, the sequential scanning signal is converted to the interlace scanning signal.
[0008]
In FIG. 9, the exposure information detected by the exposure detection circuit 131 from the Y signal is taken into the MPU 132, and the aperture value of the imaging optical system 101 and the amplification circuits 105 and 106 are adjusted so that the exposure becomes an appropriate value. The amplification factor is controlled by the MPU 131.
[0009]
Next, the configuration of the color separation circuit 113 is shown in FIG. 13, and the operation is shown in FIG.
Of the signals corresponding to the three scanning lines input to the color separation circuit 113, the undelayed signal (the signal from the selector 133 in FIG. 1) is delayed by one horizontal scanning period (hereinafter 1H) in FIG. The signal (signal from the 1H memory 111) is shown in FIG. 5B, and the signal delayed by 2H (the signal from the 1H memory 112) is shown in FIG.
[0010]
Of these three system signals, the signal shown in (a) and the signal shown in (c) are added by an adder 140 to generate a signal interpolated in the vertical direction. This signal is shown in FIG. When the signal (d) and the 1H delay signal (b) are alternately selected for each dot by the selector 141, a G signal is obtained. The selection signal of the selector 141 is shown in FIG. 14E, and the output signal of the selector 141 is shown in FIG. Further, the output of the adder 140 and the signal via the flip-flop 142 are sent to the selector 144. Further, the signal (b) and the signal via the D flip-flop 143 are sent to the selector 145. The R signal is obtained by selecting the outputs of the selectors 144 and 145 with the selector 146, and the B signal is obtained by selecting the outputs of the selectors 144 and 145 with the selector 147.
[0011]
[Problems to be solved by the invention]
However, the above conventional example has the following problems.
The amplification factors of the signals in the amplification circuits 105 and 106 are controlled by the MPU 132. Since the amplification circuits 105 and 106 have individual operation variations, the MPU 132 sets the same amplification factor. Even in this case, the actual amplification factors of the amplifiers 105 and 106 are different from each other. For this reason, there may be a difference in the level of the video signal between the odd-numbered scanning lines and the even-numbered scanning lines.
[0012]
For this reason, when color separation is performed by the method shown in FIG. 13, if there is a difference in signal level between the odd-numbered scanning lines and the even-numbered scanning lines, vertical stripes of a two-dot period are mixed in the G signal. This significantly degrades the image.
In addition, if the band of the G signal is limited using a low-pass filter that suppresses the frequency of the vertical stripes for the purpose of removing the vertical stripes, even necessary information is suppressed, and as a result, the resolution of the G signal is deteriorated. There was a problem of doing.
[0013]
The present invention has been made to solve the above-described problems, and an object of the present invention is to obtain an imaging apparatus that can automatically calibrate an error in gain between two amplifier circuits.
[0014]
[Means for Solving the Problems]
In the present invention, a primary color RGB filter and a first color RGB filter arranged in an odd-numbered row in which R filters and G filters are alternately arranged, and in an even-numbered row in which G filters and B filters are arranged alternately. 1 and 2 having an output terminal, scanning signals of all pixels on the imaging surface by non-addition and sequentially reading out, and scanning line signals of the odd-numbered rows out of the first and second output terminals Imaging means configured to output from one terminal and simultaneously output the scanning line signal of the even-numbered row from the other terminal, and the odd-numbered and even-numbered obtained from the first and second output terminals Each of the primary colors RGB by vertical interpolation based on the signals amplified by the first and second amplifying means having variable amplification factors for amplifying the second scanning line signal, respectively, and the first and second amplifying means. Color separation means for separating signals Same as the first and second amplification means obtained from the output of the color separation means based on the detection means for detecting a frequency component of ½ of the pixel period and the signal detected by the detection means. Storage means for storing correction data indicating the difference between the actual amplification factors when the amplification factor is set, and the amplification factors of the first and second amplification units are controlled. During this control, Control means configured to correct the amplification factor given to each amplification unit with the stored correction data and to give the corrected amplification factor is provided.
[0015]
[Action]
According to the present invention, the data for correcting the difference between the actual amplification factors when the two amplification means based on the result of detecting the frequency component of ½ of the pixel period from the output of the color separation circuit is set as the same amplification factor. Is stored in advance, and the amplification factors given to the first and second amplification means are individually corrected using the correction data, thereby amplifying the two amplification means during normal shooting without using a special subject. It is possible to effectively automatically calibrate the rate error.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to FIG.
In FIG. 1, 1 is an image forming optical system for forming a subject image on an image pickup device, 2 is a CCD as an all-pixel solid-state image pickup device that reads out signals by sequential scanning, and 3 and 4 are noise reductions in the output signal of the CCD 2. Noise reduction circuits 5 and 6 are amplification circuits that amplify the signal of the CCD 2 to an appropriate level, and 7 and 8 are AD conversion circuits that convert the signal of the CCD 2 into a digital signal.
[0017]
Reference numerals 9 and 10 denote input terminals of the process circuit 42, reference numerals 11, 12, 13, and 14 denote line memories for delaying the CCD signal by one scanning line time, and reference numeral 15 denotes signals for six scanning lines synchronized with the line memories 11 to 14. , A vertical contour extraction circuit for extracting a vertical contour signal, 16 a color separation circuit for generating a GBR primary color signal from signals synchronized in the line memories 11 to 14, and 17 a base for slicing a small amplitude portion of the vertical contour signal Clip circuit, 18 is a low-pass filter that limits the band of the vertical contour signal, 19 is a horizontal contour extraction circuit that extracts a horizontal contour signal from the G signal, 20 is a base clip circuit that slices a small amplitude portion of the horizontal contour signal, and 21 is A gain adjusting circuit for adjusting the gain of the horizontal contour signal, 22 is an adder for adding the horizontal contour signal and the vertical contour signal, and 23 is for adjusting the gain of the contour signal. Is obtained adjustment circuit.
[0018]
Reference numeral 24 denotes a detection circuit that detects a frequency component of 1/2 times the pixel sampling frequency from the G signal, reference numerals 25, 26, and 27 denote low-pass filters that limit the signal band, reference numeral 28 denotes a white balance circuit that adjusts white balance, and reference numeral 29 denotes An adder for adding the contour signal to the G signal, 30 is a high-pass filter for extracting the high frequency band of the G signal, and 31 and 32 are adders for adding the high frequency band of the G signal extracted by the high-pass filter 30 to the R and B signals. , 33, 34, and 35 are γ correction circuits.
[0019]
Reference numeral 36 denotes a matrix circuit that generates a luminance signal Y and color difference signals RY and BY from primary color signals, 37 and 38 low-pass filters that limit the band of the color difference signal, 39 a scan conversion circuit, and 40 and 41 process circuits. 42 is an output terminal, 42 is a process circuit for processing a CCD signal, 43 is an MPU for controlling the amplification factors of the amplifier circuits 5 and 6, 44 is a state of exposure to the CCD 2 from Y, RY, and BY signals. An exposure detection circuit 45 for extracting the exposure information shown is a memory.
[0020]
Next, the operation according to the above configuration will be described.
Incident light from the subject is imaged on the imaging surface on the CCD 2 which is an all-pixel imaging element by the imaging optical system 1 and converted into an electrical signal by the CCD 2. A color filter array for color imaging is attached to the imaging surface on the CCD 2. This color filter array is, for example, an RBG Bayer array as shown in FIG.
[0021]
The converted electric signal is output in parallel from two output terminals of the CCD 2 for two scanning lines. That is, one side outputs an odd-numbered scanning line signal, and the other side outputs an even-numbered scanning line signal. The output signal from the CCD 2 is processed by the noise reduction circuits 3 and 4 and the amplification circuits 5 and 6 and then converted into digital signals by the AD conversion circuits 7 and 8. The signals for two scanning lines converted into digital signals are input to the process circuit 42 from the input terminals 9 and 10, and the signals for the six scanning lines are synchronized by the line memories 11, 12, 13, and 14.
[0022]
The signals for the six scanning lines are input to the vertical contour extraction circuit 15, and a vertical contour signal is extracted. The vertical contour extraction circuit 15 is constituted by a high-pass filter in the vertical direction. When the number of pixels of the CCD 2 is 640 × 480, for example, it has a transfer characteristic as shown in FIG. Further, the vertical contour signal for two scanning lines extracted by the vertical contour extraction circuit 15 is converted into a dot sequential signal having a frequency twice that of reading from the CCD 2 in order to reduce the circuit scale of the subsequent stage. That is, as shown in FIG. 4, the signals of the odd-numbered scanning lines and the even-numbered scanning lines are converted into point sequential signals that are time-division multiplexed. The vertical contour signal extracted by the vertical contour extraction circuit 15 slices a small amplitude portion for noise reduction by the pace clip circuit 17, and then an unnecessary band in the horizontal direction is removed by the low-pass filter 18.
[0023]
On the other hand, the signals synchronized by the line memories 11 to 14 are separated into GBR primary color signals in the color separation circuit 16 in accordance with a timing signal from a timing generation circuit (not shown). These signals are also in a form in which signals for two scanning lines are time-division multiplexed. Each signal is limited to an appropriate band by the low-pass filters 25, 26, and 27.
[0024]
For example, in the case of the color filter array as shown in FIG. 2, since G sampling points exist in each column, the horizontal band is ½ of the horizontal sampling frequency of the CCD 2. Accordingly, the pass band of the low-pass filter 25 is set to about ½ of the sampling frequency of the CCD 2. On the other hand, since there are only one sampling point for R and B in two columns, the band is ½ of the G signal, and the pass band of the low-pass filters 26 and 27 is about ¼ of the sampling frequency of the CCD 2. Set to An example of transfer characteristics of the low-pass filter 25 is shown in FIG. 5A, and examples of transfer characteristics of the low-pass filters 26 and 27 are shown in FIG. 5B.
[0025]
The G signal separated at the same time is input to the horizontal contour extraction circuit 19 and a horizontal contour signal is extracted. The extracted horizontal contour signal is processed by the base clip circuit 20 and the gain adjustment circuit 21 and then added to the above-described vertical contour signal by the adder 22. The added contour signal is adjusted to an appropriate level by the gain adjusting circuit 23 and then added to the G signal by the adder 29.
[0026]
On the other hand, the level of the frequency component that is ½ the sampling frequency of the G signal detected by the detection circuit 24 is input to the MPU 43 in order to detect an error in the amplification factor between the amplifier circuits 5 and 6. Further, each signal band-limited by the low-pass filters 25, 26, and 27 is changed in the gain of the RB signal by the white balance circuit 28 to match the level of the white portion. Next, the high-pass filter 30 extracts the high frequency band signal, and the adders 31 and 32 add it to the R and B signals. The characteristics of the high-pass filter 30 are complementary to those of the low-pass filters 26 and 27, and the purpose is to align the bands of the G signal and the RB signal.
[0027]
Thereafter, after γ correction is performed by the γ correction circuits 33, 34, and 35, the matrix circuit 36 converts the luminance signal Y and the color difference signals RY and BY into the low-pass filter 37. , 38 is band-limited to about ½ of the Y band. The three signals Y, RY, and BY are input to the exposure detection circuit 44, and exposure information is detected. The detected exposure information is taken into the MPU 43, and the aperture value of the imaging optical system 1 and the amplification factors of the amplification circuits 5 and 6 are controlled by the MPU 43 so that the exposure becomes an appropriate value according to the illuminance of the subject.
[0028]
Finally, the Y, RY, and BY signals are input to the scan conversion circuit 39. In the scan conversion circuit 39, the input Y, RY, and BY are switched by a switching pulse generated by a timing generation circuit (not shown) to generate two systems of video signals for each scanning line. This signal is, for example, in the form of time division in the order of Y, RY, Y, BY, by selecting RY and BY for each of two Y pixels. Y, RY, Y, and BY at this time are all signals on the same scanning line. The operation timing is shown in FIG. At this time, one clock of the output signal is twice the sampling rate of the Y signal. When one of the two systems is taken out, this is an interlaced scanning signal, and when both are used, it becomes a sequential scanning signal. The generated two systems of video signals are output from the output terminals 40 and 41.
[0029]
FIG. 7 shows the configuration of the color separation circuit 16 in detail. FIG. 8 shows the operation of the color separation circuit 16. The signals shown in FIGS. 8A to 8F correspond to the signals of the respective parts in FIGS. 7A to 7F.
The color separation circuit 16 separates a signal for one scanning line based on signals for three scanning lines. Since the G signal is obtained in a form in which the sampling phase is shifted by π for each scanning line, the CCD signal (the output terminal of the CCD 2) is obtained by alternately sampling the signal of the central scanning line and the signal obtained by adding the upper and lower scanning lines. G signal having a period equal to the sampling frequency of) is obtained. 7 and 8 (a) show the signal of the center scanning line, and (b) shows the signal obtained by adding the upper and lower scanning lines. By selecting these signals with the switch 73 at the timing shown in (c), the G signal shown in (d) is obtained. Although the description has been given of the 3H input signal in FIG. 7, the G signal is also separated by the same process for the 4H signal.
[0030]
Further, since the RB signal is obtained in a form sampled at a frequency half of the sampling frequency of the CCD signal every other scanning line, each of the signals obtained by adding the signal of the central scanning line and the upper and lower scanning lines is obtained from the CCD 2. It is obtained by sampling and holding at a period twice as high as the sampling of the output signal and alternately selecting at a period twice as high as the horizontal scanning frequency. That is, the signals shown in FIGS. 7E and 8F are obtained by sample-holding the signals of FIGS. 7, 8A and 7B by the D flip-flops 72 and 74 and the switches 75 and 76. FIG. In the case where the center scanning line is a signal represented by 4H, the RB signal is similarly separated.
After that, the signals for the two scanning lines obtained as described above are time-division multiplexed at a frequency twice that of sampling by the CCD signal switches 79, 80, 81 to be combined into one signal system.
[0031]
On the other hand, the amplification circuits 5 and 6 are controlled so that the amplification factor is controlled by the MPU 43 so that the level of the video signal is appropriate. However, as described above, there is an error in the amplification factors of the amplification circuits 5 and 6. In this case, there is a difference in signal level between the odd-numbered scan lines and the even-numbered scan lines. When color separation is performed by the above method in this state, as shown in FIG. 8D, vertical stripe noise having a 2-dot period is mixed in the G signal, and the image is significantly deteriorated. In order to solve this problem, the error data of the amplification factor between the two amplification circuits 5 and 6 for each amplification factor is stored in advance in the memory 45 of the MPU 43, and the MPU 43 uses the stored error data. The error may be corrected and the corrected data may be supplied to the amplifier circuits 5 and 6, respectively.
[0032]
The error data is created at the time of initial adjustment (for example, at the time of factory shipment) and stored in the memory 45. As a method for creating error data, for example, a method is used in which an object having a uniform screen is imaged and an error in amplification factor is detected based on the amount of frequency components with a 2-dot period detected by the detection circuit 24. Then, by correcting the gain value given to the amplifier circuits 5 and 6 with this error data, the error of the gain between the two amplifier circuits 5 and 6 is automatically detected during normal photographing without using a special circuit. Can be calibrated.
[0033]
【The invention's effect】
As described above, according to the present invention, the difference between the actual amplification factors of the two amplifying means is detected by detecting the vertical stripe noise in the Bayer array with the frequency component of 1/2 of the pixel period, and the detected signal. By storing correction data obtained based on the above in advance and using this correction data, automatic calibration can be effectively performed with high accuracy during normal shooting without using a special circuit.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an embodiment of the present invention.
FIG. 2 is a configuration diagram illustrating an example of a color filter array.
FIG. 3 is a characteristic diagram showing transfer characteristics of a vertical contour emphasizing circuit.
FIG. 4 is a timing chart showing the operation of time division multiplexing.
FIG. 5 is a characteristic diagram showing transfer characteristics of a low-pass filter.
6 is a timing chart showing a format of an output signal from the process circuit in FIG. 1. FIG.
7 is a configuration diagram of the color separation circuit 16 in FIG. 1. FIG.
8 is a timing chart showing the operation of the color separation circuit in FIG.
FIG. 9 is a block diagram illustrating a conventional imaging apparatus.
10 is a timing chart showing the operation of the 1H memory in FIG. 9. FIG.
11 is a configuration diagram of a scan conversion circuit in FIG. 9. FIG.
FIG. 12 is a timing chart showing the operation of the scan conversion circuit.
13 is a configuration diagram of a color separation circuit in FIG. 9. FIG.
FIG. 14 is a timing chart showing the operation of the color separation circuit.
[Explanation of symbols]
2 All-pixel readout CCD
5, 6 Amplifier circuits 11-14 1H memory 16 Color separation circuit 36 Matrix circuit 42 Process circuit 43 MPU
44 Exposure detection circuit 45 Memory

Claims (4)

The primary color RGB filters and the first and second colors of the Bayer array arranged in odd-numbered rows in which R filters and G filters are alternately arranged, and in even-numbered rows in which G filters and B filters are alternately arranged. It has an output terminal, scans all pixel signals on the imaging surface in a non-addition manner and sequentially reads them, and outputs the odd-numbered row scanning line signal from one of the first and second output terminals. And at the same time, imaging means configured to output the scanning line signal of the even-numbered row from the other terminal;
First and second amplification means with variable gain for amplifying the odd-numbered and even-numbered scanning line signals obtained from the first and second output terminals, respectively;
Based on the signals amplified by the first and second amplification means, color separation means for separating the signals of primary colors RGB by vertical interpolation;
Detection means for detecting a frequency component of ½ of the pixel period from the output of the color separation means;
Storage means for storing correction data, which is obtained based on the signal detected by the detection means, and indicates correction data indicating the difference between the actual amplification factors when the same amplification factor is set in the first and second amplification units;
The amplification factors of the first and second amplification means are controlled. At this time, the amplification factors given to the amplification devices are corrected with the stored correction data, and the corrected amplification factors are given. An image pickup apparatus comprising control means configured as described above.   Matrix means for converting the RGB signals color-separated by the color separation means into luminance signals and color-difference signals, and detection means for detecting the state of exposure to the imaging means from the signals processed by the matrix means, and the control 2. The means according to claim 1, wherein amplification means to be applied to the first and second amplification means is determined according to detection by the detection means, and the determined amplification ratio is corrected with the correction data. Imaging device.   2. The correction data according to claim 1, wherein the correction data is created based on a signal detected by the detection means when the imaging means picks up a uniform subject on the imaging surface. Imaging device.   The imaging apparatus according to claim 1, wherein the imaging unit includes an all-pixel readout CCD.

Images