JP2003037848A - Imaging apparatus and its signal processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the light utilization efficiency of an imaging apparatus having imaging elements of many pixels and prevent the deterioration of the image quality, with a raised frame rate for photographing moving pictures. SOLUTION: A plurality of filter units use each a color filter array 11 composed of a 2×2 array of color elements red(R), green(G) and blue(B). Among a plurality of pixel blocks each composed of a square array of 3×3 pixels greater than the filter unit (2×2 array), signal charges of all pixels belonging to the pixel blocks are added to obtain color information condensed every pixel block from the addition result of each pixel block, using the 2×2 array of the pixel block as a large filter unit.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus capable of color image pickup and a signal processing method thereof.

[0002]

2. Description of the Related Art In a single plate type solid-state image pickup device, a color filter array is superposed on a single image pickup device. CCD
In a (Charge Coupled Device) type image sensor, a plurality of photodiodes (PD) are two-dimensionally arranged and
A vertical CCD (VCCD) is provided for each of the vertical columns, and one horizontal CCD is provided at one end of these VCCDs.
(HCCD) are commonly connected. Each PD constitutes a pixel for converting incident light into a signal charge according to the amount of light. The color filter array has a plurality of two-dimensionally arranged filter units. For example, each of the plurality of filter units is composed of a 2 × 2 array of first, second, third and fourth color elements, and each color element is arranged on the image pickup device so as to correspond to each pixel of the image pickup device. Can be stacked. Primary color Bayer
r) In a color filter array called an array, the first, second,
Each of the third and fourth color elements has, for example, a selective transmission characteristic of red (R), green (G), green (G) and blue (B). G sandwiched between two Rs is expressed as “Gr”, and G sandwiched between two Bs is represented as “Gr”.
It is convenient to write "b". In addition, as one of complementary color filter arrays, the first, second, third, and fourth color elements selectively transmit magenta (Mg), green (G), cyan (Cy), and yellow (Ye), respectively. Color filter arrays having characteristics are also known.

In recent years, the number of pixels of the image pickup device has been increased to realize higher resolution image pickup. At present, a digital still camera (DSC) equipped with a CCD type image pickup device having 2 million pixels or 4 million pixels has been commercialized, and there is a report of a prototype of an image pickup device having multiple pixels exceeding 10 million pixels.

On the other hand, it is required to obtain not only still images but also moving images by DSC. In the case of still image capturing, there are not many time restrictions after the shutter is released, so even in the case of a multi-pixel image sensor, it is possible to slowly read out the signal charges of all pixels individually using VCCD and HCCD. Is. However, for example, in a monitoring mode in which a moving image of a subject is displayed on a liquid crystal monitor so as to indicate a photo opportunity, the maximum drive frequency of each of the VCCD and the HCCD is restricted by the frequency characteristics of the amplifier provided on the HCCD output side. As the number of pixels increases, the frame rate of moving images decreases.

Therefore, according to the technique disclosed in Japanese Unexamined Patent Publication No. 2000-209599, 3
Transfer of signal charges from the VCCD to the HCCD is prohibited for every two columns. As a result, the signal charges of the pixels are discarded for every two columns for every three columns, so that the frame rate can be increased. It is also known that the frame rate can be increased by discarding the signal charges of pixels in every two rows in the VCCD. Moreover, as long as a filter unit consisting of a 2 × 2 array of color elements is adopted, the required color information can be obtained from the signal charges left undiscarded.

[0006]

However, when the technique of selectively discarding the signal charge of the pixel is adopted as described above, the utilization efficiency of light is deteriorated and it is not suitable for image pickup particularly in low illuminance. There were difficulties. Further, when the signal charges of the pixels are discarded in only one of the row and the column, the spatial frequency characteristics are unbalanced in the vertical direction and the horizontal direction, resulting in deterioration of the image quality.

SUMMARY OF THE INVENTION An object of the present invention is to improve the utilization efficiency of light while increasing the frame rate in moving image pickup in an image pickup apparatus having a multipixel image pickup element, and prevent deterioration of image quality.

[0008]

In order to achieve the above object, according to the present invention, a plurality of pixel blocks each having a square array of odd × odd pixels larger than a filter unit (2 × 2 array) are used. The signal charges of all the pixels belonging to the pixel block are added.

More specifically, the present invention relates to an image pickup device having a plurality of pixels two-dimensionally arranged so as to convert incident light into signal charges corresponding to the amount of light in an image pickup device capable of color image pickup. A plurality of filter units arranged two-dimensionally, each of the plurality of filter units is a 2 × 2 array of first, second, third and fourth color elements, and each color element is an image sensor. A color filter array superimposed on the image sensor so as to correspond to each pixel, and a plurality of pixels of the image sensor are grouped into a plurality of pixel blocks each formed of a square array of odd number × odd number pixels larger than the filter unit, For each of the plurality of pixel blocks, a drive circuit for driving the image sensor so as to add the signal charges of all the pixels belonging to the pixel block, and the pixel block
The signal processing circuit is provided for obtaining condensed color information for each pixel block from the result of addition for each pixel block with the × 2 array as a large filter unit. Each of the plurality of pixel blocks has, for example, a 3 × 3 pixel array. Two of the first, second, third and fourth color elements may be the same color.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 shows a structural example of an image pickup apparatus according to the present invention. The image pickup apparatus is a DSC capable of color image pickup having a still image mode (also referred to as all-pixel mode or mode 1) and a moving image mode (also referred to as monitoring mode or mode 2), and includes a CCD-type image sensor 10, a color filter. The array 11, the lens 12, the drive circuit 13, the signal processing circuit 14, and the mode switching unit 15 are provided. The CCD type image pickup device 10 is an image pickup device having a plurality of pixels which are two-dimensionally arranged so as to convert incident light into signal charges corresponding to the amount of light. The color filter array 11 has three or more color elements arranged two-dimensionally, and each color element is overlaid on the image sensor 10 so as to correspond to each pixel of the image sensor 10. The lens 12 guides the incident light from the subject to the color filter array 11. The drive circuit 13 is a circuit for supplying a plurality of drive pulses for driving the image sensor 10. The signal processing circuit 14 is a circuit for producing a luminance signal Y and color difference signals Cr and Cb based on the CCD output signal Vout obtained from the image pickup device 10 and outputting these as a video signal. However, R as a video signal
A GB signal may be output in some cases. The mode switching unit 15 is
It is means for controlling both circuits 13 and 14 to operate in the designated mode by supplying a mode signal MD indicating the designation of mode 1 or mode 2 to the drive circuit 13 and the signal processing circuit 14.

FIG. 2 shows a schematic structure of the CCD type image pickup device 10 in FIG. In the image sensor 10 of FIG. 2, a plurality of PDs 20 are two-dimensionally arranged, and VCCDs 22 are provided for the vertical columns of these PDs 20, respectively.
One HCCD 23 is commonly connected to one end of D22. Each PD 20 converts incident light into a signal charge according to the amount of the light and stores the signal charge. Each PD20
The read gate 21 is interposed between the CCD 22 and the VCCD 22, and the signal charge accumulated in the PD 20 is read to the VCCD 22. Four-phase transfer gates φV1, φV2, φV3, and φV4 are provided for each PD 20, and φV3 also serves as the read gate 21. The HCCD 23 has two-phase transfer gates φH1 and φH2.
Is provided. The charge transfer destination of the HCCD 23 is connected to a floating diffusion (FD) 25 via an output gate (OG) 24. FD2
The electric charge stored in 5 is detected and amplified by the amplifier 26,
The voltage signal Vout is output.
The charges remaining in the FD 25 are reset gate (RG) 27.
By applying a reset pulse φR to the reset drain (RD) 28, the discharge is performed.
The four-phase transfer gates φV1 to φV4 of the VCCD 22,
And a drive pulse to each of the two-phase transfer gates φH1 and φH2 of the HCCD 23 and a reset pulse φ to the RG 27.
R is supplied from the drive circuit 13 in FIG.

FIG. 3 shows an example of the color filter array 11 in FIG. The color filter array 11 of FIG. 3 is the above-mentioned primary color Bayer array and has a plurality of filter units arranged two-dimensionally, and each filter unit is a color element R, Gr, G.
It consists of a 2 × 2 array of b and B.

When mode 1 is designated by the DSC of FIG. 1, the drive circuit 13 and the signal processing circuit 14 operate so as to obtain individual color information from the signal charge of each pixel of the CCD type image pickup device 10. Thus, high-resolution still image capturing is executed.

FIG. 4 shows a mode 1 charge transfer state in the VCCD 22 shown in FIG. 2 together with drive pulse waveforms. The upper part of FIG. 4 shows the flow of signal charges read from three PDs (pixels) 20 adjacent to the HCCD 23. The signal charge accumulated in the PD 20 is V by applying a high voltage (VH) read pulse to φV3.
It is read by the CCD 22. The signal charge read out to the VCCD 22 is applied to φV1 to φV4 by applying a drive pulse that changes between an intermediate voltage (VM) and a low voltage (VL) at the timings shown in FIG. Minutes transferred. By this operation, each VCCD
22 final stage signal charge, that is, PD of 1 horizontal line
The 20 signal charges are transferred into the HCCD 23. Then, after one horizontal line transfer by the HCCD 23, the signal charge of the PD 20 of the next horizontal line is transferred into the HCCD 23. By repeating this operation, a still image for one screen can be obtained.

FIG. 5 shows the driving pulse waveform of mode 1 in the HCCD 23 in FIG. 2 together with the waveform of the CCD output signal Vout. The signal charges transferred to the HCCD 23 transfer binary drive pulses of mutually opposite phases to the transfer gate φ.
By being applied to H1 and φH2, they sequentially pass through the OG 24 and are stored in the FD 25. The electric charge stored in the FD 25 is detected and amplified by the amplifier 26, and the voltage signal Vout
Is output as. The charges accumulated and detected in the FD 25 are discharged to the RD 28 by applying a reset pulse φR to the RG 27 every time the HCCD 23 transfers one stage. By repeating this operation, the signal charge of one horizontal line is detected and output as the voltage signal Vout. Since the processing in the signal processing circuit 14 is the same as the conventional one, detailed description will be omitted.

In the mode 2, the drive circuit 13 operates so that the signal charge in the CCD image pickup device 10 is added, and the signal processing circuit 14 performs the processing corresponding thereto.
High-sensitivity moving image pickup with a high frame rate is executed.

FIG. 6 uses the color filter array 11 of FIG.
The range of signal charge addition is shown. Image sensor 1
The plurality of pixels of 0 are each a square array of 3 × 3 pixels.
It is divided into a plurality of pixel blocks. Figure 6
The color element blocks corresponding to these are indicated by broken lines. Figure 6
Color element block in the upper left corner of the inside (first color element block)
Is 4 R, 2 Gr, 2 Gb, 1
Including B and. The color element block to the right of it (the second color element
Block) is 2 R, 4 Gr, and 1 Gb
And two B's. Next to the bottom of the first color element block
The color element block (third color element block) includes two R
, 1 Gr, 4 Gb, and 2 B are included.
Then, the color element block to the right of that (the fourth color element block
Is 1 R, 2 Gr, 2 Gb,
Includes four B's. These first to fourth color element blocks
Signal charge addition in the first to fourth pixel blocks corresponding to
Let A, Ba, Bc, and C be the results of 6 x 6 pixels
The new color information of the area ^*, Gr^*, Gb^*, B^*Represented by
If so,     A = 4R^*+ 2Gr^*+2 Gb^*+ 1B^*          (1)   Ba = 2R^*+ 4Gr^*+1 Gb^*+ 2B^*          (2)   Bc = 2R^*+ 1Gr^*+ 4Gb^*+ 2B^*          (3)     C = 1R^*+ 2Gr^*+2 Gb^*+ 4B^*          (4) Holds. In this way, the drive circuit 13 has a plurality of pixel blocks.
All of the locks belonging to the pixel block
Driving the image sensor 10 so as to add the signal charges of the pixels
It is.

FIG. 7 shows the result of signal charge addition based on FIG. Equations (1) to (4) are converted to R ^* , Gr ^* , Gb
Solving for ^* and B ^* , R ^* = (4A-2Ba-2Bc + 1C) / 9 (5) Gr ^* = (-2A + 4Ba + 1Bc-2C) / 9 (6) Gb ^* = (-2A + 1Ba + 4Bc-2C) / 9 (7) ) B ^* = (1A-2Ba-2Bc + 4C) / 9 (8) is obtained. The signal processing circuit 14 has 2 × pixel blocks.
Two arrays, that is, a region of 6 × 6 pixels is used as a large filter unit, and the condensed color information R ^* , Gr ^* , Gb for each pixel block is calculated from the result of signal charge addition for each pixel block.
^* , B ^* are obtained according to equations (5) to (8), and Y, Cr, Cb video signals or RGB video signals are produced in a conventional manner.

Next, an example of a method of adding signal charges for each pixel block will be briefly described with reference to FIGS. 8 and 9. Figure 8
Shows the mode 2 charge transfer in the VCCD 22, and signal charge addition in the vertical direction is performed in the HCCD 23 by continuously transferring the signal charges of 3 pixels from the VCCD 22 to the HCCD 23. Figure 9 shows HCC
The drive pulse waveform of mode 2 in D23 is shown, and the signal charge addition in the horizontal direction is performed at the input side of the amplifier 26 by applying the reset pulse φR once every three stages of transfer of the HCCD 23. To be seen.

As can be seen by comparing FIGS. 4 and 8, in mode 2, the number of horizontal line transfers by the HCCD 23 is one-third that in mode 1. 1 by VCCD22
Alternatively, the time required to transfer the signal charge of 3 pixels is HCCD
This is sufficiently shorter than the time required to transfer one horizontal line by 23. Therefore, in mode 2, it is possible to realize a high frame rate which is about three times as high as that in the case where the signal charge addition for each pixel block is not performed. Further, since the signal charges of all the pixels are effectively used even in the mode 2, the utilization efficiency of light is improved and it is also suitable for imaging with low illuminance. Moreover, there is no imbalance in the spatial frequency characteristics of the image in the vertical and horizontal directions.

The color filter array 11 is not limited to the primary color Bayer array shown in FIG. For example, it is also possible to employ the complementary color filter array composed of Mg, G, Cy and Ye described above. FIG. 10 and FIG. 11 are diagrams when such a complementary color filter array is used. In mode 2, the signal charge addition result α for each pixel block,
β, γ, δ and new color information Mg ^* , G ^* , Cy ^* , Ye ^*
With respect to α = 4Mg ^* + 2G ^* + 2Cy ^* + 1Ye ^* (9) β = 2Mg ^* + 4G ^* + 1Cy ^* + 2Ye ^* (10) γ = 2Mg ^* + 1G ^* + 4Cy ^* + 2Ye ^* (11) δ = 1Mg ^* + 2G ^* + 2Cy ^* + 4y ^* (12) is established, and from these equations (9) to (12), Mg ^* = (4α-2β-2γ + 1δ) / 9 (13) G ^* = (-2α + 4β + 1γ-2δ) / 9 (14) Cy ^* = (-2α + 1β + 4γ-2δ) / 9 (15) Ye ^* = (1α-2β-2γ + 4δ) / 9 (16) is obtained. The signal processing circuit 14 calculates the condensed color information Mg ^* , G ^* , Cy ^* , Ye ^* for each pixel block by the equation (1).
3) to (16). Note that equation (13)-
The 16 coefficients in (16) can be appropriately changed according to human visual characteristics. The same applies to the coefficients in equations (5)-(8). Furthermore, for example, a 5 × 5 pixel array may be adopted as the size of the pixel block.

The present invention is applicable not only to DSC but also to video cameras. In this case, high resolution and low frame rate moving image capturing is achieved in mode 1, and low resolution and high frame rate moving image capturing is achieved in mode 2.

The present invention is not limited to the CCD type image pickup device 10, but can be applied to an image pickup device having another type of image pickup device such as a MOS image sensor as long as it has a signal charge adding means or an averaging means. Is.

[0025]

As described above, according to the present invention, an odd number larger than each filter unit (2 × 2 array) ×
Since the signal charges of all the pixels belonging to the pixel block are added to each of the plurality of pixel blocks formed of a square array of odd-numbered pixels, the light utilization efficiency is improved while increasing the frame rate in moving image capturing, In addition, it is possible to prevent deterioration of image quality.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration example of an image pickup apparatus according to the present invention.

FIG. 2 is a schematic configuration diagram of a CCD type image pickup device in FIG.

FIG. 3 is an explanatory diagram showing an example of a color filter array in FIG.

FIG. 4 is a timing chart showing the state of charge transfer in mode 1 in the vertical CCD shown in FIG. 2 together with drive pulse waveforms.

5 is a timing chart showing a driving pulse waveform of mode 1 in the horizontal CCD shown in FIG. 2 together with a waveform of a CCD output signal.

6 is an explanatory diagram showing a range of signal charge addition when the color filter array of FIG. 3 is used.

FIG. 7 is an explanatory diagram showing a result of signal charge addition based on FIG. 6;

8 is a timing chart showing the state of charge transfer in mode 2 in the vertical CCD shown in FIG. 2 together with drive pulse waveforms.

9 is a timing chart showing a driving pulse waveform of mode 2 in the horizontal CCD in FIG. 2 together with a waveform of a CCD output signal.

10 is an explanatory diagram showing a range of signal charge addition when a complementary color filter array is used instead of the color filter array of FIG.

11 is an explanatory diagram showing a result of signal charge addition based on FIG.

[Explanation of symbols]

10 CCD image sensor 11-color filter array 12 lenses 13 Drive circuit 14 Signal processing circuit 15 Mode switching unit 20 Photodiode (PD) 21 Read gate 22 Vertical CCD (VCCD) 23 Horizontal CCD (HCCD) 24 Output Gate (OG) 25 Floating Diffusion (FD) 26 amplifiers 27 Reset gate (RG) 28 Reset drain (RD)

Continued front page    (72) Inventor Akito Kitera             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. (72) Inventor Toshiya Fujii             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. F-term (reference) 2H083 AA02 AA26                 5C024 AX01 BX01 CX37 CX41 EX52                       GX03 GY01 GZ29 HX28                 5C065 AA03 BB22 CC01 CC08 DD02                       DD17 EE03 EE06 EE08 GG21

Claims (6)

[Claims] 1. An image pickup device capable of color image pickup, comprising: an image pickup device having a plurality of pixels arranged two-dimensionally so as to convert incident light into a signal charge corresponding to the amount of light; and an image pickup device arranged two-dimensionally. A plurality of filter units, each of the plurality of filter units consisting of a 2 × 2 array of first, second, third and fourth color elements, each color element corresponding to a pixel of the image sensor; As described above, a color filter array is superimposed on the image sensor, and a plurality of pixels of the image sensor are grouped into a plurality of pixel blocks each of which is a square array of odd number × odd pixel larger than each filter unit, For each of the plurality of pixel blocks, a drive circuit for driving the image sensor so as to add the signal charges of all the pixels belonging to the pixel block, and a 2 × 2 array of the pixel blocks is large. As filter unit, an imaging apparatus characterized by comprising a signal processing circuit for obtaining a condensed color information of each pixel block from the result of the addition of each of the pixel blocks. 2. The image pickup device according to claim 1, wherein the drive circuit and the signal processing are performed so as to obtain individual color information from signal charges of individual pixels of the image pickup element, without depending on addition for each pixel block. An image pickup apparatus further comprising means for controlling a circuit. 3. The image pickup device according to claim 1, wherein each of the plurality of pixel blocks comprises a 3 × 3 pixel array. 4. The image pickup device according to claim 1, wherein the first, second, third, and fourth color elements have selective transmission characteristics of red, green, green, and blue, respectively. Image pickup device. 5. The image pickup device according to claim 1, wherein the first, second, third, and fourth color elements have selective transmission characteristics of magenta, green, cyan, and yellow, respectively. Image pickup device. 6. An image pickup device having a plurality of pixels arranged two-dimensionally so as to convert incident light into a signal charge corresponding to the amount of light, and a two-dimensional arrangement and a 2 × 2 arrangement of color elements. A signal processing method for an image pickup apparatus, comprising: a plurality of filter units; and a color filter array that is superimposed on the image pickup element so that each color element corresponds to each pixel of the image pickup element, A plurality of pixel blocks of the image sensor are grouped into a plurality of pixel blocks each having a square array of odd number × odd number pixels larger than each filter unit, and signal charges of all pixels belonging to the pixel block for each of the plurality of pixel blocks. And a condensed color for each pixel block from the result of addition for each pixel block, with the 2 × 2 array of pixel blocks as a large filter unit. Signal processing method characterized by comprising the steps of: obtaining a distribution.