JP2002084548A - Color image pickup element and image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To maintain high SN and a high-speed frame rate by add reading, while obtaining a superior image with a resolution higher than that in the conventional processings. SOLUTION: In an image pickup device using a veneer type color image pickup element, in which a three-color filter of RGB is arranged, a filter arrangement of the image pickup element is set as a plural staged columns in which the unit arrangement is 4 rows × 6 columns which consist of a first row: RRGGBB, a second row: RRGGBB, a third row: GBBRRG and a fourth row: GBBRRG. After a vertical neighbor addition is carried out for a 4-pixel addition, at the time of horizontal neighbor addition, an add phase is reversed in each line, thereby suppressing reduction in resolution, accompanying the pixel addition.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a single-chip type color image sensor for arranging three colors, such as RGB, and performing color imaging.
The present invention relates to an imaging device using the imaging device.

[0002]

2. Description of the Related Art In recent years, digital still cameras (electronic cameras) for capturing a subject image with an image sensor such as a CCD and obtaining a video signal have been actively developed. In such an image pickup apparatus, addition of pixel information adjacent to the image pickup element, for example, addition of a signal of a total of 4 pixels of 2 pixels in the vertical direction and 2 pixels in the horizontal direction regardless of whether the image pickup apparatus is a moving image pickup apparatus or a still image pickup apparatus It is said that the resolution can be reduced, but the sensitivity can be improved.

As a specific method of this kind of addition, an external digital addition method in which a pixel signal is read out from an image pickup element by a usual method (one pixel at a time) and A / D converted and added in a digital system, for example, Or, for example, CCD in the image sensor
There is known an in-device analog addition method in which the transfer drive of an image pickup device is devised and charges are added in a transfer path. The intra-element addition method also has the effect of shortening the reading time of one frame image (improving the frame rate), which cannot be obtained by the external digital method.

[0004]

However, in the conventional technique of adding the signals of four pixels as described above, the resolution is deteriorated due to its principle. For example, FIG.
2 × 2 for the RGB Bayer array element as shown in FIG.
If the four-pixel addition is performed, all the RGB colors are mixed by the addition, so that color imaging cannot be performed.

Therefore, a multi-Bayer array as shown in FIG. 6B is used. However, since the color sampling frequency is に 対 し て of that of the original Bayer array of FIG. Both in the vertical direction are deteriorated by half. In other words, the number of pixels per unit area is reduced to 1/4,
Therefore, the size of one pixel is equivalent to that of a large-pixel image sensor having about four times the size. That is, there is a problem that the color sampling frequency is lowered by the double Bayer filter for addition, and the resolution is deteriorated.

The present invention has been made in consideration of the above circumstances, and an object of the present invention is to suppress a decrease in resolution due to pixel addition, thereby obtaining a good image having a higher resolution than before. It is an object of the present invention to provide a color image pickup device and an image pickup device which are used.

[0007]

(Structure) In order to solve the above problem, the present invention employs the following structure.

That is, the present invention has an image pickup surface in which a plurality of photoelectric conversion pixels are arranged in a matrix, and each of the pixels on the image pickup surface has different first to third spectral characteristics α, β, and γ. A single-chip color image pickup device that performs point-sequential color separation by a color arrangement in which colors are arranged, wherein the color arrangement is as follows: first row: ααββγγ second row: ααββγγ third Row: βγγααβ Fourth row: A periodic array (double staggered array) having a unit array of 4 rows × 6 columns consisting of βγγααβ.

Here, the three colors α, β, and γ are three colors of additive color mixture.
It is desirable that the colors be primary colors RGB.

According to another aspect of the present invention, there is provided an image pickup apparatus using the color image pickup device having the above-described configuration, wherein a driving means for driving the image pickup device to read out an image pickup signal is provided in each of a row and a column in a pixel array of the image pickup device. It is characterized by having a same-color adding means for adding every four adjacent same-color pixel signal information, and an image signal generating means capable of generating an image signal of a predetermined format based on an output of the adding means.

Here, preferred embodiments of the present invention include the following. (1) The addition of the pixel signal information in the column direction by the addition means
When transferring electric charges from the vertical transfer path to the horizontal transfer path of the image sensor by the driving means, two
It is performed by vertical two-addition driving for transferring charges for pixels (two transfer units).

(2) The addition of the pixel signal information in the row direction by the adding means is such that, when the driving means transfers electric charges from the horizontal transfer path of the image sensor to the output amplifier, two pixels are required for one reset operation in the output amplifier. (2 transfer units) horizontal 2 addition drive for transferring charges for
It is performed by an additive phase inversion type horizontal two-addition drive that shifts the pixel addition area in the horizontal direction by one pixel (inverts the phase relationship between the reset operation and the horizontal transfer) in a new row unit by the addition drive.

(3) The addition of the pixel signal information in the row direction by the addition means is performed by A / D conversion of the image sensor output image signal read out by the vertical two addition drive (and the non-addition horizontal transfer drive).
After the D conversion, the digital addition is performed by two pixels in the horizontal direction, and the pixel addition area in the horizontal direction is shifted by one pixel for each new row by the vertical two addition drive (the addition phase is changed in units of rows). While digitally adding).

(4) The driving means reads out the pixel signal information of the image pickup device by non-addition driving, and the adding means performs all processing related to the addition of four adjacent pixel signal information of the same color. A / D conversion is performed on an image sensor output image signal read by driving and then digital addition is performed.

(Function) In the present invention, the color arrangement of the pixels is set to the above-mentioned double staggered arrangement, and the adjacent four pixels of the same color are added. However, the double staggered arrangement is realized by the color filter of the three primary colors RGB of the additive color mixture. Then, with respect to color sampling, R and B exist at a pitch of 2p in the vertical direction and 3p in the horizontal direction. Here, in the non-addition Bayer array shown in FIG. 6A, R and B exist at a pitch of 2p in both the vertical and horizontal directions with respect to color sampling. In the multiple Bayer array shown in FIG. R and B are vertical for sampling,
It exists at a pitch of 4p in the horizontal direction.

Therefore, according to the present invention, the double staggered RG
By using the B color filter and adding four pixels of the same color adjacent to each other, the vertical color sampling frequency is the same and the horizontal color sampling frequency is 2/3 as compared with the non-addition Bayer array, so the vertical resolution is Does not deteriorate, and the horizontal resolution can be secured to 2/3, which is a significant improvement over the conventional multi-bayer arrangement.

[0017]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The details of the present invention will be described below with reference to the illustrated embodiments.

FIG. 1 is a block diagram showing a schematic configuration of an electronic camera according to an embodiment of the present invention.

In the figure, reference numeral 101 denotes a lens system including various lenses; 102, a lens driving mechanism for driving the lens system 101; 103, an exposure control mechanism for controlling the aperture of the lens system 101; A filter for cutting, 105 is a CCD color image sensor having a built-in color filter, and 106 is a CCD for driving the image sensor 105.
CD driver 107, a pre-processing circuit including an A / D converter, etc., 108, a digital processing circuit for performing color signal generation processing, matrix conversion processing, and other various digital processing, 109, a card interface, 11
0 denotes a memory card such as a CF, and 111 denotes an LCD image display system.

In the figure, reference numeral 112 denotes a system controller (CPU) for comprehensively controlling each unit, 113 denotes an operation switch system composed of various SWs, and 114 denotes an operation display for displaying an operation state, a mode state, and the like. System, 115 is a lens driver for controlling the lens driving mechanism 102, 116 is a strobe as a light emitting means, 117 is an exposure control driver for controlling the exposure control mechanism and the strobe 116, and 118 is various kinds of setting information and the like. Non-volatile memory (EEPROM) is shown.

In the electronic camera according to the present embodiment, the system controller 112 performs overall control, and the CCD driver 106 controls the CCD image pickup device 10.
5 is controlled to perform exposure (charge accumulation) and signal readout, which are taken into a digital process circuit 108 via a preprocess circuit 107, subjected to various signal processings, and then subjected to a card interface 109 via a card interface 109. The data is recorded on the memory card 110.

FIG. 2 is a plan view showing the device structure of the CCD image pickup device 105. Photodiodes 201 are arranged in a matrix as light receiving elements.
A plurality of vertical CCDs 202 are arranged in the column direction between them,
One horizontal CCD in the row direction at the end of the vertical CCD 202
203 are arranged. And the photodiode 2
The signal charge stored in the first CCD 01 is read out to the vertical CCD 202 by the charge transfer pulse TG, and is transferred inside the vertical CCD 202 downward in the drawing. The signal charge transferred from the vertical CCD 202 is transferred to the horizontal CCD 203,
D203 is transferred leftward on the paper, and finally output via an FDA (floating diffusion amplifier) 204 which is a readout amplifier.

The basic configuration up to this point is the same as that of the conventional apparatus, but in the present embodiment, in addition to this, an adding means is provided in the digital process circuit 108 as described later. The addition reading in the image sensor 105 is performed by the CCD driver 1 under the control of the system controller 112.
The digital summation and all other signal processing are performed by the digital process circuit 108 under the control of the system controller 112.

It is assumed that the image sensor 105 used in the present embodiment is of a square pixel type in which the vertical and horizontal pixel pitches are equal (the value is p), of a progressive scanning type. The image sensor 105 has a color filter pattern in which 2 × 2 4 pixels are the minimum unit as shown in FIG. When the minimum unit (pixel) of the substantial image information is composed of a plurality of pixels (photoelectric conversion units) of the image sensor 102, this is referred to as a multi-pixel. Similarly, terms such as multiple lines (multiple lines) are introduced.

In the filter pattern of FIG. 3, the filters of the three colors RGB are arranged on the first line: RRGBBBB second line: RRGGBB third line: GBBBRG fourth Rows: A periodic array having a unit array of 4 rows × 6 columns consisting of GBBBRG.

That is, the filter pattern forms a multi-pixel array in units of 2 × 2 4 pixels.
In a so-called staggered arrangement in which the pixel position is inverted in phase for each row in the row direction, and when attention is paid to only one color (for example, R), the staggered configuration in which the pixel position is inverted for each row in the row direction is also used. It is arranged. Note that such a multi-pixel arrangement in a staggered arrangement is referred to as a double-staggered arrangement.

For each pixel of the image sensor 105,
Two-line addition driving (output of V clock in two transfer units during the H blanking period) is used. Therefore, the frame rate is improved to twice that of the non-addition driving. Note that the horizontal addition is executed in an output amplifier section inside the element or outside as described later. Then, a unit pixel having a pixel density of 1/4 (the position is indicated by a black circle) with respect to the original pixel is generated by a four-pixel addition process as shown in FIG. The generated unit pixel is
This is a so-called staggered arrangement.

The filter system 104 has a horizontal separation width of 1.5.
An optical low-pass filter having p and a vertical separation width of p is used. In this description, the horizontal and vertical components of the frequency trap are 1 / 3p and 1 / 2p (cycls / l
angle unit) so that the horizontal
This means that the OLPFs whose vertical components are 1.5p and p are used, respectively, and the specific direction of the light beam separation is not limited to the horizontal and vertical directions. As a preferred example, "a diamond having horizontal and vertical directions diagonally"
A four-point equally separated type having a rhombus with a diagonal length of 3p horizontally and 2p vertically can be used.

Next, the image pickup operation in this embodiment, particularly the addition processing of the pixel signals, will be described.

First, two-line addition drive is performed for vertical addition of pixel signals. The driving pattern is as shown in FIG.
Since the V transfer clock φV for two pixels (two rows) is output in each horizontal blanking period, the charges of the upper and lower two pixels are added in the horizontal transfer path of the image sensor 105. Therefore, in the state after the vertical addition, one line includes addition information for two lines of original pixels.

The added charges are transferred in the horizontal transfer path, and are sequentially sent to the diffusion section of the FDA 204, which is an output amplifier, to be converted into charge voltages. Here, as shown in FIG. 5, the reset gate pulse RS for resetting the electric charge of the diffusion section prior to each transfer is thinned out unlike the non-addition driving. Specifically, it is added only once in two horizontal transfer units (two clocks of the transfer clock φH). That is, charge transfer for two pixels (two transfer units) is performed for one reset operation. Therefore, the output signal is a signal obtained by adding two horizontally adjacent pixels (four pixels in terms of original pixels).

As shown in FIG. 5, the RS thinning is performed by inverting the phase of each line (expressed after vertical addition = corresponding to multiple lines). As a result, a horizontal pixel addition area corresponds to one pixel per line. Will be off by a minute. That is, the phase of the addition is inverted for each line, and the addition of the pattern surrounded by the thick line in FIG. 3 is performed.

In the horizontal addition, the image pickup device 10
After reading out the signal from 5 through 2 vertical line addition and horizontal non-addition, two adjacent pixels may be added by digital operation. Also in this case, the addition phase is inverted for each line (multiple lines) in the same manner as described above.

By interpolation processing including synchronization from dot-sequential color information, which is pixel information of 1/4 of the number of pixels (pixel density) obtained as described above, for example, the same number of pixels as the image sensor 105 ( (The same pixel density). The specific interpolation method is arbitrary. Further, known signal processing such as gamma processing, knee, setup, and matrix processing of, for example, 0.45 is performed to perform a recording process including compression as necessary as a color image signal. To record.

As described above, according to the present embodiment, the three-color staggered RGB three-color filter shown in FIG.
After performing vertical adjacent addition for 4-pixel addition, by inverting the addition phase for each line at the time of horizontal adjacent addition, it is possible to suppress a decrease in resolution due to pixel addition.
A good image with higher resolution than before can be obtained.

Here, supplementing the horizontal resolution in the present embodiment, in the case of a single-chip color image sensor, color moire occurs near the color sampling frequency fcs.
It is essential to use an optical low-pass filter (OLPF) having a zero point (trap) at this fcs. Note that the trap frequency is the reciprocal of the beam separation width. Therefore, the limit resolution is determined by fsc (about 90 to 95% of fsc).

The separation width required in the case of FIG. 6A used for non-addition is p for both the horizontal and vertical directions because R and B are arranged at intervals of 2p. In FIG. 6B, which is a conventional four-pixel addition, since R and B are arranged at intervals of 4p, both horizontal and vertical are 2p. For this reason, the resolution has been reduced to almost half both in the horizontal and vertical directions. On the other hand, in the present embodiment, the same is true for any of the RGB colors, and is arranged at intervals of 2p in the vertical direction and at intervals of 3p in a staggered arrangement in the horizontal direction.

The present invention is not limited to the above embodiment. In the embodiment, the RS is thinned out during the horizontal addition inside the image pickup device. However, the H transfer is doubled by setting the RS to the same frequency as the non-addition and inverting the phase for each line (multiple lines). By doing so,
The one-screen signal reading time is further reduced to １ ／.
That is, the frame rate is four times that of the normal (non-addition) driving.

After all the pixels are individually read out without addition, the same processing as described above may be performed in the all digital addition processing. In this case, the frame rate is not improved, but the sensitivity, SN, and D range can be improved while suppressing a decrease in resolution. In particular, when this all-digital addition processing is performed, since the addition processing does not depend on the horizontal and vertical directions, a filter obtained by rotating the double-staggered color array of FIG. Array ("Vertical type"
Can be used in exactly the same way. For this reason, the effect that the priority design of the resolution with respect to the horizontal and vertical can be freely performed can be obtained. In fact, by using the vertical-eye double staggered color arrangement, the horizontal resolution, which is considered to be the most important in human vision, has the same performance as that when pixel addition is not performed (FIG. 6A). Is obtained,
Extremely effective.

As is clear from the above, "row / column"
Directions such as “horizontal / vertical” are irrelevant to the orientation of the subject (imaging device), and may be associated in any way. Further, the color filters are not necessarily limited to RGB, but may be three colors of cyan, magenta, and yellow, which are the three primary colors of subtractive color mixing. Further, instead of using a color filter, a photodiode itself constituting a pixel may have spectral characteristics. Further, the present invention is not limited to a still camera, but can be applied to a video movie. In addition, without departing from the gist of the present invention,
Various modifications can be made.

[0041]

As described above in detail, according to the present invention, three colors of α, β, and γ, which are the first to third different spectral characteristics for each pixel on the imaging surface, are arranged in a double staggered arrangement. Adjacent same color 4
By performing the pixel addition, the vertical resolution is not degraded and the horizontal resolution is also 2 in comparison with the non-addition Bayer array.
/ 3 can be secured, and a decrease in resolution can be suppressed while maintaining the effects of improving the high-speed frame rate, sensitivity, SN, and D range by addition reading.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of an electronic camera according to an embodiment of the present invention.

FIG. 2 is a plan view showing an element structure of a CCD image sensor.

FIG. 3 is a view for explaining generation of a unit pixel by addition of a filter pattern according to the embodiment;

FIG. 4 is a timing chart showing pulse phases of vertical addition driving.

FIG. 5 is a timing chart showing pulse phases of horizontal addition driving.

FIG. 6 is a diagram for explaining unit pixel generation by a conventional filter pattern and addition.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 101 ... Lens system 102 ... Lens drive mechanism 103 ... Exposure control mechanism 104 ... Filter system 105 ... CCD color image sensor 106 ... CCD driver 107 ... Pre-process part 108 ... Digital process part 109 ... Card interface 110 ... Memory card 111 ... LCD image Display system 112 ... System controller (CPU) 113 ... Operation switch system 114 ... Operation display system 115 ... Lens driver 116 ... Strobe 117 ... Exposure control driver 118 ... Non-volatile memory (EEPROM)

Claims (7)

[Claims] An imaging surface having a plurality of photoelectric conversion pixels arranged in a matrix, and three colors of α, β, and γ, which are first to third different spectral characteristics, are arranged for each pixel on the imaging surface. A single-chip color imaging device that performs point-sequential color separation according to the color arrangement, wherein the color arrangement is: first line: ααββγγ second line: ααββγγ third line: β γγααβ Fourth row: A color image sensor having a periodic array having a unit array of 4 rows × 6 columns composed of βγγααβ. 2. The α, β, and γ are three primary colors RG of additive color mixture.
2. The color image pickup device according to claim 1, wherein each of the colors is B. 3. A color image sensor according to claim 1, wherein said image sensor is driven to read out an image signal. An image pickup apparatus comprising: a same-color adding means for adding two pieces of same-color pixel signal information; and an image signal generating means capable of generating an image signal of a predetermined format based on an output of the adding means. 4. The addition of the pixel signal information in the column direction by the adding means is performed by transferring two pixels (every horizontal blanking period) each time a charge is transferred from the vertical transfer path to the horizontal transfer path of the image sensor by the drive means. The imaging apparatus according to claim 3, wherein the imaging is performed by vertical two-addition driving that performs charge transfer for two transfer units. 5. The method according to claim 1, wherein the addition of the pixel signal information in the row direction by the adding means is performed in response to one reset operation in the output amplifier when the driving means transfers a charge from a horizontal transfer path of the image sensor to an output amplifier. Horizontal two-addition drive for transferring charges for two pixels (two transfer units)
5. The image pickup apparatus according to claim 4, further comprising an addition phase inversion type horizontal two addition drive that shifts a pixel addition area in the horizontal direction by one pixel in a new row unit by the vertical two addition drive. . 6. The addition of the pixel signal information in the row direction by the addition means is performed by digitally adding two pixels in the horizontal direction after A / D conversion of an image sensor output image signal read by the vertical two addition drive. 5. The image pickup apparatus according to claim 4, wherein the horizontal pixel addition area is shifted by one pixel for each new row by the vertical two addition drive. 7. The driving unit reads out the pixel signal information of the image sensor by non-addition driving, and the addition unit performs all processes related to the addition of the four adjacent pixel signal information of the same color. 4. The method according to claim 3, wherein the A / D conversion is performed on the image sensor output image signal read out by the non-addition drive, and then the digital image addition is performed.
An imaging device according to claim 1.