JP2005198119A - Image pick-up device and image photographing element - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image photographing device compatible in both a still image with a high pixel and a moving video image in photography, while an increase in the driving frequency of the moving image is restricted and deterioration in moving image quality caused by turnaround noise is restricted. <P>SOLUTION: A color filter of two-line/four-pixel repetition is used. As for the row of the color filter, Mg, Cy, G and Cy are arranged in an N line, and G, Ye, Mg and Ye are arranged in an N+1 line. In a vertical direction, three pixels in the same color signal are mixed for every one line to reduce read-out speed at the moving image, and in a horizontal direction, adjoining two pixels are mixed. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image pickup device that can read out pixels by mixing them and an image pickup apparatus using the image pickup device.

  As background arts in the above technical field, for example, those described below are known. The technique of Patent Document 1 uses a pickup device having a vertical stripe color filter arrangement to mix pixel signals in the vertical direction, thereby lowering the driving frequency during pixel mixing. The technique of Patent Document 2 reads out odd-numbered and even-numbered data separately in odd-numbered columns and even-numbered columns, and mixes two pixels in the horizontal direction when transferring charges from the light receiving element to the vertical CCD. Then, charges are transferred in the vertical direction by the vertical CCD. Furthermore, when transferring charges from the vertical CCD to the horizontal CCD, mixing for two pixels is performed in the vertical direction, thereby realizing four-pixel mixing and lowering the drive frequency during pixel mixing.

  In Patent Document 2, Mg, Cy, G, and Ye are arranged on the Nth line and G, Ye, Mg, and Cy are arranged on the N + 1 line as color filters that repeat in 2 lines and 4 pixels. By adopting such a filter arrangement, it is possible to output signals from the image sensor in the order of (Mg + Ye), (G + Ye), (G + Cy), and (Mg + Cy) both when mixing pixels and when performing independent reading without mixing pixels. Yes. (Mg + Ye) and (G + Cy) and (G + Ye) and (Mg + Cy) are signals on the same line, and (Mg + Ye) (G + Cy) and (G + Ye) (Mg + Cy) are signals on different lines.

JP 2000-295531 A

JP 2000-184385 A

  When a still image and a moving image are picked up using an imaging device having a large number of pixels (for example, 4 to 6 million pixels), pixels may be mixed in the moving image. In addition, even in an imaging device that does not use both still images and moving images, for example, when capturing moving images using an image sensor with a large number of pixels, or when capturing still images with increased sensitivity in dark scenes, etc. There are things to do. Pixel mixing is performed regardless of whether it is a still image or a moving image.

  Since the technology using the vertical stripe filter as in Patent Document 1 cannot perform horizontal mixing, the driving speed during pixel mixing increases. Further, in the technique in which the filter is arranged as in Patent Document 2, signals of different lines are output in a dot-sequential manner, and thus cannot be directly processed by an LSI that processes a signal in which pixels are mixed. Further, since the barycentric positions of the signals after pixel mixing are not equally spaced, aliasing noise is likely to occur.

  An object of the present invention is to provide an image pickup device that can obtain a good image quality when read by mixing pixels and an image pickup apparatus using the image pickup device.

  In order to solve the above problems, an imaging apparatus of the present invention includes a light receiving unit in which a plurality of pixels for photoelectric conversion are arranged in a horizontal direction and a vertical direction, and a mixing unit that mixes signals obtained from the pixels of the light receiving unit. The signal processing means for processing the output signal from the mixing means, and the mixing means mixes signals of three pixels that are continuous every other pixel in the vertical direction and two pixels that are continuous in the horizontal direction in the light receiving unit. It was set as the structure to output.

  Furthermore, a color separation filter that separates colors corresponding to each pixel is provided on the front surface of the light receiving unit, and the color separation filter is repeatedly arranged in the order of green, yellow, magenta, and yellow in the vertical direction in the Nth line in the vertical direction. The (N + 1) th line in the vertical direction is repeatedly arranged in the order of magenta, cyan, green, and cyan in the horizontal direction.

  According to the present invention, it is possible to provide an image pickup device that can obtain a good image quality when the pixel mixture is read and an image pickup apparatus using the image pickup device.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram showing a first embodiment of an image sensor according to the present invention, FIG. 2 is a diagram showing the arrangement of color separation filters arranged in front of the light receiving portion of the image sensor in FIG. 1, and FIG. FIG. 4 is a diagram showing the operation of the vertical transfer unit of the image sensor, and FIG. 4 is a diagram showing pulses applied to the vertical transfer unit of FIG.

  1 includes a vertical transfer unit 1, a horizontal transfer unit 2, a charge voltage conversion 3, an output buffer 4, a light receiving unit 5 arranged in n rows and m columns from P11 to Pnm, and a vertical transfer unit to a horizontal transfer unit. It is composed of a transfer circuit portion 6 for sending charges. Drive pulses V 1, V 2, V 3 for transferring charges are applied to the vertical transfer unit 1, and drive pulses H 1, H 2 for transferring charges are applied to the horizontal transfer unit 2. A pulse RG for resetting the charge is applied.

  In the present embodiment, the charges photoelectrically converted by the light receiving unit 5 are transferred to the vertical transfer unit 1 by independent read pulses R1, R2, and R3 for each row divided into N rows, N + 1 rows, and N + 2 rows (N is a natural number). Reading out. The charges photoelectrically converted by the light-receiving unit transfer the N-th row charge to the vertical transfer unit 1 by the read pulse R1, and then send the vertical transfer unit 1 downward by two lines in the drawing. Next, the charges of the (N + 1) th row are transferred to the vertical transfer unit 1 by the read pulse R2. Further, the vertical transfer unit 1 is sent two lines downward in the drawing. Finally, the charges on the (N + 2) th row are sent to the vertical transfer unit 1 by the read pulse R3. By this operation, all charges photoelectrically converted by the light receiving element are sent to the vertical transfer unit 1.

  The reading unit in FIG. 3 schematically represents this transfer operation. The light receiving portions P11 to P13, 1 in FIG. 3 show a part of the first row of P11 to Pnm in FIG. Further, T0 to T17 in FIG. 3 represent temporal changes. The pulse-like figure shown to the right of T0 to T17 represents the potential well of the vertical transfer unit 1, and shows how charges are transferred. The charges obtained at P11, P41, P71,... Of the light receiving unit 5 are transferred to the vertical transfer unit 1 at the timing T1. Two lines are sent from T2 to T5. At T6, the charges obtained at P31, P61, P91,... Of the light receiving unit 5 are transferred to the vertical transfer unit 1 and become two-line mixed charges P11 + P31, P41 + P61, P71 + P91. Similarly, two lines are sent from T7 to T10. At T11, the charges obtained at P51, P81, P11, 1... Of the light receiving unit 5 are transferred to the vertical transfer unit 1 and become P11 + P31 + P51, P41 + P61 + P81, P71 + P91 + P11,1, and the charges for three lines are mixed at the vertical transfer unit 1. Is done.

  In this way, all the charges obtained in the light receiving unit 5 are mixed every three lines and stored once in the vertical transfer unit 1, and the stored charges are calculated by the number of lines in the light receiving unit 5 in the horizontal blanking period (HBLK). Transfer 3 lines. The lowermost charges of the vertical transfer unit 1 in FIG. 1 are sent to the horizontal transfer unit 2, and the charges are sequentially transferred from the horizontal transfer unit 2 to the charge-voltage converter 3 and the output buffer 4 in the horizontal video period. A signal is read as an output voltage at 10 outputs.

  The transfer circuit portion 6 is provided so that the charge transfer from the vertical transfer unit 1 to the horizontal transfer unit 2 can be performed independently in the odd-numbered columns and the even-numbered columns. At the time of moving image shooting in which three lines are mixed vertically, an odd number column is sent to the horizontal transfer unit 2, and further, an even number column is sent to the horizontal transfer unit 2 so as to be added to the charge of the odd number column. The signal is read by operating.

  The portion described as “Repeat with HBLK” in FIG. 3 represents the transfer state of the vertical transfer unit 1 during the horizontal blanking period. In FIG. 4, the pulse applied to the vertical transfer unit 1 and the read pulse R1, R2 and R3 are specifically represented.

  In this embodiment, when shooting a moving image, three vertical pixels and two horizontal pixels are mixed and read out. On the other hand, when taking a still image, reading from the light receiving unit 5 to the vertical transfer unit and reading from the vertical transfer unit to the horizontal transfer unit can be independently read for each pixel per line. By performing a series of operations for 6 frames without any pixel mixing, information of all pixels can be read out independently. In addition, it is possible to reduce the readout time of moving images to 1/6 with respect to still images. Furthermore, since the number of transfer stages of the vertical transfer unit can be reduced, the image sensor can be downsized.

  Next, the relationship between the charge readout from the image sensor and the color separation filter provided in front of the light receiving unit 5 will be described with reference to FIG. The subscripts G, Ye, Mg, and Cy in FIG. 2 coincide with the subscript P in FIG. 1 and represent the position of the color filter arranged in front of the pixel. For example, G11 is a green filter provided in front of P11, and Ye12, Mg13, and Cy22 are yellow, magenta, and cyan filters provided in front of P12, P13, and P22, respectively.

  In this embodiment, the N line is repeated in the order of green, yellow, magenta, and yellow, and the N + 1 line is repeated in the order of magenta, cyan, green, and cyan.

  At the time of moving image shooting, (G11 + G31 + G51 + Ye12 + Ye32 + Ye52), (Mg13 + Mg33 + Mg53 + Ye14 + Ye34 + Ye54),... Read out a signal in one horizontal scanning period, . Considering the above output in terms of the number of pixels and lines seen from the output of the image sensor after mixing, four types of signals G + Ye, Mg + Ye, Mg + Cy, and G + Cy are output from adjacent two lines and two pixels. These four types of outputs are the same as those of the pixel mixture type image sensor used in the normal movie shown in FIG. In normal movie camera processing, RGB signals are obtained by adding or subtracting the above four types of signals with predetermined constants. Further, since the luminance signal is obtained by performing a low-pass filter process excluding the color signal, it can be processed only by changing the signal processing constant of the existing camera circuit.

  In the case of a still image, it is necessary to read over 6 frames and temporarily store it in a memory such as an SDRAM. By performing reading or writing in consideration of the pixel position on the image sensor, a signal can be obtained with the same output as when the pixels of the image sensor are sequentially scanned, and the four colors G, Ye, Mg, and Cy are displayed. It is output from two adjacent lines and two pixels. This is because the pixel mixing type image sensor used in the normal movie shown in FIG. 8 reads without pixel mixing, and once writes to the SDRAM, signals are sequentially read from the SDRAM in the pixel order of the image sensor. It is the same as the signal obtained when Therefore, processing can be performed only by changing the signal processing constants of the conventional signal processing after moving still, like the moving image.

  According to the present embodiment, since signals can be obtained from the image sensor by conventional signal processing, it is possible to realize a moving image and still image imaging device using a high pixel image sensor without developing a large-scale LSI. it can. Further, since 6 pixels are mixed during a moving image, a moving image can be obtained at a practical readout frequency.

  A second embodiment of the present invention will be described with reference to FIGS. In this embodiment, signal processing from the image sensor 10 is specifically shown.

  FIG. 6 shows a light receiving portion of the image sensor, and square pixels are used for the light receiving portion. FIG. 7 shows a block of signal processing. When shooting a moving image, 1440 pixels, which is three times the number of effective TV lines in the vertical direction, are used, and 1920 pixels, which is 4/3 times the vertical, are used in the horizontal direction. In actual camera processing, the above-mentioned 1440 and 1920 are slightly expanded to process the signal, including a transitional part due to filter processing. When used in a moving image, since 2 pixels in the horizontal direction and 3 pixels in the vertical direction are mixed, the image is output from the image sensor 10 as 960 pixels in the horizontal direction and 480 pixels in the vertical direction. A signal component is extracted from the output signal by the CDS, AGC, and AD unit 11 and converted into a digital signal. Then, the YUV generation unit 12 performs camera processing to generate a luminance and color difference signal. The luminance and color difference signals are subjected to video enlargement processing by the electronic zoom 13, and the pixel aspect ratio conversion 14 generates horizontal 960 pixels to 720 pixels. Horizontal 720 × vertical 480 frame video data to be recorded, or horizontal 720 × vertical 240 interlaced video is obtained in the Odd and Even fields. The generated video is recorded by the video recording unit 18 and converted into a TV display signal by the TV output signal generation 16 via the selection 15 and output.

  On the other hand, in still image shooting, the image sensor 10 is scanned six times without pixel mixing in the image sensor 10, the scan result is temporarily written in the memory 20, and the horizontal 1920 + α pixel × vertical 1440 + β pixel video is temporarily stored in the memory 20. To do. The stored images are read from the memory 20 in the order of the pixel arrangement of the image sensor 10 and input to the YUV generation 12 as with the moving image to perform camera processing and record. Here, α and β are margin pixels provided for camera shake correction processing at the time of moving image shooting, and are added by about 20% to 40% of the vertical and horizontal pixels.

  In this embodiment, even when an image sensor that can capture a still image with a high pixel by using 6-pixel mixing is used, there is no significant change in the angle of view between moving images and still images. Further, the number of pixels of a still image is 4.6 million pixels when the image stabilization area is ± 30% even when a square pixel imaging device is employed, and a still image with a high pixel can be obtained. Furthermore, since the moving image quality can be processed using all pixels, the occurrence of aliasing noise can be suppressed. In addition, the carrier frequency of the color signal itself can be increased by a factor of 1.5 by setting the horizontal to the two-pixel mixing while mixing the three pixels in the vertical. It is possible to suppress degradation of frequency characteristics due to filter processing that deletes the carrier when obtaining a luminance signal, to suppress degradation of limit resolution, and to realize moving image quality equivalent to or higher than that of a conventional movie including aliasing noise. be able to.

  FIG. 5 is a diagram showing a third embodiment (color separation filter) of the present invention. In this embodiment, the color separation filter in FIG. 2 is changed from a complementary color to a primary color, and the same configuration and operation as those of the first and second embodiments are obtained. it can.

  As described above, according to the embodiment of the present invention, since the signal amount can be reduced to 1/6, for example, at 1.5 times the readout speed when shooting a moving image of H × V = 640 × 480 pixels, Reading of 2,700,000 pixels of 1920 × 1440 becomes possible. If extra pixels are added to ± 10% to 20%, 4 to 5.5 million pixels can be realized with a still image. Since the difference in the angle of view between the moving image and the still image is only the margin pixel, no problem occurs. Furthermore, since signals of different lines do not come out point-sequentially, it can be processed even by a conventional moving image processing LSI. In addition, the horizontal repetition cycle of the color filter is 4 pixels compared to 2 pixels of the conventional moving image pickup device, but the number of horizontal pixels itself is tripled, so image quality degradation caused by aliasing noise is reduced. Can be suppressed. Furthermore, when creating a moving image from a 4: 3 or 16: 9 area on the light receiving surface of the image sensor, the data read from the image sensor is a signal obtained by equivalently multiplying the horizontal sampling frequency by 1.5. Therefore, since the carrier frequency of the output color signal is 1.5 times that of the prior art, the luminance and color can be separated from the image sensor output signal without causing deterioration of the frequency characteristic of the luminance signal. .

  In each of the above-described embodiments, a case has been described in which a moving image is captured and read out by mixing three vertical pixels and two horizontal pixels. However, the present invention is not limited to this, and can be extended to a case where a mixture of N pixels in the vertical direction and M pixels in the horizontal direction (where N> M and M ≧ 2) is read out, and similar effects can be obtained. Can do.

  The present invention is applicable to a video camera and a digital still camera.

The figure which shows the 1st Example of the image pick-up element by this invention. FIG. 3 is a diagram illustrating an arrangement of color separation filters in the first embodiment. The figure which shows operation | movement of the vertical transfer part in a 1st Example. The figure which shows the pulse applied to the vertical transfer part in a 1st Example. The figure which shows the 3rd Example (color separation filter) of this invention. The figure which shows the 2nd Example (mode of signal processing) of this invention. The signal processing block diagram in a 2nd Example. The figure which shows the arrangement | sequence of the color separation filter of the conventional image pick-up element.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Vertical transfer part 2 ... Horizontal transfer part 3 ... Charge voltage conversion 4 ... Output buffer 5 ... Light-receiving part 6 ... Transfer circuit part 10 ... Imaging element 11 ... CDS, AGC, AD
12 ... YUV generation 13 ... Electronic zoom 14 ... Pixel aspect ratio conversion 15 ... Selection 16 ... TV output signal generation 17 ... Image sensor drive circuit 18 ... Video recording unit 19 ..Synchronization signal generator 20 ... memory

Claims (8)

A light receiving unit in which a plurality of pixels for photoelectric conversion are arranged in a horizontal direction and a vertical direction;
Mixing means for mixing signals obtained at the respective pixels of the light receiving unit;
Signal processing means for processing the output signal from the mixing means,
The image pickup apparatus according to claim 1, wherein the mixing unit mixes and outputs signals of three pixels continuous every other pixel in the vertical direction and two pixels continuous in the horizontal direction in the light receiving unit. The imaging device according to claim 1,
A mode in which signals of a plurality of pixels are mixed and output using the mixing unit;
A mode for independently outputting a signal of each pixel without using the mixing means;
An imaging device comprising: The imaging device according to claim 1,
The image processing apparatus, wherein the signal processing means includes means for equivalently correcting an aspect ratio of a pixel with respect to the signal mixed by the mixing means. The imaging device according to claim 1,
A color separation filter that separates colors corresponding to the respective pixels is provided on the front surface of the light receiving unit,
In the color separation filter, the Nth line in the vertical direction is repeatedly arranged in the order of green, yellow, magenta, and yellow in the horizontal direction, and the (N + 1) th line in the vertical direction includes magenta, cyan, green, and cyan in the horizontal direction. An image pickup apparatus that is repeatedly arranged in order. The imaging device according to claim 1,
A color separation filter that separates colors corresponding to the respective pixels is provided on the front surface of the light receiving unit,
In the color separation filter, the Nth line in the vertical direction is repeatedly arranged in the order of green, green, red and green in the horizontal direction, and the (N + 1) th line in the vertical direction is red, blue, green and blue in the horizontal direction. An image pickup apparatus that is repeatedly arranged in order. A light receiving unit in which a plurality of pixels for photoelectric conversion are arranged in a horizontal direction and a vertical direction;
Mixing means for mixing signals obtained at the respective pixels of the light receiving unit;
Signal processing means for processing the output signal from the mixing means,
The image pickup apparatus characterized in that the mixing unit mixes and outputs signals of N pixels in the vertical direction and M pixels in the horizontal direction (where N> M and M ≧ 2). A light receiving unit in which a plurality of pixels for photoelectric conversion are arranged in a horizontal direction and a vertical direction;
Mixing means for mixing signals obtained at the respective pixels of the light receiving unit;
A color separation filter that separates colors corresponding to the pixels on the front surface of the light receiving unit;
The color separation filter is repeatedly arranged in units of 2 vertical lines and 4 horizontal pixels.
The mixing unit reads out the electric charge generated in each pixel in the light receiving unit to the vertical transfer unit independently for each of the vertical N lines, (N + 1) lines, and (N + 2) lines, and reads out the charges from the respective lines. In the meantime, an image pickup device is characterized in that the electric charge is read out by shifting the vertical transfer unit by two lines in the vertical direction. The image pickup device according to claim 7,
The mixing unit independently reads the vertical transfer N lines, the (N + 1) lines, and the (N + 2) lines to the vertical transfer unit, and reads the vertical transfer unit 2 in the vertical direction during the reading from each line. An image pickup device that shifts line by line and sends two adjacent rows in the horizontal direction to the same horizontal transfer unit to read out charges.

Images