JP2010171856A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of capturing a still image of long-time exposure with an excellent S/N, without interrupting photographing of a moving image or reducing a frame rate. <P>SOLUTION: In an imaging apparatus, an imaging device 102 includes a moving image readout operation for reading out signals of partial pixels in a plurality of pixels 200 in a prescribed cycle while shifting the pixel to be read out in a direction of rows by frame and a still image readout operation for reading out signals of all the pixels in the plurality of pixels 200. After it is instructed to photograph a still image during moving image photographing, a moving image is produced from signals of pixels obtained by the moving image readout operation until the lapse of a prescribed time Ts. After the lapse of the prescribed time Ts, a still image is produced from pixel signals obtained by the still image readout operation and from the pixel signals already obtained by the moving image readout operation. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus such as a digital camera or a digital video camera that captures still images and moving images.

  In an imaging device such as a digital camera or a digital video camera, when a moving image is captured using an image sensor such as a CMOS image sensor, a mechanical shutter cannot be used. Therefore, an electronic shutter function is provided to control the exposure amount of the image sensor. Necessary.

  As electronic shutters, a batch electronic shutter that sequentially reads out charges after being accumulated for each pixel of an image sensor during exposure and a slit rolling electronic shutter that can obtain appropriate exposure with high brightness have been proposed (Patent Document 1).

  Some image pickup apparatuses including this type of electronic shutter can record a still image at an arbitrary timing during recording of a moving image. In addition to recording image data, some moving images continuously display image data on a display device such as an LCD in order to realize an electronic viewfinder function.

  In an imaging apparatus having such a function, a moving image generated by continuously acquiring images from an imaging element is displayed on a display device without being recorded. After that, when a still image is instructed by a shutter button or the like of the image pickup apparatus, the electronic viewfinder function by moving image shooting is temporarily stopped and a still image is shot.

  By the way, in such an image pickup apparatus, when taking a still image while shooting a moving image, the shooting of the still image is usually performed after the shooting of the moving image is interrupted. It is desired that a still image can be taken at an arbitrary timing while continuing without interruption.

  However, the performance required for still images and moving images is different. In other words, a still image is required to have a high spatial resolution, and a temporal resolution is given priority to a moving image.

  For example, the number of pixels used for still images often exceeds 10 million pixels. On the other hand, in the case of moving images, the required number of pixels is about 300,000 pixels, and even if it corresponds to the full standard of HDTV (HDTV), it is about 2 million pixels. Required.

  For this reason, when capturing a moving image in an imaging apparatus capable of capturing still images, the number of pixels required is smaller than that of still images, and a method of reading out information of all pixels of the image sensor in the vertical direction is taken. Sometimes.

  Still image shooting is significantly different from moving images in that there is a demand for shooting a single still image by exposure much longer than the frame rate of the moving image.

  In this case, if all the pixels of the image sensor used for the still image can be read out at the frame rate of the moving image, all the moving images acquired from the image sensor during the exposure time of the still image are added for each pixel. Thus, an image corresponding to a long-exposure still image can be obtained.

JP 2007-028337 A

  However, since the noise from the readout circuit (hereinafter referred to as readout noise) is superimposed on the signal of the image sensor every time readout is performed, readout noise is also added as much as the addition of a plurality of pixels. N will deteriorate.

  Accordingly, an object of the present invention is to provide an imaging apparatus capable of obtaining a long-exposure still image with good S / N without interrupting moving image shooting or reducing the frame rate. To do.

  In order to achieve the above object, an image pickup apparatus according to the present invention includes an image pickup element in which a plurality of pixels including a photoelectric conversion element for photoelectrically converting a subject image are arranged in a row direction and a column direction, and the pixel from the image pickup element. And a control means for controlling the readout operation of the signal of the imaging device, wherein the imaging device shifts, in the row direction, pixels to be read out for each frame with respect to signals of some of the plurality of pixels. However, it has a readout operation for moving images that are read out in a predetermined cycle and a readout operation for still images that reads out the signals of all the pixels of the plurality of pixels, and instructs to take a still image during shooting of a moving image. After that, a moving image is generated from a pixel signal obtained by the moving image reading operation until a predetermined time elapses, and after the predetermined time elapses, the still image reading operation is performed. Generating still image from the pixel signal that has already been obtained by the read operation for pixel signals, and the moving image obtained I, characterized in that.

  According to the present invention, it is possible to obtain a long-exposure still image with good S / N without interrupting the shooting of moving images or reducing the frame rate.

It is a block diagram for demonstrating the structural example of the imaging device which is an example of embodiment of this invention. It is a circuit diagram for explaining a configuration example of one pixel among a plurality of pixels of an image sensor. It is a circuit diagram for demonstrating an example of the circuit which reads the signal from each pixel in common. It is a block diagram for demonstrating the example of a whole structure of an image pick-up element. It is explanatory drawing for demonstrating the example of arrangement | positioning of the pixel in an image pick-up element. It is a figure which shows typically the read-out operation | movement of the pixel of an image pick-up element. FIG. 10 is a timing chart for explaining a signal readout operation for pixels of one row. It is a timing chart for demonstrating the reset operation | movement of the signal of the pixel for 1 row.

  Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram for explaining a configuration example of an imaging apparatus which is an example of an embodiment of the present invention.

  As shown in FIG. 1, the imaging apparatus of the present embodiment includes an imaging element 102 that photoelectrically converts a subject image formed by an imaging lens 101, and the imaging element 102 is a CMOS image sensor in the present embodiment. Is used.

  The analog image signal output from the image sensor 102 is converted into a digital signal by the AFE 103. A DSP (Digital Signal Processor) 104 performs various types of image processing, compression / decompression processing, and the like on the digital image signal output from the AFE 103.

  A recording medium 105 is detachably attached to the imaging apparatus and records captured image data. The display unit 106 is configured by an LCD or the like, and displays captured images and various menu screens. A TG (timing generator) 107 supplies a drive signal to the image sensor 101. The CPU 108 controls the AFE 103, the DSP 104, and the TG 107. A RAM 109 is a memory for temporarily storing image data and the like.

  Next, a configuration example of the image sensor 102 will be described with reference to FIGS.

  FIG. 2 is a circuit diagram for explaining a configuration example of one image among the plurality of pixels 200 of the image sensor 102.

  In FIG. 2, a PD (photodiode) 201 is a photoelectric conversion element that photoelectrically converts an incident optical signal and accumulates charges according to the exposure amount. The transfer gate 202 sets the signal tx to the high level, whereby the charge accumulated in the PD 201 is transferred to the FD (floating diffusion) 203.

  The FD 203 is connected to the gate of the FD amplifier 204, and the charge amount transferred from the PD 201 by the FD amplifier 204 is converted into a voltage amount.

  The FD reset switch 205 is a switch for resetting the FD 203, and the FD 203 is reset by setting the signal resf to a high level. When resetting the charge of the PD 201, the signal tx and the signal resf are simultaneously set to the high level, so that both the transfer gate 202 and the FD reset switch 205 are turned on, and the PD 201 is reset via the FD 203. It will be.

  The pixel selection switch 206 outputs the pixel signal converted into a voltage by the FD amplifier 204 to the output vout of the pixel 200 by setting the signal sel to a high level.

  FIG. 3 is a circuit diagram for explaining an example of a circuit for commonly reading signals from the respective pixels 200.

  In FIG. 3, the vertical output line 301 is provided for each column, and the output vout of the pixel 200 for one column is connected thereto.

  The S signal transfer switch 302 is a switch for transferring the pixel signal S read from the pixel 200 to the storage capacitor 304. By setting the signal ts to High level, the pixel signal S of the vertical output line 301 is held in the holding capacitor 304 via the S signal transfer switch 302.

  The N signal transfer switch 303 is a switch for transferring the noise signal N read from the pixel 200 to the storage capacitor 305. By setting the signal tn to the high level, the noise signal N of the vertical output line 301 is held in the holding capacitor 305 via the N signal transfer switch 303.

  The horizontal transfer switches 306 and 307 are turned on when the signal ph becomes High level, and transfer the pixel signal S of the storage capacitor 304 and the noise signal N of the storage capacitor 305 to the horizontal output line 308.

  The horizontal output line 308 is connected to the input of the differential amplifier 309. The differential amplifier 308 obtains a difference between the pixel signal S and the noise signal N and simultaneously applies a predetermined gain to output a final image signal. Output to the terminal.

  Here, the column common readout circuit 300 is arranged for each pixel column, and the vertical output line 301, the S signal transfer switch 302, the N signal transfer switch 303, the holding capacitors 304 and 305, and the horizontal transfer switches 306 and 307 are used for imaging. There are as many pixel columns as the elements 102 have.

  FIG. 4 is a diagram for explaining an example of the overall configuration of the image sensor 102.

  As shown in FIG. 4, in the image sensor 102, a plurality of pixels 200 are arranged in a row direction and a column direction, and each column is connected to a column common readout circuit 300. In FIG. 4, the horizontal transfer switches 306 and 307 are shown separately from the column common readout circuit 300 for convenience, but are actually included in the column common readout circuit 300 as described above.

  The vertical scanning circuit 401 supplies drive signals such as resp1, resf1, tx1, sel1, and the like to each pixel 200. The horizontal scanning circuit 402 sets the output signal ph to High level, whereby the signals of the storage capacitors 304 and the storage capacitors 305 in each column are sequentially transferred to the horizontal output line 308 and further output via the differential amplifier 309. Output to the terminal.

  FIG. 5 is an explanatory diagram for explaining an arrangement example of the pixels 200 in the image sensor 102. In FIG. 5, for convenience of explanation, the number of rows of pixels 200 is only 15 rows. Further, when configuring a still image, one screen is configured based on the signals of all the pixels 200, and when generating a moving image, the signals of the pixel row 501 that are some pixels of the image sensor 102 are used. Based on this, one screen is constructed. This readout operation of the pixel row 501 corresponds to a readout operation of a solid line frame 601 in FIG. The pixels in the pixel row 502 and the pixel row 503 are pixels that are not used in the moving image.

  In FIG. 5, the 5n + 1 row is the pixel row 501, but when long-exposure still image shooting starts during moving image shooting, pixels used for the moving image in the subsequent moving image frame The row 501 is shifted in the row direction (arrow A direction) for each frame. Similarly, the pixel rows 502 and 503 are shifted in the row direction.

  At this time, as will be described later, in the pixel row 502, in order to read out the charges accumulated so far in order to generate a still image, a reading operation of a hatched frame 603 in FIG. 6 described later is performed. Further, it is assumed that the pixel row 503 is thinned out without being read from the image sensor 102 when reading a signal for a moving image.

  Next, referring to FIG. 6 to FIG. 8, the pixel signal reading operation at the time of capturing a moving image and the pixels at the time of capturing a long-exposure still image during capturing of the moving image in the image sensor 102. A signal reading operation will be described.

  FIG. 6 is a diagram schematically illustrating a signal read operation of the pixel 200 of the image sensor 102. The horizontal axis in FIG. 6 indicates time, and the vertical axis indicates the number of pixel rows.

  In FIG. 6, a solid line frame 601 and a hatched frame 603 indicate an operation of reading pixel signals for one row of the image sensor 102 from the PD 201 to the output terminal of the image sensor 102 via the FD 203.

  A solid line frame 601 is a pixel signal reading operation for generating a moving image of a frame, and the read signal is used to generate a moving image and is also signaled to a finally generated still image. Is temporarily stored in the RAM 109.

  The hatched frame 603 is an operation for reading out the pixel signal accumulated so far in order to start accumulation of the pixel signal for the moving image of the next frame, and this signal is applied to the finally generated still image. For use, it is temporarily stored in the RAM 109. In either case of the solid line frame 601 and the hatched frame 603, the signal reading operation is the same. In these readout operations, as will be described later, in the readout operation, the charge of the PD 201 is reset immediately after the readout of the signal and enters the accumulation state in preparation for the next readout.

  A broken line frame 602 is a reset operation for clearing the charge of the PD 201 in a pixel row that does not use a pixel signal in the moving image by one row. At this time, the operation of the readout circuits after the vertical output line 301 is unnecessary, and the power consumption can be reduced by turning off the currents of the readout circuits.

  Next, with reference to FIG. 7 and FIG. 8, the operation timings for one row of the solid line frame 601, the broken line frame 602, and the hatched frame 603 will be described in order.

  FIG. 7 is a timing chart for explaining the reading operation for one row of the solid line frame 601 and the slanted line frame 603.

  In FIG. 7, numbers added after resf, sel, and tx correspond to the numbers of pixel rows from which signals are to be read. Here, the first line is described as an example. The number added to ph indicates the number of the pixel column, ph1 is a signal ph input to the first column read circuit, and ph2 is a signal ph input to the second column read circuit. Is shown.

  First, the signal sel is set to High level, and the pixel selection switch 206 is turned on. Thereafter, the signal resf is set to the Low level, the FD reset switch 205 is turned OFF, and the reset of the FD 203 is released.

  Next, the signal tn is turned ON and the N signal is stored in the holding capacitor 305 via the N signal transfer switch 303. Subsequently, the signal tn is set to the low level, the N signal transfer switch 303 is turned off, the signal ts is set to the high level, the S signal transfer switch 302 is turned on, and the transfer gate 202 is turned on by setting the signal tx to the high level. To do.

  By this operation, the signal accumulated in the PD 201 in the selected row is output to the vertical output line 301 via the FD amplifier 204 and the pixel selection switch 206, and further, the storage capacitor 304 via the S signal transfer switch 302. Remembered.

  Next, the signals tx and ts are set to Low level, the transfer gate 202 and the S signal transfer switch 302 are closed, the signal resf is set to High level, the FD reset switch 205 is turned on, and the FD 203 is reset.

  Thereafter, the horizontal transfer switches 306 and 307 are sequentially turned on by the signal ph of each column, and the signal of the storage capacitor 304 and the signal of the storage capacitor 305 of each column are output to the output terminal via the horizontal output line 308 and the differential amplifier 309. To do.

  The signals resf and tx are set to the high level again during the period when the signal of each column is output by the signal ph, but the PD 201 is reset via the FD reset switch 205 and the transfer gate 202 by this operation.

  FIG. 8 is a timing chart for explaining the reset operation for one row of the broken line frame 602.

  In the reset operation, the signal sel of the signal read operation for one row (FIG. 7) is simply fixed at the low level. Since the pixel selection switch 206 in the reset row is OFF, the operations of the signals ts, tn, and ph are not directly related to each other, and are indicated by dotted lines.

  Here, the ts, tn, and ph signals in FIG. 8 are the same as in the readout operation shown in FIG. 7, but since the pixel signals are not read out, these signals are fixed at the low level to reduce power consumption. It is also possible to plan. When the signals resf and tx are at a high level during the period when the signal ph is output, the signal of the PD 201 is reset.

  Returning to FIG. 6, the operation of acquiring a moving image and acquiring a still image during shooting of the moving image will be described.

  During shooting of a moving image before a still image shooting instruction is given, pixel signals in a row (here, 5n + 1 row) used for the moving image are picked up at a predetermined cycle by a reading operation of the solid line frame 601. A moving image is generated by reading from the element 102. Regarding the lines other than the lines used for the moving image, the charge of the PD 201 is reset for each frame by the reset operation of the broken line frame 602.

  Since the read pixel signals are thinned out in the row direction (vertical direction), if the image is generated as it is, the aspect ratio of the image does not match, so the DSP 104 changes the pixel information in the column direction (horizontal direction). A moving image is generated after thinning processing.

  Next, at time t0, a still image is instructed by pressing a shutter switch (not shown). When an instruction to shoot a still image is given, first, in one frame period (frame a) from time t1, pixels in the row (5n + 1 row) used for the moving image read the signal accumulated in the PD 201. After being discharged, the charge of the PD 201 is reset and enters the next accumulation state.

  Further, the pixels in the rows (5n + 2, 5n + 3, 5n + 4, 5n + 5) other than the row used for the moving image are reset without reading the signal of the PD 201 and enter the next accumulation state. In the DSP 104, one frame for moving image is generated from the obtained pixel signals in the 5n + 1th row in the same manner as before time t1.

  In the frame b period from time t2, the pixel signals of the 5n + 1 and 5n + 2 rows are read out, and the other rows (5n + 3, 5n + 4, 5n + 5 row) are not reset and read out and are stored as they are. Will continue.

  Even in the frame b, a moving image is generated from the signal in the (5n + 1) th row. At the same time, the signals in the 5n + 1 and 5n + 2 rows are temporarily stored in the RAM 109 as still image signals.

  In the frame c period from the time t3, the pixel signals of the 5n + 2 and 5n + 3 rows are read, and the other rows (5n + 1, 5n + 4, 5n + 5 rows) are not reset and read, and are stored as they are. Will continue.

  In the frame c, a moving image is generated from the signal on the 5n + 2nd row. At the same time, the signals in the 5n + 2 and 5n + 3 rows are temporarily stored in the RAM 109 as still image signals. The signal in the 5n + 2th row is added to the previous pixel data (frame b data) already stored in the RAM 109 and stored.

  In the frame d period from the time t4, the pixel signals of the 5n + 3th row and the 5n + 4th row are read, and the other rows (5n + 1, 5n + 2, 5n + 5th row) are not reset and read, and are stored as they are. Will continue.

  In frame d, a moving image is generated from the signal in the (5n + 3) th row. At the same time, the signals on the 5n + 3 and 5n + 4 rows are temporarily stored in the RAM 109 as still image signals. The signal on the 5n + 3th row is added to the previous pixel data (frame c data) already stored in the RAM 109 and stored.

  In the frame e period from time t5, the pixel signals of the 5n + 4th row and the 5n + 5th row are read out, and the other rows (5n + 1, 5n + 2, 5n + 3th row) are not reset and read out, and are stored as they are. Will continue.

  In the frame e, the moving image is generated from the signal on the 5n + 4th row. At the same time, the signals in the 5n + 4th and 5n + 5th rows are temporarily stored in the RAM 109 as still image signals. The signal in the 5n + 4th row is added and stored with the previous pixel data (frame d data) already held in the RAM 109.

  In the frame f period from time t6, the pixel signals of the 5n + 5th row and the 5n + 1th row are read out, and the other rows (5n + 2, 5n + 3, 5n + 4th row) are not reset and read out and are stored as they are. Will continue.

  In the frame f, a moving image is generated from the signal on the 5n + 5th row. At the same time, the signals of the 5n + 5th row and the 5n + 1th row are temporarily stored in the RAM 109 as still image signals. The signals in the 5n + 5th row and the 5n + 1th row are added and stored with the previous pixel data already stored in the RAM 109 (data before the frame e).

  Then, when a predetermined accumulation time Ts of the still image has elapsed, the operation of the frame g is started. If the still image accumulation time is not reached even after the time t7, the operation of the frames b to f is repeated, the moving image is acquired while shifting the row to be read for each frame, and the read signal is stored in the RAM 109. The data is stored while being added to the data for the still image held.

  In the frame g, the signals of all the rows used for the still image including the row used for the moving image (5n + 1th row) are read out. In frame g, a moving image is generated from the signal on the (5n + 1) th row. Further, the read signals of all rows are added to the still image data held in the RAM 109, respectively.

  Thereby, in all the pixels of the image sensor 102, an accumulation signal corresponding to the time of the still image accumulation time Ts is obtained, and a long-time accumulation still image is generated.

  Here, the period B in FIG. 6 indicates the accumulation period of the moving image signal, and the period C indicates the accumulation period of the still image signal (the signal accumulated in the period B is finally Is also used for still images).

  The signal accumulated in the period B is read out every frame, but the signal accumulated in the period C is read out after the accumulation is continued over a plurality of frames. As a result, the number of readouts for obtaining a single still image signal can be suppressed, and mixing of noise can be suppressed as much as possible.

  As described above, in this embodiment, although the still image accumulation time Ts is a time corresponding to 6 frames, each pixel is read out only three times. For this reason, readout noise is also reduced to three times, and a still image with less noise can be obtained. As a result, it is possible to obtain a long-exposure still image with good S / N without interrupting the shooting of a moving image or reducing the frame rate.

  Further, when the still image accumulation time Ts is longer, it can be dealt with by repeating the operations of the frames b to f in FIG. When the thinning cycle of a pixel row for generating a moving image is large, the number of times of reading out each pixel can be suppressed to a smaller value, so that a still image with less noise can be obtained.

  In this embodiment, a slit rolling shutter has been described as a configuration that realizes an electronic shutter (however, since it is premised on shooting in the dark that requires long exposure, a configuration in which the PD 201 is reset almost simultaneously with reading. ) However, a configuration using a collective electronic shutter that simultaneously transfers the charges of the PDs 201 of all pixels to the FD 203 may be used.

  In the present embodiment, the same row (5n + 1 row in the embodiment) is read out every frame during the period for capturing only moving images. However, as in the case of accumulation of still images, the readout row is read every frame. You can change it. Furthermore, the period for capturing only a moving image may be configured to read out signals of all pixels every frame.

  In this embodiment, while still images are being accumulated, the row from which a signal is read is shifted by one row for each frame. However, the combination is not limited to this as long as it is different for each frame. In consideration of the arrangement, a method such as shifting every two rows may be adopted.

  Further, in this embodiment, it is described that all pixels can be read out during one frame period. Therefore, it is possible to continue shooting a moving image at the same frame rate even after acquiring a still image. is there. However, the configuration may be such that once the still image is acquired, the shooting of the moving image is interrupted. In that case, a configuration in which all the pixels cannot be read within one frame period in time may be used.

  In addition, this invention is not limited to what was illustrated to the said embodiment, In the range which does not deviate from the summary of this invention, it can change suitably.

102 Image sensor 201 PD
202 Transfer gate 203 FD
205 FD reset switch 206 Pixel selection switch 300 Column common readout circuit

Claims (4)

An imaging device comprising: an imaging device in which a plurality of pixels including a photoelectric conversion device for photoelectrically converting a subject image are arranged in a row direction and a column direction; and a control unit that controls a reading operation of a signal of the pixel from the imaging device. A device,
The image pickup device, for a signal of a part of the plurality of pixels, for a moving image that is read out in a predetermined cycle while shifting pixels to be read out for each frame in a row direction;
A readout operation for a still image that reads out signals of all the pixels of the plurality of pixels, and
A moving image is generated from a pixel signal obtained by the moving image readout operation until a predetermined time elapses after a still image is instructed during moving image shooting.
After the predetermined time has elapsed, a still image is generated from a pixel signal obtained by the still image readout operation and a pixel signal already obtained by the moving image readout operation. Imaging device.   The imaging apparatus according to claim 1, wherein the moving image readout operation does not reset charges of the photoelectric conversion elements of pixels that are not read out in each frame.   The imaging apparatus according to claim 1, wherein in the moving image readout operation, pixels of the imaging element are read out in a row direction.   The imaging apparatus according to any one of claims 1 to 3, wherein, in the moving image readout operation, pixels to be read out for each frame are read out while being shifted by one row or two rows in the row direction.

Images