JP2010136078A - Imaging apparatus and driving control method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To correct smears included in short-time exposure data when adding each of imaged data in long-time exposure and the short-time exposure to achieve a wide dynamic range. <P>SOLUTION: The imaging apparatus simultaneously starts exposing pixels for long-time exposure and pixels for short-time exposure, and forms an empty packet in an empty space other than the space where a packet for signal charge provided in a charge transfer path, to transfer the accumulated charge when transferring and outputting the accumulated charge of the pixels for short-time exposure to an adjacent charge transfer path in a packet for readout signal charge when the short-time exposure finishes during the long-time exposure, and obtains data by subtracting data (smear) corresponding to a charge amount transferred in the empty packet from data (signal charge+smear) corresponding to a charge amount transferred in the packet for signal charge as captured image data by the short-time exposure. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus capable of capturing an image with a wide dynamic range and high quality, and a drive control method thereof.

  When a high-brightness subject is imaged by a digital camera or the like equipped with a solid-state image sensor, smear occurs. Since smear deteriorates the quality of the subject image, smear correction becomes important.

  For example, in the prior art described in Patent Document 1 below, signal data is read from a photodiode to a charge transfer path and transferred through the charge transfer path in the current field, and image data (actually [signal charge amount + smear] In the next field, the charge transfer path is idled to output image data based only on the smear charge amount, and correction is performed so that a subject image based only on the true signal charge amount is obtained. Yes.

  However, in the case of this prior art, there is a time difference because the readout field for signal charges is different from the readout field for smear charges only, and there is a time difference between the smear charge amount superimposed during signal charge read transfer and the smear charge amount read in the next field. There will be a difference between Further, since it is necessary to provide a smear charge reading field (a field for empty transfer of the charge transfer path) after the signal charge reading field, it takes time to read the captured image data of the subject.

  In order to capture a subject image with a wide dynamic range, the imaging device captures about half of the many pixels mounted on the solid-state imaging device in one imaging process as described in Patent Document 2 below. There is a technique in which long-time exposure is performed, and the remaining half of the pixels are subjected to short-time exposure that overlaps during long-time exposure, and a long-time exposure image and a short-time exposure image are combined.

  Although smear correction is important even in such an imaging apparatus, it is impossible to perform highly accurate smear correction simply by applying the smear correction described in Patent Document 1 as it is.

Japanese Patent Laid-Open No. 11-177888 JP 2007-235656 A

  What kind of smear correction is used in an imaging device that uses about half of the plurality of pixels (photodiodes) formed on the surface of the solid-state imaging device for long-time exposure and the other half for short-time exposure? It has not yet been proposed whether high quality and wide dynamic range subject image data can be obtained.

  An object of the present invention is to provide an imaging apparatus capable of performing smear correction with high accuracy in an imaging apparatus equipped with a solid-state imaging device in which pixels that are exposed for a long time and pixels that are exposed for a short time are mixed, and a drive control method thereof. There is.

  In the image pickup apparatus and the drive control method thereof according to the present invention, a plurality of first pixels that perform long-time exposure shooting and a plurality of second pixels that perform short-time exposure shooting overlapping a part of the long-time exposure are mixed. The first pixel and the first pixel of the solid-state imaging device, in which a charge transfer path is formed along each of a plurality of pixel columns that are both formed in a two-dimensional array and configured with the first pixel and the second pixel. The exposure of two pixels is started at the same time, and when the short-time exposure is completed during the exposure of the first pixel, the accumulated charge of the second pixel is read out to the adjacent charge transfer path and transferred as a signal charge packet and output. When the signal charge packet is formed in an empty space other than the signal charge packet in the charge transfer path, the data corresponding to the amount of charge transferred in the empty packet is transferred by the signal charge packet. Charge that has been transferred Characterized in that the captured image data by the short exposure data that is obtained by subtracting from the data corresponding to the.

  According to the present invention, when the detection charge of the short-time exposure pixel (second pixel) for which deterioration of image quality due to mixing of smear charges is a concern is read out and transferred to the charge transfer path, an empty packet is transferred at the same time. Since output data of only smear charge is obtained, it is possible to obtain image data by short-time exposure without the influence of smear. By adding this to image data obtained by long-time exposure, high quality and wide dynamic range can be obtained. The subject image can be obtained.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram of the imaging apparatus according to the present embodiment. The imaging device (digital still camera in this example) 10 includes a CCD solid-state imaging device 11, a mechanical shutter 12 placed in front of the solid-state imaging device 11, a photographing lens 13, a diaphragm (iris) 14, A CDSAMP (correlated double sampling (CDS), gain control amplifier (AMP)) 15 that performs analog signal processing on an output signal (captured image signal) of the solid-state imaging device 11, and an analog digital that converts the output signal of the CDSAMP 15 into a digital signal ( A / D) converter 16.

  The imaging apparatus 10 further includes an image input controller 21 that captures a captured image signal that is a digital signal output from the A / D converter 16, and an arithmetic processing unit (CPU) 22 that performs overall control of the entire imaging apparatus 10. The image signal processing circuit 23 that processes the captured image signal, the AF & AE & AWB detection circuit 24 that automatically detects the focal position, the exposure amount, and the white balance from the image data output from the solid-state imaging device 11, and the memory used as a work memory An SDRAM 25, a frame memory (VRAM) 26, a line memory 27, a compression processing circuit 28 for compressing captured image data after image processing into a JPEG image or an MPEG image, and a liquid crystal display provided on the back of the camera A video encoder 30 for displaying a captured image or a through image on the device 29; a recording medium; 31 includes a media controller 32 that stores captured image data, a face detection circuit 33, a camera shake detection circuit 35 that detects a camera shake amount from a signal from the angular velocity sensor 34 and through image data, and a bus 36 that interconnects them. Prepare.

  The imaging apparatus 10 further includes a motor driver 41 that supplies a driving pulse to the driving motor 12a of the mechanical shutter 12, a motor driver 42 that supplies a driving pulse to the motor 13a that drives the focus lens position of the photographing lens 13, and an aperture. A motor driver 43 that supplies a drive pulse to a drive motor 14a that controls the aperture position 14; a timing generator 44 that supplies a drive timing pulse (electronic shutter pulse, readout pulse, transfer pulse, etc.) to the solid-state imaging device 11; A CCD shift drive motor driver 45 that supplies drive pulses to a motor 11a that shifts the mounting position of the image pickup device 11 and cancels camera shake, and operates based on a command from the CPU 22. The CDSAMP 15 also operates based on a command from the CPU 22.

  The CPU 22 is further connected to a switch 48 for switching between the photographing mode / reproduction mode, a shutter release button 49 for a two-stage shutter, and a photographing mode input unit 50, and these are input from the switches 48 and 49 and the input unit 50. Based on the user instruction, the CPU 22 controls the imaging device 10.

  FIG. 2 is a schematic view of the surface of the CCD type solid-state imaging device 11 shown in FIG. In the solid-state imaging device 11, a plurality of photodiodes (portions shown by octagons in the figure) 51 are arranged in a two-dimensional array on the surface portion of the semiconductor substrate.

  When the odd-numbered photodiode group is the first pixel group and the even-numbered photodiode group is the second pixel group, the first pixel group and the second pixel group are formed by being shifted by 1/2 pixel pitch. As a whole, a so-called honeycomb pixel arrangement in which the first pixel group and the second pixel group are mixed is formed.

  If only the first pixel group is seen, each pixel 51 is arranged in a square lattice, and color filters of the three primary colors of red (R), green (G), and blue (B) are arranged on each pixel in a Bayer arrangement. Even when only the second pixel group is seen, the pixels 51 are arranged in a square lattice, and the color filters of the three primary colors of red (r), green (g), and blue (b) are Bayer arranged on each pixel. “R, G, B” and “r, g, b” are the same color, but in order to distinguish between the first pixel group and the second pixel group, the color of the color filter is expressed in uppercase and lowercase letters. Yes.

  A vertical charge transfer path (VCCD) 52 is formed to meander along each pixel column, and a horizontal charge transfer path (HCCD) 53 is formed along the transfer direction end of each vertical charge transfer path 52 for horizontal charge transfer. An amplifier 54 is provided at the output end of the path 53 to output a voltage value signal corresponding to the amount of transferred signal charge as a captured image signal.

  Each pixel 51 is provided with two transfer electrodes (see FIG. 5) not shown in FIG. 2 for each pixel, and one of the two transfer electrodes, for example, the lower (HCCD 53 side) transfer electrode. Is also used as a readout electrode. A readout pulse for the first pixel group (VA: see FIG. 4) is commonly applied to the readout electrodes of the first pixel group, and a readout pulse for the second pixel group (VB: (See FIG. 4).

  Hereinafter, the first pixel group is described as a pixel for long exposure time, and the second pixel group is described as a pixel for short exposure time. The terms “vertical” and “horizontal” are used, but this merely means “one direction” along the surface of the semiconductor substrate and “a direction substantially perpendicular to the one direction”.

  FIG. 3 is a flowchart showing an operation procedure of the imaging apparatus described with reference to FIGS. 1 and 2, and FIG. 4 is a timing chart of a read operation according to this operation procedure.

  When the shutter release button 49 of the imaging device 10 is fully pressed, the processing procedure of FIG. 3 is started. Note that image data (through image data) output from the solid-state imaging device 11 and displayed on the display device 29 in the half-pressed state before the shutter relay button 49 is fully pressed is analyzed by the CPU 22 to determine the focus position, the exposure amount, and the like. In addition, the ratio between the exposure time for long exposure and the exposure time for short exposure is determined according to the user designation or the dynamic range width required for the shooting scene.

  When the shutter release button 49 is fully pressed, the application of the electronic shutter pulse is stopped at timing t1 in FIG. 4 under the “open” state of the mechanical shutter 12. Each time an electronic shutter pulse is applied to the semiconductor substrate of the solid-state imaging device 11, the accumulated charge of the pixel 51 is discarded on the substrate side, and by stopping the application of the electronic shutter pulse, a signal corresponding to the exposure amount is sent to each pixel 51. Charge accumulation, that is, exposure is started. That is, the long exposure of the first pixel group and the short exposure of the second pixel group are started simultaneously (step S2).

  In the next step S3, it is determined whether or not the exposure time of short exposure has elapsed from the start of exposure. If not, the process returns to step S2 and executes step S3 again. At the timing t2 when the exposure time of the short exposure has elapsed, the process proceeds from step S3 to step S4, where the readout pulse VB is applied to each pixel 51 belonging to the second pixel group, and the signal charge from each pixel 51 becomes the adjacent vertical charge. Read out to the transfer path 52.

  In the next step S5, a vertical transfer pulse is applied to each vertical charge transfer path 52 to perform charge transfer driving in the vertical direction. As a result, as shown in FIG. 5, an empty packet is sent on the vertical transfer path adjacent to the first pixel group, and only the “smear charge” (indicated by a circle in FIG. 5) is transferred in the vertical direction. In the vertical transfer path adjacent to the pixel group, “signal charge + smear charge” is transferred in the vertical direction. Along with the transfer in the vertical direction, the transfer in the horizontal direction through the horizontal charge transfer path 53 is also performed, and the amplifier 54 outputs image data corresponding to the transferred charge amount by a timing t3 before a timing t4 described later. End up. The output data is stored in the memory 26.

  Even during the transfer drive in the vertical direction and the horizontal direction, the mechanical shutter 12 is in the “open” state, so that each pixel 51 is maintained in the exposure state (step S6) and charges are accumulated. In the next step S7, it is determined whether or not the exposure time of long exposure has elapsed since the start of exposure.

  If the exposure time of the long exposure has not elapsed, the process returns to step S6. If the exposure time has elapsed, a close command signal is applied to the mechanical shutter 12, and the mechanical shutter 12 is closed (step S8). Then, the vertical charge transfer path is driven at a high speed while the mechanical shutter 12 is closed, so that smear charges are swept out from the vertical transfer path 52 (step S9).

  At timing t5 after this high-speed sweep, a read pulse VA is applied to each pixel 51 of the first pixel group (step S10). As a result, the signal charge is read from each pixel 51 of the first pixel group to the adjacent vertical charge transfer path 52, and the vertical charge transfer path 52 and the horizontal charge transfer path 53 are driven to transfer, thereby performing long exposure. Image data is output (step S11). The output image data is stored in the memory 26. Since the image data by this long exposure is performed under the mechanical shutter “closed” state, the smear charge can be regarded as zero in principle.

  After the image data obtained by the short exposure and the image data obtained by the long exposure are output from the solid-state imaging device 11, smear correction is performed using the image data output in step S5 in the next step S12.

  That is, the output data based on the charges read and transferred from each pixel of the pixel array for the short-time exposure becomes output data [signal charges + charges] obtained by adding the smear charges to the signal charges by the short-time exposure. ing. On the other hand, data obtained by idle transfer in the vertical charge transfer path along the long exposure pixel column adjacent to the short exposure pixel column is output data of only [smear charge].

  Therefore, smear correction can be performed by subtracting the data of [smear charge] from the data of [signal charge + smear charge] using data obtained from the same vertical position or a neighboring vertical position.

  Strictly speaking, the amount of smear charge is different when the vertical charge transfer path is different, but in the present embodiment, smear correction is performed based on the smear charge amount of the adjacent vertical charge transfer path, so the smear charge amount is accurate. Correction can be performed. Smear correction is performed on the data obtained by transferring the middle vertical charge transfer path using the value obtained by averaging the smear charge amounts obtained by empty transfer of the adjacent vertical charge transfer paths across the pixel column for short-time exposure. Thus, smear correction with higher accuracy is possible.

  In the next step S13, subject image data obtained from the first pixel row by long exposure and subject image data obtained from the second pixel row and subjected to smear correction in step S12 are added and combined on a pixel basis. Thus, subject image data with a wide dynamic range is obtained. In FIG. 2, the addition synthesis is performed by adding the data of a certain pixel in the first pixel group and the data of the pixel of the second pixel group that is diagonally lower right of the same color with respect to this pixel.

  In the next step S14, well-known signal processing such as γ correction processing and RGB / YC conversion processing is performed on each of the added and synthesized image data, and in the case of a still image, it is converted into JPEG image data and recorded on the recording medium 31. (Step S14), this process is terminated, and the next shooting is awaited.

  As described above, it is possible to eliminate the influence of smear and obtain a subject image with a wide dynamic range.

  The smear correction process in step S12 described above is performed after the image data of the long exposure is not output but immediately after step S5, in parallel with the processes of step S6 and step S7. Shortening is also possible.

  In this case, for example, every time data for one horizontal line is read from the solid-state imaging device 11, it is temporarily stored in the line memory 27 shown in FIG. 1, where smear charge subtraction processing is performed, and the captured image data after smear correction is performed. Are stored in the A plane of the frame memory 26 shown in FIG. Captured image data obtained by long-time exposure is stored in the B surface of the memory 26, and then the image data of a wide dynamic range can be obtained by adding the A surface data and the B surface data and performing signal processing. .

  FIG. 6 is a schematic view of the surface of a solid-state imaging device 60 mounted on an imaging apparatus according to another embodiment of the present invention. In the solid-state imaging device 60, each pixel 61 is arranged in a square lattice, a vertical charge transfer path 62 is formed along each pixel column, and a horizontal charge transfer path 63 is formed along the transfer direction end of each vertical charge transfer path 62. And an output amplifier 64 is provided at the output end of the horizontal charge transfer path 63.

  In the illustrated example, three vertical transfer electrodes are provided for each pixel 61, one of which is provided between the pixels 61 in the vertical direction. Among the pixels 61, the pixels at the checkered position are defined as the first pixel group (for long exposure), and the color filters R, G, and B of the three primary colors are stacked, and the remaining pixels at the checkered position are defined as the second pixel group ( Three primary color filters r, g, and b are stacked as a short exposure). The green (G, g) color filters are provided in a vertical (vertical) stripe shape, and red (R, r) and blue (B, b) are provided so as to be continuous by two pixels in the vertical direction. That is, the vertical pixel addition (long exposure pixel + short exposure pixel) is facilitated.

  In the solid-state imaging device 60 shown in FIG. 6, each pixel 61 of the first pixel group (denoted as R, G, B) and the lower (horizontal charge transfer path side) electrode among the two adjacent vertical transfer electrodes. Between each pixel 61 of the second pixel group (indicated as r, g, b) and the upper electrode (on the anti-horizontal charge transfer path side) of the two adjacent vertical transfer electrodes; A readout gate 66 is provided between the pixel 61 and the signal charge of each pixel 61 of the first pixel group and the signal charge of each pixel 61 of the second pixel group so that they can be read out to the vertical charge transfer path 62 at different timings. It has become.

  In the imaging apparatus equipped with the solid-state imaging device 60 having such a configuration, when the signal charge of the second pixel group after the short exposure is read and transferred to the vertical charge transfer path 62 as shown in FIG. 7 is expressed as a charge amount [r + ◯, g + ◯, b + ◯] obtained by adding smear charges to each other). The vertical direction of the signal charge transfer packet By transferring an empty packet to an intermediate position, it is possible to detect only the smear charge [O].

  It is possible to detect the amount of signal charge due to true short-time exposure by correcting the amount of charge sent in the signal charge packet immediately before or immediately after the vertical direction by the smear charge detected by this empty packet. Become. Of course, the data corresponding to the charge amount sent in the signal charge packet may be smear corrected with the data obtained by averaging the data corresponding to the smear charge amount obtained in the empty packet immediately before and after the vertical direction. Good.

  After reading out the signal charge and smear charge due to the short exposure, the signal charge due to the long exposure is read out. In this case, the mechanical shutter 12 is performed in the “closed” state. There is no smear charge. However, even in this case, it is also possible to send an empty packet between the signal charge packets, and to correct the charge amount sent in the signal charge packet with the charge amount obtained by the empty packet.

  As described above, the imaging apparatus and the drive control method thereof according to the above-described embodiments have a plurality of first pixels that perform long-exposure shooting and a plurality of short-exposure shootings that overlap a part of the long-time exposure. Solid-state image pickup device in which a charge transfer path is formed along each of a plurality of pixel columns composed of the first pixel and the second pixel. The first pixel and the second pixel are simultaneously exposed, and when the short-time exposure ends during the exposure of the first pixel, the accumulated charge of the second pixel is read out to the adjacent charge transfer path. When a charge packet is transferred and output, an empty packet is formed and transferred in an empty space other than providing the signal charge packet in the charge transfer path, and the charge is transferred according to the amount of charge transferred in the empty packet. Data is stored in the signal charge Data received obtained by subtracting from the data corresponding to the amount of charge coming transferred in Tsu preparative, characterized in that the captured image data by the short exposure.

  In the image pickup apparatus and the drive control method thereof according to the above-described embodiment, the first pixels provided in the solid-state image pickup device are arranged in a square lattice, and three primary color filters are Bayer arranged in the first pixels. The second pixels arranged in the vertical direction and the horizontal direction with a ½ pixel pitch shift between the first pixels are also arranged in a square lattice, and three primary color filters are Bayer arranged in the second pixels, The empty packet is provided in the charge transfer path adjacent to the first pixel when the stored charge of the second pixel is read and transferred by the signal charge packet. Furthermore, in the image pickup apparatus and the drive control method thereof according to the above-described embodiment, the subtraction is performed using data stored in a line memory that temporarily stores data for one horizontal line output from the solid-state image sensor. It is characterized by performing.

  Furthermore, the imaging apparatus and the drive control method thereof according to the above-described embodiment are characterized in that, in the above, the exposure of the first pixel is completed with a mechanical shutter.

  Thus, in each of the above-described embodiments, an empty space formed at the time of reading signal charges (a vacant vertical charge transfer path in the embodiment of FIG. 2) is not separately prepared for smear charge detection. In the embodiment shown in FIG. 6, it is only necessary to insert and transfer an empty packet by effectively using (between signal charge packets), that is, without adding extra control, highly accurate smear correction is performed to achieve high quality and wide range. A subject image having a dynamic range can be obtained.

  The image pickup apparatus and the drive control method thereof according to the present invention can be applied to a digital camera, a mobile phone with a camera, a video camera capable of taking a still image, and the like because it can take a high-quality and wide dynamic range subject image without the influence of smear. Then it is useful.

It is a block block diagram of the imaging device of one Embodiment of this invention. It is a surface schematic diagram of the solid-state image sensor shown in FIG. 3 is a flowchart showing an imaging processing procedure of the imaging apparatus shown in FIG. 1. 3 is a timing chart of an imaging operation of the imaging apparatus shown in FIG. It is a figure explaining the relationship between a smear charge and a signal charge in the imaging device shown in FIG. It is a surface schematic diagram of the solid-state image sensor of embodiment different from FIG. It is operation | movement explanatory drawing in the solid-state image sensor shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Imaging device 11 Solid-state image sensor 12 Mechanical shutter 22 CPU
23 Signal processing circuit 26 Frame memory 27 Line memory 45 Timing generator (image sensor driving means)
51,61 pixels (photodiode)
52,62 Vertical charge transfer (VCCD)
53, 63 Horizontal charge transfer path (HCCD)
54, 64 Amplifier 65, 66 Read gate

Claims (8)

  A plurality of first pixels that perform long-time exposure imaging and a plurality of second pixels that perform short-time exposure imaging overlapping a part of the long-time exposure are mixed and arranged in a two-dimensional array. A solid-state imaging device in which a charge transfer path is formed along each of a plurality of pixel columns composed of a first pixel and a second pixel, and exposure of the first pixel and the second pixel are started simultaneously. When the short-time exposure is completed during the exposure, the charge stored in the second pixel is transferred to the adjacent charge transfer path as a read signal charge packet and output, and the signal charge packet in the charge transfer path is output. A solid-state image sensor driving means for forming and transferring an empty packet in an empty space other than providing the data, and an electric charge transferred by the signal charge packet for data corresponding to the amount of electric charge transferred by the empty packet Data according to quantity An imaging device and a signal processing means for the captured image data al subtraction obtained was data by the short exposure.   2. The imaging device according to claim 1, wherein the first pixels provided in the solid-state imaging device are arranged in a square lattice, and color filters of three primary colors are Bayer arranged in the first pixels. The second pixels arranged with a ½ pixel pitch shifted in the vertical and horizontal directions are also arranged in a square lattice, and color filters of three primary colors are Bayer arranged in the second pixels, and the empty packet is the second packet. An image pickup apparatus provided in the charge transfer path adjacent to the first pixel when the charge stored in the pixel is read and transferred by the signal charge packet.   The imaging apparatus according to claim 1 or 2, further comprising a line memory that temporarily stores data for one horizontal line output from the solid-state imaging device, and using the stored data of the line memory. An imaging device that performs the subtraction.   The imaging apparatus according to claim 1, wherein the imaging apparatus includes a mechanical shutter that terminates exposure of the first pixel.   A plurality of first pixels that perform long-time exposure imaging and a plurality of second pixels that perform short-time exposure imaging overlapping a part of the long-time exposure are mixed and arranged in a two-dimensional array. Exposure of the first pixel and the second pixel of the solid-state imaging device in which a charge transfer path is formed along each of a plurality of pixel columns composed of the first pixel and the second pixel is started at the same time. When the short-time exposure is completed during the exposure, the charge stored in the second pixel is transferred to the adjacent charge transfer path as a read signal charge packet and output, and the signal charge packet in the charge transfer path is output. An empty packet is formed and transferred in an empty space other than providing the data, and data corresponding to the amount of charge transferred in the empty packet is obtained from data corresponding to the amount of charge transferred in the signal charge packet. Data obtained by subtraction Drive control method for an imaging apparatus that captured image data by the short exposure.   6. The drive control method for an image pickup apparatus according to claim 5, wherein the first pixels provided in the solid-state image sensor are arranged in a square lattice, and color filters of three primary colors are Bayer arranged in the first pixels. The second pixels arranged with a ½ pixel pitch shift between the first pixels in the vertical direction and the horizontal direction are also arranged in a square lattice, and color filters of three primary colors are Bayer arranged in the second pixels. Is a drive control method for an imaging device provided in the charge transfer path adjacent to the first pixel when the stored charge of the second pixel is read and transferred by the signal charge packet.   7. The drive control method for an imaging apparatus according to claim 5, wherein the subtraction is performed using data stored in a line memory that temporarily stores data for one horizontal line output from the solid-state imaging device. The drive control method of the imaging device which performs.   8. The drive control method for an imaging apparatus according to claim 5, wherein the exposure of the first pixel is terminated by a mechanical shutter. 9.

Images