JP5629568B2 - Imaging device and pixel addition method thereof - Google Patents

Description

  The present invention relates to an imaging apparatus and a pixel addition method thereof.

  In recent years, a solid-state imaging device using a MOS transistor circuit as a signal readout circuit, such as a CMOS type image sensor, has been increased in number of pixels, and it is common to mount 10 million pixels or more. For this reason, each pixel (pixel) is miniaturized and the saturation charge amount of each pixel (photoelectric conversion element) is reduced, so that the dynamic range of the captured image is narrowed and it is difficult to capture a bright image. It is becoming.

  For this reason, when performing high-sensitivity imaging, wide dynamic range imaging, or the like, pixel addition for adding detection signals of a plurality of pixels is performed. In this pixel addition, for example, as described in Patent Document 1 below, a first method of adding pixels by signal processing after reading a captured image signal from an image sensor, and Patent Document 2 described below. As shown, the signal charge detected by each pixel in accordance with the amount of received light is directly added inside the image sensor (mixing of signal charges), and a signal corresponding to the added signal charge amount is read out to the outside of the image sensor. There is a method.

JP 2007-124137 A JP 2002-199284 A

  Of the above two methods for performing pixel addition, the first method of adding a signal processing after reading a picked-up image signal from the image sensor increases both the light shot noise and dark noise due to the addition and adds to the addition processing. While there is a disadvantage that time is required, there is an advantage that the result of pixel addition is not saturated.

  On the other hand, the second method for adding signal charges inside the image sensor is limited by the saturation charge amount of the FD transistor because the place where the addition is performed is a floating diffusion (hereinafter referred to as FD) transistor. In addition, there is a disadvantage that the light shot noise increases due to addition, but there is an advantage that the added signal can be read out at high speed because the dark noise does not increase and the addition processing does not take time. .

  In this way, the two addition methods have their merits and demerits, and if these are used separately to compensate for each other's disadvantages, it is expected to improve the usability of the imaging device, It is expected to obtain a high-quality subject image according to the response. Conventionally, however, little consideration has been given to what kind of image sensor the two addition methods are applied to and how to use them properly.

  An object of the present invention is to provide an image pickup apparatus and a pixel addition method for selectively using pixel addition inside the image pickup device and pixel addition of a signal after reading from the image pickup device.

According to the imaging apparatus and the pixel addition method of the present invention, a plurality of photoelectric conversion elements are formed in a two-dimensional array on a semiconductor substrate, and two adjacent photoelectric conversion elements are used as a pair pixel. An image sensor in which a color filter is Bayer arrayed as a unit and a signal readout circuit composed of a MOS transistor circuit is formed on the semiconductor substrate for each paired pixel, and an output from the signal readout circuit of the image sensor attached to the image sensor An image pickup apparatus including a signal processing circuit for processing a picked-up image signal and a pixel addition method thereof,
A first pixel addition drive mode in which captured image signals corresponding to the detected charges of the two photoelectric conversion elements constituting the pair pixel are individually read out by the signal readout circuit and added by the signal processing circuit; and the pair pixel The detection charges of the two photoelectric conversion elements are mixed, and a second pixel addition drive mode in which a captured image signal corresponding to the mixed charge amount is read by the signal readout circuit, and a subject image is captured by the imaging element Switching according to the shooting state, and dividing the light receiving surface of the imaging element on which the photoelectric conversion element is formed into a plurality of blocks, and the first pixel addition driving mode and the second pixel addition for each divided block. Switch between drive modes,
When the signal readout circuit reads the captured image signal from the paired pixels in the same row in a state where the first pixel addition drive mode and the second pixel addition drive mode coexist, the second pixel addition drive mode Among the output timings of the image sensor, a dummy signal is output at the same position as the output timing of the captured image signal read out first among the pair of pixels output in the first pixel addition drive mode, and the dummy signal and the The captured image signal corresponding to the mixed charge amount is signal-added by the signal processing circuit, and the signal processing by the signal processing circuit is added to the signal output from the image sensor in the first pixel addition drive mode. It is characterized by having the same processing as .

  According to the present invention, since the two pixel addition methods can be appropriately selected and switched depending on the shooting state, it is possible to take a high-quality, wide dynamic range image of the subject.

It is a functional block block diagram of the imaging device which concerns on one Embodiment of this invention. It is a surface schematic diagram of the image sensor shown in FIG. It is a figure which shows the color filter arrangement | sequence of the image pick-up element shown in FIG. FIG. 4 is a diagram in which two main pixels and two sub-pixels are extracted from the image sensor of FIG. 3. FIG. 5 is a circuit diagram of a signal readout circuit having a total of four pixels shown in FIG. 4. FIG. 6 is an internal circuit diagram of the signal readout circuit shown in FIG. 5. It is a sequence diagram of pixel addition by charge mixing. It is a sequence diagram of pixel addition by signal addition. It is a flowchart which shows the pixel addition method selection process sequence by 1st Embodiment of this invention. It is explanatory drawing of the threshold value (alpha) used in FIG. It is a flowchart which shows the pixel addition method selection process sequence by 2nd Embodiment of this invention. It is explanatory drawing of the threshold value (beta) used in FIG. It is explanatory drawing of embodiment which divides an image pick-up element light-receiving surface into several blocks. It is a figure which shows the signal read-out sequence of embodiment described in FIG. It is explanatory drawing of the mask signal generation circuit used by embodiment described in FIG. It is a flowchart which shows the process sequence which uses an image pick-up element light-receiving surface separately for several blocks and uses a pixel addition method for every block.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a functional block diagram of a digital camera (imaging device) according to an embodiment of the present invention. The digital camera 10 of the present embodiment has a function of capturing a still image or a moving image of a subject and digitally processing a captured image signal in the camera 10, and includes a photographic lens 20 including a telephoto lens and a focus lens, and a photographic lens 20. The analog image data output from each pixel of the solid-state image sensor 21 and the solid-state image sensor 21 placed on the imaging surface of the solid-state image sensor 21 and the analog image data such as automatic gain adjustment (AGC) and correlated double sampling processing. From an analog signal processing unit 22 to be processed, an analog / digital conversion unit (A / D) 23 that converts analog image data output from the analog signal processing unit 22 into digital image data, and a system control unit (CPU) 29 described later The A / D 23, the analog signal processing unit 22, the solid-state imaging device 21, and the photographic lens 20 are driven and controlled by It includes a moving section 24, a flash 25 that emits light in response to an instruction from the CPU 29.

  In response to an instruction from the CPU 29 that executes a pixel addition method selection processing procedure to be described later, the drive unit 24 performs signal readout drive from the image sensor 21 based on the corresponding pixel addition method.

  The digital camera 10 of the present embodiment further includes a digital signal processing unit 26 that takes in digital image data output from the A / D 23 and performs pixel addition processing, interpolation processing, white balance correction, RGB / YC conversion processing, and the like. A compression / decompression processing unit 27 that compresses image data in JPEG format or the like, a display unit 28 that displays menus and displays through images and captured images, and overall control of the entire digital camera A system control unit (CPU) 29, an internal memory 30 such as a frame memory, and a media interface (I / F) unit 31 that performs interface processing between a recording medium 32 that stores JPEG image data and the like. In addition, the system control unit 29 receives instructions from the user. Operation unit 33 are connected.

  In the present embodiment, the solid-state image sensor 21 is a MOS image sensor, for example, a CMOS image sensor, in which a MOS transistor circuit formed on a semiconductor substrate is a signal readout circuit. The output signal of the solid-state image sensor 21 is an analog signal processing unit. (AFE: analog front end) 22 is processed, but this AFE part (a circuit that performs correlated double sampling processing or clamping processing, a signal amplification circuit that performs gain control, etc.) is provided as a peripheral circuit on the semiconductor chip. It is normal. In addition, on the semiconductor chip of the solid-state imaging device 21, in addition, a horizontal scanning circuit, a vertical scanning circuit, and the like are formed as peripheral circuits around the light receiving unit, and the A / D conversion unit 23 of FIG. 1 is also formed. There is.

  FIG. 2 is a schematic diagram of the surface of the solid-state imaging device 21 shown in FIG. In the solid-state imaging device 21, a large number of pixels (photoelectric conversion elements: photodiodes) 43 are formed on a light receiving surface 42 of a semiconductor substrate 41 in a two-dimensional matrix. The solid-state imaging device 21 of the present embodiment has a so-called honeycomb pixel array in which the even-numbered pixels 43 are shifted from the odd-numbered pixels 43 by 1/2 pixel pitch. A horizontal scanning circuit 44 is provided on the lower side of the semiconductor substrate 41, and a vertical scanning circuit 45 is provided on the left side of the semiconductor substrate 41.

  Each pixel 43 is provided with a signal readout circuit (MOS transistor circuit) that reads a signal corresponding to the amount of signal charge accumulated by the pixel 43 according to the amount of received light. A color filter is stacked on each pixel 43. In FIG. 2, the red filter is indicated by “R” and “r”, the green filter is indicated by “G” and “g”, and the blue filter is indicated by “B” and “b”. ing. The difference between uppercase RGB and lowercase rgb will be described with reference to FIG.

  In the present embodiment, since the pixels 43 are arranged in a so-called honeycomb arrangement, the horizontal wiring 46 (only one is shown in FIG. 2) connected to the vertical scanning circuit 45 connects each pixel 43 on the light receiving surface 42. The vertical wiring 47 (only one is shown in FIG. 2) connected to the horizontal scanning circuit 44 is also provided meandering in the vertical direction.

  FIG. 3 is a diagram illustrating the color of the color filter stacked on each pixel in the honeycomb image array of the solid-state image sensor 21. On each pixel, three primary color filters, R = r = red, G = g = green, and B = b = blue are stacked.

  When only the odd-numbered pixel rows are viewed, the pixels are arranged in a square lattice, and the color filters RGB are arranged in a Bayer arrangement thereon. Even if only the even number of pixel rows are viewed, each pixel is arranged in a square lattice, and a color filter rgb is arranged thereon in a Bayer arrangement. As a result, filters (Rr) (Gg) (Bb) of the same color are arranged in two pixels adjacent obliquely.

  Pixels equipped with uppercase RGB color filters constitute the first group pixels, and pixels equipped with lowercase rgb color filters constitute the second group pixels. R and r, G and g, and B and b are exactly the same color filters, but in order to distinguish between a subject image captured with only the first group pixel and a subject image captured with only the second group pixel, capital letters are used. And lowercase letters.

  In this embodiment, two pixels of the same color filters (Rr) (Gg) (Bb) that are diagonally adjacent to each other constitute a pair pixel, and each detection signal of the pair pixel is read out from the solid-state imaging device 21 and then the pixel It is configured to add or read after adding pixels inside the solid-state imaging device 21.

  Hereinafter, one pixel of each pair pixel (first diagonally upper right group pixel) is referred to as a “main pixel”, and the other pixel (lower second diagonal group pixel) is referred to as a “subpixel”. Capture the subject image with the exposure time of the main pixel and sub-pixel as the same time, and shoot by doubling the sensitivity by adding each detection signal of the main pixel and sub-pixel, or the exposure time of the main pixel On the other hand, wide dynamic range imaging can be performed by adding each detection signal to a pixel with the subpixel exposure time being short. Alternatively, if it is determined that the detection signal of the main pixel has a sufficient signal amount, a subject image is generated using only the detection signal of the main pixel, and the detection signals of the main pixel and sub-pixel are separately set. It is also possible to generate a high-definition subject image without reading and pixel addition.

  FIG. 4 is a diagram in which two main pixels 43a adjacent in the horizontal direction and the sub-pixels 43b each paired with these are extracted from the large number of pixels shown in FIG. In the imaging device 21 of the present embodiment, the main pixel 43a and the sub-pixel 43b that form a pair are configured to share one signal readout circuit 50. Reference numeral 51c will be described with reference to FIG.

  FIG. 5 is a diagram showing a signal readout circuit of the four pixels shown in FIG. The main pixel 43a and the sub-pixel 43b paired with the main pixel 43a are connected to a common (shared) signal readout circuit 50 via readout gates (TG: abbreviation of transfer gate) 51a and 51b, respectively. The output is connected to the output line 52. The signal readout circuit 50 is provided with a reset terminal, a power supply terminal, and a row selection terminal.

  The readout gate 51a is connected to a readout pulse application line (denoted as main pixel TG line) 54 of the main pixel, and the readout gate 51b is connected to a readout pulse application line (denoted as subpixel TG line) 55 of the subpixel. Although omitted, the reset terminal is connected to the reset line, the power supply terminal is connected to the power supply line, and the row selection terminal is connected to the row selection line.

  A total of four signal wirings, that is, a readout pulse application line 54 for the main pixel, a readout pulse application line 55 for the sub-pixel, and a reset line and a row selection line are connected to the pixel row 2 from the vertical scanning circuit 45 as the horizontal wiring 46 in FIG. Laid for each row. The power supply line and the output line 52 are laid for each pixel column as the vertical wiring 47 in FIG. 2, and the signal readout circuit 50 of the pixel column in which the power supply voltage Vdd is applied from the horizontal scanning circuit 44 to the corresponding power supply line is enabled. A captured image signal is output to the output line 52.

  FIG. 6 is a configuration diagram of the signal readout circuit employed in the present embodiment. The signal readout circuit 50 includes an output transistor (FD transistor) 51, a row selection transistor 54 connected in series to the output transistor 51, and a reset transistor 53.

  The drain of the FD transistor 51 is connected to the power supply terminal 50 a to which the power supply voltage Vdd is supplied from the horizontal scanning circuit 44, and the reset transistor 53 is connected between the gate and drain of the FD transistor 51. The gate terminal 53a of the reset transistor 53 is connected to the reset line, and the drain is connected to the power supply terminal 50a.

  The source of the FD transistor 51 is connected to the drain of the row selection transistor 54, the source terminal 54a of the row selection transistor 54 is connected to the output line 52, and the gate terminal 54b of the row selection transistor 54 is connected to the row selection line.

  The n region 43a of the main pixel and the n region 43b of the sub pixel are provided close to each other, and an n region 51c is provided between the n region 43a and the n region 43b. This n region 51c is connected to the gate of the output (FD) transistor 51, and one FD transistor 51 is provided in common for the main pixel 43a and the sub-pixel 43b.

  In other words, the readout gate 51a is provided between the main pixel (n region) 43a and the n region 51c, and the readout gate 51b is provided between the sub pixel (n region) 43b and the n region 51c. Each detected charge is read out to a common n region 51c.

  FIG. 7 shows a sequence in which the signal charges detected by the main pixel 43a and the sub-pixel 43b are added to each other within the image pickup device 21 (charge mixing) and read using the signal readout circuit described in FIGS. FIG. When the readout pulse 61 is applied to the readout pulse application line 54, the accumulated charge in the n region 43a of the main pixel is transferred to the n region 51c in FIG. 6, and the signal charge of the main pixel 43a becomes empty. Next, when the readout pulse 62 is applied to the readout pulse application line 55, the accumulated charge in the n region 43b of the subpixel is transferred to the n region 51c in FIG. 6, and the signal charge of the subpixel 43b becomes empty.

  As a result, the detection charge of the main pixel and the detection charge of the subpixel are accumulated in the n region 51c in a mixed state, and the charge (main pixel charge + subpixel charge) added and mixed in the n region 51c is output. This is applied to the gate of the transistor 51.

  Assume that the power supply voltage Vdd is supplied from the horizontal scanning circuit 44 to the power supply terminal 50a. When the row selection signal 63 is applied from the vertical scanning circuit 45 to the gate terminal 54b of the row selection transistor 54 and the transistor 54 is turned on (conductive), the output transistor 51 is also turned on (conductive), and the above-described output line 52 is connected to the output line 52. A signal corresponding to the added signal charge amount is output as a captured image signal. The captured image signal 64 is taken into the analog signal processing unit 22 through the output line 52, and the subsequent image processing is performed.

  Thereafter, when a reset signal is applied to the reset transistor 53, the signal charge (addition charge) applied to the gate of the output transistor 51 is discarded to the power supply line Vdd through the reset transistor 53, and the n region 51a becomes empty. .

  8 uses the signal readout circuit described in FIG. 4, FIG. 5, and FIG. 6, and uses the imaging device to generate a captured image signal corresponding to the signal charge amount of the main pixel and a captured image signal corresponding to the signal charge amount of the sub-pixel. 21 is a sequence diagram for individually reading out to the outside. In this case, the two captured image signals are digitally added by the digital signal processing unit 26 of FIG.

  In this case, first, the read pulse 61 is applied to the read pulse application line 54. As a result, the accumulated charge in the n region 43a of the main pixel is transferred to the n region 51a in FIG. 6, and the signal charge in the main pixel 43a becomes empty. A power supply voltage Vdd is applied to the power supply line, and the signal readout circuit 50 is in an enabled state.

  Since the signal charge amount detected by the main pixel 43a is applied to the gate of the output transistor 51, when the row selection signal 65 is applied to the row selection transistor 54 at the next timing, the accumulated charge amount of the main pixel 43a is increased. A corresponding captured image signal 66 is output to the output line 52.

  When the reset signal 67 is applied to the reset transistor 53 at the next timing, the signal charge applied to the gate of the output transistor 51 is discarded to the power supply line, and the n region 51c in FIG. 6 becomes empty.

  When the readout pulse 62 is applied to the readout pulse application line 55 at the next timing, the accumulated charge in the n region 43b of the subpixel is read out to the n region 51c of FIG. 6, and the signal charge of the subpixel 43b becomes empty. . Since the signal charge amount detected by the sub-pixel 43b is applied to the gate of the output transistor 51, when a row selection signal 68 is applied to the row selection transistor 54, a captured image corresponding to the accumulated charge amount of the sub-pixel 43b. A signal 69 is output to the output line 52.

  Next, when the reset signal 70 is applied to the reset transistor 53, the signal charge applied to the gate of the output transistor 51 is discarded to the power supply line, and the n region 51c becomes empty. The digital signal processing unit 26 at the subsequent stage adds the captured image signal 66 of the main pixel and the captured image signal 69 of the sub-pixel to obtain a highly sensitive captured image signal or a captured image signal with a wide dynamic range. The signal addition of the captured image signal 66 and the captured image signal 69 may be performed in the state of an analog signal (voltage value signal) before digital conversion.

  FIG. 9 is a flowchart showing a pixel addition method selection processing procedure according to an embodiment of the present invention. This pixel addition method selection process is executed after the system control unit 29 in FIG. 1 captures a subject image using an image sensor and each pixel accumulates signal charges corresponding to the amount of received light.

  After the photographing process by the imaging device 10 is completed and signal charges corresponding to the shutter speed, exposure, and the like are accumulated in the pixels 43a and 43b of the image sensor 21, first, in step S1, the accumulated charges of the main pixel 43a are stored. Read to n region 51c.

  In the next step S2, it is determined whether or not the magnitude of the signal charge amount of the main pixel is equal to or greater than a threshold value α (50% of the saturation amount of FD). The signal charge amount read to the n region 51c can be determined by outputting a captured image signal corresponding to the signal charge amount to the output line 52.

  FIG. 10 is an explanatory diagram of the threshold value α. Since the light receiving areas of the main pixel 43a and the sub pixel 43b are the same area and the exposure time is also the same time, the light receiving amounts of the main pixel and the sub pixel and the signal level have the same proportional relationship. If the signal level (detected charge) of the main pixel 43a is less than or equal to the saturation charge amount (FD saturation amount) of the n region 51c, it can be estimated that it is the same level as the detected charge of the main pixel. It can be determined that the probability that the n region 51c overflows is small even if the detected charges of the n region 51c are read out.

  Therefore, if the signal charge amount read from the main pixel 43a is not equal to or greater than the threshold value α, the process proceeds from step S2 to step S3, and the accumulated charge of the sub-pixel 43b is read to the n region 51c. As a result, the total signal charge amount of the main pixel and the sub-pixel is read out to the n region 51c. In the next step S4, a picked-up image signal corresponding to the added and mixed signal charge amount is output to the output line 52, and this process is terminated.

  If the detection charge amount of the main pixel 43a is equal to or greater than the threshold value α as a result of the determination in step S2, the n region 51c overflows if the detection charge of the sub-pixel 43b is read together into the n region 51c. Therefore, in this case, the process proceeds from step S2 to step S5, and the picked-up image signal corresponding to the signal charge amount of the n region 51c (only the signal charge of the main pixel 43a) is output to the output line 52. A captured image signal of only pixels is stored in the memory 30 (FIG. 1).

  In the next step S7, a reset signal (signal 67 in FIG. 8) is applied, and the n region 51c is emptied (step S8). In the next step S9, the detected charge of the sub-pixel 43b is read out to the n region 51c. In step S10, a captured image signal corresponding to the signal charge amount in the n region 51c is output to the output line 52, and the captured image signal of the sub-pixel and the main pixel waiting in the memory 30 in step S6. A signal addition (voltage addition) is performed on the captured image signal (step S11), and this process ends. The pixel addition signal obtained by signal addition may be used as it is, or an average value may be obtained as a captured image signal.

  Note that the comparison with the threshold value α in step S2 is performed by using the exposure data obtained by the AE process performed before the imaging process on the light receiving surface of the image sensor 21, and using the exposure data obtained by the AE process performed before the imaging process. You may judge by the signal charge amount read from the pixel.

  FIG. 11 is a flowchart showing a pixel addition method selection processing procedure according to another embodiment of the present invention. Basically, it is the same as the pixel addition method selection processing procedure of FIG. 9, but the present embodiment is different only in that processing steps are added when the signal charge amount of the main pixel is large.

  In this embodiment, the determination process of step S15 is performed after step S1. In this determination process S15, the signal charge amount read from the main pixel of the partial area determined to have the largest incident light amount to the n area 51c in the imaging element light receiving surface is larger than a predetermined threshold value β that is larger than the threshold value α. It is determined whether or not.

  When the signal charge amount read from the main pixel is larger than the threshold value β, the process proceeds from step S15 to step S16, and a captured image signal corresponding to the signal charge amount of the main pixel is output to the output line 52, and this process is terminated. That is, in this case, “pixel addition” is not performed, and the subject image is generated only with the detection signal of the main pixel.

  As a result of the determination in step S15, when the detected charge of the main pixel is smaller than the threshold value β, the process proceeds to step S2, and the process proceeds with the same procedure as the process step procedure of FIG.

  In this manner, when the signal charge amount (detection charge) of the main pixel is larger than the threshold value β, as shown in FIG. 12, a sufficient signal amount can be obtained only by the signal charge amount of the main pixel, and only the signal charge of the main pixel is obtained. Since it can be determined that the probability of obtaining a captured image signal with a wide dynamic range is high, pixel addition can be skipped. The threshold value β may be 80% or 70% of the saturation amount of the FD.

  In the processing procedure of FIG. 9 and the processing procedure of FIG. 11, the pixel addition method is a method based on charge mixing of signal charges (step S4: second pixel addition drive mode) or a method based on signal addition (step S11: first step). Whether or not to select the one-pixel addition drive mode) is determined based on the signal charge amount of the main pixel (steps S2 and S15). However, depending on the shooting conditions, it may not be necessary to determine the signal charge amount of the main pixel one by one. is there.

  For example, when the user selects the “continuous shooting mode” from the operation unit 33 of the imaging apparatus 10 before the shooting process, the signal charge amount of the main pixel exceeds the threshold value α because the shutter speed of each shooting is short. In some cases, it can be judged as “small”. In this case, for example, after step 1, it is determined whether or not shooting has been performed in the “continuous shooting mode”, and in the continuous shooting mode, the process may be forcibly advanced to step S3. Thereby, pixel addition due to time-consuming signal processing is avoided.

  Similarly, when the user specifies the ISO sensitivity before the shooting process, or when the imaging device 10 automatically sets the ISO sensitivity by auto setting, the ISO sensitivity becomes equal to or higher than a predetermined sensitivity (for example, ISO 400 or higher). In this case, since the shooting scene is dark, it can be determined that the signal charge amount accumulated in the main pixel is “smaller” than the threshold value α, as described above. In this case as well, the ISO sensitivity is determined after step S1, and when the sensitivity is higher than the predetermined ISO sensitivity, the process may be forced to proceed to step S3. In the case of low ISO sensitivity (less than ISO 400), the shooting scene is bright, and forcibly proceeding to step S5 makes it possible to capture a subject image with a wide dynamic range.

  In this way, when it can be determined that the signal charge amount of the main pixel does not reach the threshold value α according to the shooting conditions, it is possible to forcibly select the pixel addition based on the charge mixture of the signal charges, thereby eliminating unnecessary processing and processing time. Can be shortened.

  When shooting outdoors, a clear sky, a portion illuminated by the sun (bright portion), and a shadow portion exist, and a wide dynamic range is required. For example, in the outdoor shooting scene shown in FIG. 13A, a bright part and a dark part are mixed.

  In such a case, the image sensor light-receiving surface is divided into a plurality of blocks, and for each divided block, whether the pixel addition method is pixel addition by signal addition or pixel addition by signal charge mixture It is good to do it separately.

  For example, as shown in FIG. 13B, the light receiving surface is divided into a total of 25 divided blocks of 5 rows and 5 columns, and the blocks of rows A and B are subjected to pixel addition by signal addition, and rows D and F Row blocks perform pixel addition by charge mixing. In the C row, signal addition is performed for columns A, B, D, and E, and pixel addition by charge mixing is performed for column C.

  In the case of an image pickup device using a MOS transistor circuit as a signal readout circuit, signal readout to each pair pixel can be performed by random access, and the pixel addition method can be changed for each pair pixel and block. .

  The signal readout from each block of the A row, B row, D row, and F row is the same as the signal addition method for each pixel in the same row. Therefore, the description thereof is omitted, and the signal readout method from each block of the C row. (Pixel addition method) will be described.

  When reading the signal of the C row block, first, signal reading is performed from the captured image signal of the block of column C. In this case, the supply of the power supply voltage Vdd to the signal readout circuits in the first row, second row, second row, and third row is stopped, and each signal read circuit 50 is disabled. Then, the power supply voltage Vdd is applied only to the power supply line of the signal readout circuit in the row C to enable it. Of course, the power supply voltage Vdd is applied by the horizontal scanning circuit 44 for each pair of pixel columns.

  Then, when a readout pulse 61 (see FIG. 8) is applied to the main pixel TG line 54 of FIG. 5, all of the main pixels 43a in the same row of C rows (including columns A, B, D, and E) are displayed. The signal charge is read out to the n region 51c. Next, when the row selection signal 65 is applied, a captured image signal corresponding to the detection signal of the main pixel in column C is output to the output line 52. However, since the signal readout circuits 50 in the first row, second row, second row, and first row are inactive (disabled), no captured image signal is output.

  Next, when the reset signal 67 is applied, the signal charges in the n region 51c in the first column are discarded to the power supply line, but the signal charges in the n region 51c in the first column, the second column, the second column, and the third column are Remains in the n region 51c as it is inactive.

  Next, when the readout pulse 62 is applied to the sub-pixel TG line 55, the signal charges of all the sub-pixels 43b in the same row of C rows (including columns A, B, D, and E) are applied to the n region 51c. Read out. In the n region 51c of the A-row, B-row, D-row, and H-row, the signal charges of the main pixel + sub-pixel are mixed. However, only the signal charge of the sub-pixel 43b is accumulated in the n region 51c in the row C.

  Next, when the row selection signal 68 is applied, only the picked-up image signals 69 in the first column are output to the output line 52, and the picked-up image signals in the first column, the second column, the second column, and the third column are output from the respective signal readout circuits 50. Is not output due to inactivity. Then, by applying 70 as the reset signal, the n region 51c in the row C becomes empty.

  In this way, after the captured image signal 66 from the main pixel in the column C and the captured image signal 69 from the sub-pixel in the column C are read out separately, pixel addition is performed by signal addition by the signal processing circuit. Is called.

  On the other hand, in the n region 51c of the A-row, B-row, D-row, and H-row, the main pixel signal charge and the sub-pixel signal charge are maintained in a mixed state. For this reason, the supply of the power supply voltage Vdd to the signal readout circuit in the column C is stopped, the horizontal scanning of the power supply voltage Vdd to the signal readout circuit 50 in the columns A, B, D, and H and the vertical selection of the row selection signal. By performing scanning, a pixel addition signal in which charges are mixed is output to the output line 52.

  When performing the above-described signal readout (pixel addition), as shown in FIG. 14A, in the row C, a signal (main pixel + subpixel) 71 in which charge is mixed is output, and FIG. As shown in FIG. 6, the main pixel signal 72 and the sub-pixel signal 73 are mixed and output separately.

  The signal processing circuit 26 in the subsequent stage is often designed so that separate signal processing cannot be performed for each block, and both the signal 71 and the signals 72 and 73 exist as output signals of the same pair pixel row. If time is mixed, there is a risk of signal processing being mistaken. For example, in FIG. 14A in which charge-mixing pixel addition is performed, only the signal charge 75 of the “main pixel” is not output, but if noise or the like is superimposed at this position, this noise is expressed as “captured image signal”. May be mistaken.

  Therefore, in the present embodiment, as shown in FIG. 15, a mask signal generation circuit 76 and an addition circuit 77 are provided in the preceding stage of the analog signal processing unit 22, and the mask signal generation circuit 76 outputs an on (high) signal. The output signal of the image sensor 21 is passed through the analog signal processing unit 22 only when the signal is output, and the dummy signal 78 is output at the signal position corresponding to the read position of the main pixel charge 75 in FIG. . Thereby, there is no possibility that the digital signal processing unit 26 makes a mistake in signal processing.

  FIG. 16 is a flowchart illustrating a processing procedure of the embodiment described with reference to FIGS. First, in step S21, the AE data acquired by the AE (automatic exposure) process before the photographing process is fetched for each divided block in FIG. In the next step S22, it is determined whether or not the exposure value of each divided block is equal to or greater than a predetermined threshold value.

  If the result of determination in step S22 is that the exposure value is less than the predetermined threshold, the block is a dark scene and the process proceeds to step S23, where the pixel addition processing is performed by signal charge mixture. If the result of determination in step S22 is that the exposure value is a block greater than or equal to a predetermined threshold, the process proceeds to step S24, where the pixel addition process is performed by signal addition.

  In step S25 following step S24, in order to perform signal addition, first, a captured image signal corresponding to the signal charge amount of the main pixel is output to the output line 52 and stored in the memory 30 (step S26). A captured image signal corresponding to the signal charge amount of the pixel is output to the output line 52. In step S31, the captured image signal of the main pixel and the captured image signal of the sub-pixel are added, and this process ends.

  In the case of a block that mixes signal charges, the dummy signal 78 shown in FIG. 14A is output in step S28 following step S23, and the dummy signal 78 is stored in the memory 30 (step S29). Then, in step S30, the captured image signal in which the charge of “main pixel + subpixel” is mixed is output to the output line 52, and in step S31, it is added to the dummy signal (output 0) 78 of the memory 30, and this process is terminated. .

  The reason why Step S28 is provided for Step S25 and Step S29 is provided for Step S26 is that the processing procedure performed by the signal processing circuit 26 is the same signal processing in block rows with different pixel addition methods.

  According to the present embodiment, the pixel addition method is changed for each divided block. However, only the pixel addition method is different, and the brightness of integration is not different between blocks, so that a step occurs at the boundary of the block. There is no. According to the present embodiment, the dynamic range of the high luminance part can be expanded, and the dark noise of the low luminance part can be reduced.

  In the above-described embodiment, as shown in FIG. 3, a so-called honeycomb pixel array in which even-numbered photoelectric conversion element rows are shifted by 1/2 pitch with respect to odd-numbered photoelectric conversion element rows is used. The embodiment is not limited to this honeycomb pixel arrangement, and the pixel arrangement is a square lattice arrangement, and color filters are Bayer-arranged in pixels (photoelectric conversion elements) in odd rows or odd columns, and are arranged in even rows or even columns. The present invention can be similarly applied to an image sensor in which color filters are arranged in a Bayer array.

  In the above-described embodiment, the photodiode formed on the semiconductor substrate is used as the photoelectric conversion element. However, the present invention is not limited to this, and a photoelectric conversion film made of, for example, an organic film is stacked on the semiconductor substrate. The present invention can also be applied to an image sensor in which this photoelectric conversion film is sandwiched between a common electrode film and a pixel electrode film for each pixel.

  Further, in the above-described embodiment, the light receiving areas of the main pixel and the sub-pixel are described as the same light receiving area, but the above-described image sensor is also described in which the light receiving area of the sub-pixel is smaller than the light receiving area of the main pixel. Each embodiment is applicable.

In the imaging apparatus and the pixel addition method thereof according to the embodiments described above, a plurality of photoelectric conversion elements are arranged in a two-dimensional array on a semiconductor substrate, and two adjacent photoelectric conversion elements are used as a pair pixel. An image sensor in which a color filter is Bayer-arrayed with one pixel as a unit, and a signal readout circuit composed of a MOS transistor circuit is formed on the semiconductor substrate for each paired pixel, and the signal readout of the image sensor attached to the image sensor An image pickup apparatus including a signal processing circuit that processes a picked-up image signal output from a circuit and a pixel addition method thereof,
A first pixel addition drive mode in which captured image signals corresponding to the detected charges of the two photoelectric conversion elements constituting the pair pixel are individually read out by the signal readout circuit and added by the signal processing circuit; and the pair pixel The detection charges of the two photoelectric conversion elements are mixed, and a second pixel addition drive mode in which a captured image signal corresponding to the mixed charge amount is read by the signal readout circuit, and a subject image is captured by the imaging element And an image sensor driving means for switching in accordance with a shooting state at the time of shooting.

  Further, the imaging device of the imaging device and the pixel addition method thereof according to the embodiment is characterized in that the odd-numbered photoelectric conversion element rows are formed by being shifted by 1/2 pitch with respect to the even-numbered photoelectric conversion element rows. To do.

  In the imaging apparatus and the pixel addition method thereof according to the embodiment, one photoelectric conversion element of the pair of pixels is in the photographing state in which switching between the first pixel addition drive mode and the second pixel addition drive mode is determined. Judgment is made based on whether or not the detected charge amount is equal to or greater than a first threshold value.

  Further, in the imaging apparatus and the pixel addition method thereof according to the embodiment, when the amount of charge detected by one of the photoelectric conversion elements of the paired pixels is equal to or larger than a second threshold value that is larger than the first threshold value, the one photoelectric conversion element The subject image is generated based on the detected charges.

  In addition, the imaging apparatus and the pixel addition method thereof according to the embodiment are characterized in that the second pixel addition drive mode is forcibly executed when the subject image is captured in the continuous shooting mode.

  In addition, the imaging apparatus and the pixel addition method thereof according to the embodiment forcibly execute the first pixel addition drive mode when the ISO sensitivity when shooting the subject image is lower than the predetermined ISO sensitivity, and is higher than the predetermined ISO sensitivity. The second pixel addition drive mode is forcibly executed.

  In the imaging apparatus and the pixel addition method thereof according to the embodiment, the light receiving surface of the imaging element on which the photoelectric conversion element is formed is divided into a plurality of blocks, and the imaging element driving unit is configured to The one-pixel addition drive mode and the second pixel addition drive mode are switched.

  In the imaging apparatus and the pixel addition method thereof according to the embodiment, the signal readout circuit captures the image from the paired pixel rows in the same row in a state where the first pixel addition drive mode and the second pixel addition drive mode are mixed. When reading out an image signal, among the output timings of the image sensor in the second pixel addition drive mode, the output timing of the imaged image signal read out first among the pair of pixels output in the first pixel addition drive mode A dummy signal is output at the same position as the signal, and the dummy signal and the captured image signal corresponding to the mixed charge amount are added by the signal processing circuit, and the signal processing by the signal processing circuit is added to the first pixel. The processing is the same as the signal addition processing of the signal output from the image sensor in the driving mode.

  According to the embodiment described above, in order to make the pixel addition method appropriate according to the shooting state and shooting conditions, it is possible to always capture a subject image of optimum quality regardless of the shooting state and shooting conditions. It becomes possible.

  In the pixel addition method selection processing according to the present invention, since an appropriate pixel addition method is selected according to the shooting conditions and the like, it becomes possible to capture a high-quality and wide dynamic range captured image, and a digital still camera or digital It is useful when applied to various digital cameras such as video cameras, camera-equipped mobile phones, electronic devices with cameras, and endoscopes.

DESCRIPTION OF SYMBOLS 10 Imaging device 21 Imaging device 22 Analog signal processing part 24 Drive part 26 Digital signal processing part 29 System control part (CPU)
30 Memory 41 Semiconductor substrate 43 Pixel (photoelectric conversion element)
43a main pixel 43b subpixel 44 horizontal scanning circuit 45 vertical scanning circuit 50 signal readout circuit 51 shared output transistor (FD transistor) of main pixel and subpixel
51a, 51b Read gate 51c FD transistor main pixel, subpixel shared n region 52 Output line 53 Reset transistor 54 Row selection transistor 54 Read pulse application line for main pixel (main pixel TG line)
55 Sub-pixel readout pulse application line (sub-pixel TG line)
61, 62 Read pulse 64, 66, 69, 71, 72, 73 Output signal 63, 65, 68 Row selection line 67, 70 Reset signal 76 Mask signal generation circuit 77 Addition circuit 78 Dummy output

Claims (6)

A plurality of photoelectric conversion elements are formed in a two-dimensional array on a semiconductor substrate, and two adjacent photoelectric conversion elements are used as a pair pixel, and a color filter is Bayer arrayed using the pair of pixels as a unit. An image sensor in which a signal readout circuit composed of a MOS transistor circuit is formed on the semiconductor substrate every time,
A signal processing circuit which is attached to the image sensor and processes a captured image signal output from the signal readout circuit of the image sensor;
A first pixel addition drive mode in which captured image signals corresponding to the detected charges of the two photoelectric conversion elements constituting the pair pixel are individually read out by the signal readout circuit and added by the signal processing circuit; and the pair pixel The detection charges of the two photoelectric conversion elements are mixed, and a second pixel addition drive mode in which a captured image signal corresponding to the mixed charge amount is read by the signal readout circuit, and a subject image is captured by the imaging element and an imaging element driving means for switching in accordance with the shooting state at the time of,
The light receiving surface of the imaging element on which the photoelectric conversion element is formed is divided into a plurality of blocks, and the imaging element driving means is configured to perform the first pixel addition driving mode and the second pixel addition driving mode for each divided block. Switch
When the signal readout circuit reads the captured image signal from the paired pixels in the same row in a state where the first pixel addition drive mode and the second pixel addition drive mode coexist, the second pixel addition drive mode Among the output timings of the image sensor, a dummy signal is output at the same position as the output timing of the captured image signal read out first among the pair of pixels output in the first pixel addition drive mode, and the dummy signal and the The captured image signal corresponding to the mixed charge amount is signal-added by the signal processing circuit, and the signal processing by the signal processing circuit is added to the signal output from the image sensor in the first pixel addition drive mode. Imaging device with the same processing .   2. The imaging apparatus according to claim 1, wherein odd-numbered photoelectric conversion element rows are formed so as to be shifted by ½ pitch with respect to even-numbered photoelectric conversion element rows.   3. The imaging device according to claim 1, wherein the photographing state in which switching between the first pixel addition drive mode and the second pixel addition drive mode is determined is the photoelectric of one of the paired pixels. An imaging apparatus that determines whether or not the amount of charge detected by a conversion element is equal to or greater than a first threshold value. A plurality of photoelectric conversion elements are formed in a two-dimensional array on a semiconductor substrate, and two adjacent photoelectric conversion elements are used as a pair pixel, and a color filter is Bayer arrayed using the pair of pixels as a unit. An image sensor in which a signal readout circuit composed of a MOS transistor circuit is formed on the semiconductor substrate every time,
A pixel addition method for an imaging device, comprising: a signal processing circuit that is attached to the imaging device and processes a captured image signal output from the signal readout circuit of the imaging device,
A first pixel addition drive mode in which captured image signals corresponding to the detected charges of the two photoelectric conversion elements constituting the pair pixel are individually read out by the signal readout circuit and added by the signal processing circuit; and the pair pixel The detection charges of the two photoelectric conversion elements are mixed, and a second pixel addition drive mode in which a captured image signal corresponding to the mixed charge amount is read by the signal readout circuit, and a subject image is captured by the imaging element A drive step for switching according to the shooting state when
In the driving step, the light receiving surface of the imaging element on which the photoelectric conversion element is formed is divided into a plurality of blocks, and the first pixel addition driving mode and the second pixel addition driving mode are switched for each divided block. ,
When the signal readout circuit reads the captured image signal from the paired pixels in the same row in a state where the first pixel addition drive mode and the second pixel addition drive mode coexist, the second pixel addition drive mode Among the output timings of the image sensor, a dummy signal is output at the same position as the output timing of the captured image signal read out first among the pair of pixels output in the first pixel addition drive mode, and the dummy signal and the The captured image signal corresponding to the mixed charge amount is signal-added by the signal processing circuit, and the signal processing by the signal processing circuit is added to the signal output from the image sensor in the first pixel addition drive mode. The pixel addition method of the imaging device, which is the same processing as in FIG . 5. The pixel addition method for an image pickup apparatus according to claim 4 , wherein odd-numbered photoelectric conversion element rows are formed by being shifted by 1/2 pitch with respect to even-numbered photoelectric conversion element rows. . A pixel addition method of an imaging apparatus according to claim 4 or claim 5, wherein the imaging condition for determining the switching between the first pixel addition driving mode and the second pixel addition driving mode, the pair of pixels A pixel addition method for an imaging apparatus that determines whether the amount of charge detected by one of the photoelectric conversion elements is equal to or greater than a first threshold value.

Images