JP7292990B2 - Imaging device, computer program and storage medium - Google Patents

Description

The present invention relates to an imaging device and the like.

2. Description of the Related Art One focus detection method performed by an imaging device is an imaging plane phase difference method that performs phase difference focus detection using signals from focus detection pixels provided in an imaging device.

Japanese Unexamined Patent Application Publication No. 2002-100001 discloses an imaging device using a two-dimensional imaging element in which one microlens and a plurality of divided photoelectric conversion units are formed for one pixel. The plurality of photoelectric conversion units are configured to receive light in different regions of the exit pupil of the imaging lens via one microlens, and perform pupil division. A phase difference signal is obtained from the image shift amount of the signals of the individual photoelectric conversion units to perform focus detection, and the signals of the individual photoelectric conversion units are added to obtain an image signal for display/recording. In addition to focus detection, it is disclosed that a stereoscopic image can be displayed by displaying signals received by the right and left photoelectric conversion units of each pixel as parallax signals.

Furthermore, in Japanese Patent Application Laid-Open No. 2002-200000, the isolation region between the divided photoelectric conversion units is composed of a semiconductor of a conductivity type different from that of the photoelectric conversion units, and the photoelectric conversion units are separated by PN junctions. It is disclosed that this also photoelectrically converts the light condensed on the separation region, making it possible to reduce non-uniform sensitivity in the imaging signal.

Further, in Patent Document 3, the amount of saturated charge is increased by accumulating the charge overflowing from the photoelectric conversion unit in a capacitor during the accumulation period and reading out the accumulated charge with two charge-voltage conversion gains when reading out the signal. A possible pixel configuration is shown.

JP-A-58-24105 JP-A-2001-250931 JP 2005-328493 A

However, in a two-dimensional imaging device having one microlens and a plurality of photoelectric conversion units for each pixel, if the region between the plurality of photoelectric conversion units is made of a semiconductor of a conductivity type different from that of the photoelectric conversion units, the potential barrier becomes smaller. Therefore, the saturation charge amount stored in the PD becomes small. Therefore, even if the technique disclosed in Japanese Patent Application Laid-Open No. 2003-100001 is used as it is in an imaging device in which each pixel outputs an image signal/phase difference signal, it is not possible to increase the saturation charge amount while suppressing image quality deterioration.
SUMMARY OF THE INVENTION It is an object of the present invention to solve the above-described problems of the prior art and to provide an imaging apparatus capable of increasing the saturation charge amount while suppressing deterioration of image quality depending on the mode.

An imaging device,
having a plurality of pixels, each pixel respectively
a plurality of photoelectric conversion units for receiving and photoelectrically converting a plurality of light beams passing through different exit pupil regions of the optical system to generate electric charges;
a separation region that separates the plurality of photoelectric conversion units;
an imaging device having a plurality of first storage units in the same number as the plurality of photoelectric conversion units for respectively storing charges of the plurality of photoelectric conversion units;
a first potential barrier between the plurality of photoelectric conversion units and the plurality of first storage units that is lower than the potential barrier of the isolation region and higher than the depletion potential of the plurality of photoelectric conversion units; Control means for selectively switching between a mode and a second mode in which a potential barrier between the plurality of photoelectric conversion units and the plurality of first accumulation units is higher than the potential barrier of the isolation region. and

According to the present invention, it is possible to provide an imaging apparatus capable of increasing the saturation charge amount while suppressing deterioration of image quality depending on the mode.

1 is a block diagram of an image capturing apparatus using an image sensor according to Example 1; FIG. 1 is a schematic diagram of the overall configuration of an imaging element in Example 1. FIG. 2 is a schematic diagram of a pixel array of an imaging element in Example 1. FIG. 2A and 2B are a schematic plan view and a schematic cross-sectional view of pixels of an image sensor in Example 1. FIG. FIG. 2 is a schematic explanatory diagram of pupil division in Example 1 of the present invention; 4 is a diagram illustrating pupil intensity distribution in Example 1. FIG. 4 is a flowchart of imaging processing in Example 1. FIG. 3 is a pixel equivalent circuit diagram in Example 1. FIG. 4 is a diagram (potential A1) schematically showing an example of a minimum potential of a pixel during a phase difference signal accumulation period in Example 1. FIG. FIG. 10 is a diagram (potential A1) schematically showing an example of the minimum potential of a pixel and the accumulated charge during the phase difference signal accumulation period in Example 1; 4 is a diagram (potential B1) schematically showing an example of a minimum potential of a pixel during an image signal accumulation period in Example 1. FIG. FIG. 10 is a diagram (potential B1) schematically showing an example of the minimum potential of a pixel and the accumulated charges during the image signal accumulation period in Example 1; 4 is a diagram schematically showing an image signal readout sequence in Example 1. FIG. 4 is a diagram schematically showing a pixel signal readout sequence in Example 1. FIG. FIG. 10 is a diagram (potential A1) schematically showing the relationship between the accumulated charge of each node and the amount of light in Example 1; FIG. 10 is a diagram (potential B1) schematically showing the relationship between the accumulated charge of each node and the amount of light in Example 1; 4 is a diagram schematically showing an example of charge accumulation during an image signal accumulation period in Example 1. FIG. 4 is a diagram schematically showing the relationship between the output signal voltage/SN ratio of the phase difference signal and the amount of light in Example 1. FIG. 4 is a diagram schematically showing the relationship between the output signal voltage/SN ratio of an image signal and the amount of light in Example 1. FIG. FIG. 11 is a diagram (potential B2) schematically showing an example of a minimum potential of a pixel during an image signal accumulation period in Example 2; FIG. 11 is a diagram (potential B2) schematically showing the relationship between the accumulated charge of each node and the amount of light in Example 2; FIG. 16 is a diagram (potential A3) schematically showing an example of a minimum potential of a pixel during a phase difference signal accumulation period in Example 3; FIG. 11 is a diagram (potential A3) schematically showing the relationship between the accumulated charge of each node and the amount of light in Example 3; FIG. 11 is a pixel equivalent circuit diagram in Example 3;

BEST MODE FOR CARRYING OUT THE INVENTION Preferred embodiments of the present invention will be described below using examples with reference to the accompanying drawings. In each figure, the same members or elements are denoted by the same reference numerals, and overlapping descriptions are omitted or simplified.
In the embodiments, an example in which a digital still camera is used as an imaging device will be described. include.
(Example 1)

FIG. 1 is a block diagram of an imaging apparatus (digital still camera) 100 using an imaging element according to Embodiment 1 of the present invention.

The first lens group 101 is arranged at the tip of an imaging optical system (imaging optical system) and held so as to be able to advance and retreat in the optical axis direction. The diaphragm/shutter 102 functions as a shutter for controlling the exposure time when capturing a still image, and also functions as a diaphragm for adjusting the amount of light when capturing an image by adjusting the aperture diameter. The second lens group 103 advances and retreats in the optical axis direction integrally with the aperture/shutter 102, and realizes a zoom function in conjunction with the advance and retreat movement of the first lens group 101. FIG.

The third lens group 105 (focus lens) performs focus adjustment by advancing and retreating in the optical axis direction. The optical low-pass filter 106 is an optical element for reducing false colors and moiré in a captured image. The imaging device 107 is a CMOS image sensor, is arranged on the imaging plane of the imaging optical system, and converts incident light into electrical signals.

The zoom actuator 111 rotates a cam cylinder (not shown) to move the first lens group 101 to the second lens group 103 back and forth in the optical axis direction, thereby performing a zooming operation. A diaphragm shutter actuator 112 controls the aperture diameter of the diaphragm/shutter 102 to adjust the amount of photographing light, and controls the exposure time during still image photographing. A focus actuator 113 moves the third lens group 105 back and forth along the optical axis to adjust the focus.

Reference numeral 114 denotes an electronic flash for illuminating an object during night photography. A flash illumination device using a xenon tube is preferable, but an illumination device equipped with an LED that continuously emits light may be used. The AF auxiliary light emitting unit 115 projects an image of a mask having a predetermined aperture pattern onto the object field through a projection lens to improve focus detection capability for dark or low-contrast objects.

The in-camera CPU 121 manages various controls of the camera body, and has an arithmetic unit, ROM, RAM, A/D converter, D/A converter, communication interface circuit, and the like. The CPU 121 drives various circuits in the camera based on a predetermined computer program stored in the ROM, and controls a series of operations such as imaging, AF, image processing, and recording as control means.

A flash control circuit 122 controls lighting of the electronic flash 114 in synchronization with a photographing operation. The auxiliary light driving circuit 123 controls lighting of the AF auxiliary light emitting section 115 in synchronization with the focus detection operation. The image pickup device drive circuit 124 controls the image pickup operation of the image pickup device 107 , A/D-converts an image signal acquired from the image pickup device 107 , and transmits the signal to the CPU 121 . The image processing circuit 125 performs processing such as gamma conversion, color interpolation, and JPEG compression on the image acquired by the image sensor 107 .

A focus drive circuit 126 performs focus detection by the in-camera CPU 121 based on the signal obtained from the image pickup device 107, drives and controls the focus actuator 113 based on the result, and drives the third lens group 105 forward and backward in the optical axis direction. Adjust the focus. A diaphragm shutter driving circuit 127 drives and controls the diaphragm shutter actuator 112 to control the opening of the diaphragm shutter 102 . A zoom driving circuit 128 drives the zoom actuator 111 according to the zoom operation of the photographer to change the zoom magnification.

A display 131 is composed of an LCD or the like, and displays information about the photographing mode of the camera, a preview image before photographing, an image for confirmation after photographing, an in-focus state display image at the time of focus detection, and the like. The operation switch group 132 is a group of operation switches, and includes a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, and the like. A flash memory 133 detachable from the imaging device 100 is for recording/reproducing captured images.
Next, the imaging element 107 in this embodiment will be described. FIG. 2 is a diagram schematically showing the overall configuration of the CMOS image sensor (imaging element) of this embodiment.

In the pixel array 201, a plurality of pixels each including a photoelectric conversion unit and a signal readout circuit are arranged two-dimensionally. A predetermined row is selected by the vertical selection circuit 202, and signals are output to vertical signal lines from pixels included in the predetermined row. It is possible to provide one vertical signal line for each pixel column, one for each of a plurality of pixel columns, or a plurality of vertical signal lines for each pixel column. The column circuit 203 receives signals read out in parallel to a plurality of vertical lines. The column circuit 203 can perform processing such as signal amplification and noise removal, and hold the signal. The horizontal selection circuit 204 sequentially outputs the signals held in the column circuit 203 to horizontal output lines (not shown) at random or simultaneously. Note that the A/D conversion function installed in the image sensor drive circuit 124 may be installed for each column of the column circuit 203 or may be installed for each pixel.

Also, the CMOS image sensor may be of a stacked type, and the A/D conversion function and other signal processing circuits may be mounted in a layer different from the layer in which the pixel array is arranged.
FIG. 3 shows the pixel arrangement of the pixel array 201 in a range of 4 columns×4 rows, and the divided photoelectric conversion units in a range of 8 columns×4 rows. In the drawing, the x-axis direction is called the row direction, and the y-axis direction is called the column direction.
The pixel group 300 is a unit consisting of 2 rows×2 columns of pixels. A pixel 300B having a (blue) spectral sensitivity is arranged at the lower right.

Further, each pixel is composed of photoelectric conversion units arranged in two columns and one row. The photoelectric conversion portion, which is the light receiving portion, is composed of a photodiode (PD) capable of complete charge transfer. PD _B 302 is the PD arranged on the side of _A 301 and relatively large x (right side in the drawing).
A large number of 4 columns×4 rows of pixels (8 columns×4 rows of PDs) shown in FIG. In addition, a pixel having only one photoelectric conversion unit may exist in a part of the pixel array without dividing the pixel into two photoelectric conversion units PD _A and PD _B.

FIG. 4A shows a plan view of one pixel 300G in the pixel array shown in FIG. A cross-sectional view seen from the -y side is shown in FIG. As shown in FIG. 4, in the pixel 300G of this embodiment, a microlens 401 for collecting incident light is formed on the light receiving side of each pixel, which is divided into two in the x direction and not divided in the y direction. PD _A and PD _B are formed. A color filter 402 is formed between the microlens 401 and the PD _A and PD _B in each pixel. Moreover, if necessary, the spectral transmittance of the color filter may be changed for each focus detection pixel, or the color filter may be omitted.

Light incident on the pixel 300G shown in FIG. 4 is condensed by the microlens 401, separated into green by the green color filter 402, and then received by the PD _A and PD _{B. FIG} . In PD _A and PD _B , pairs of electrons and holes are generated according to the amount of light received, and after being separated in the depletion layer, negatively charged electrons are collected and accumulated as signal charges, and positively charged holes are collected and stored at a constant voltage. It is discharged to the outside of the imaging device through a p-type semiconductor region connected to a source (not shown). Holes may be used as signal charges by reversing the n-type semiconductor region and the p-type semiconductor region forming the photoelectric conversion portion.

The pixels 300R and 300B shown in FIG. 3 have the same configuration as the pixel 300G, and like the pixel 300G, the color filter 402 collects and accumulates signal charges according to the light split into each color.

A correspondence relationship between the pixel structure of the present embodiment shown in FIG. 4 and pupil division will be described with reference to FIGS. FIG. 5 is a schematic diagram showing the relationship between the aa section of the pixel structure of this embodiment shown in FIG. 4B and the exit pupil plane of the imaging optical system, and FIG. is an example of the pupil intensity distribution along the x (axis) of . The light-receiving surface and the pupil region 500 are generally in a conjugate relationship due to the microlenses, and the light-receiving region and the exit pupil region 501 of the PD _A and the light-receiving region and the exit pupil region 502 of the PD _B are each approximately matched. It is configured. That is, the light-receiving area of PD _A receives the luminous flux that has passed through exit pupil area 501 , and the light-receiving area of PD _B receives the luminous flux that has passed through exit pupil area 502 . As shown in FIG. 6, the pupil intensity distributions of PD _A and PD _B divide the exit pupil in the x (axis) direction.

As shown in FIG. 3, the _PDAs are regularly arranged in the x (axis) direction, and the subject signal acquired from the group of _PDAs is defined as the A image signal. Similarly, the PD _B 's are arranged regularly in the x (axis) direction, and the subject image obtained from the PD _B group is defined as a B image signal. By detecting the image shift amount (phase difference) between the A image signal and the B image signal, the defocus amount (focus shift amount) of the subject image having the luminance distribution in the x (axis) direction can be detected.
FIG. 7 is a flowchart of subject-tracking AF continuous shooting performed by the imaging apparatus 100 according to the present embodiment, and subject-tracking AF continuous shooting will be described with reference to FIG.

In the imaging apparatus 100, it is assumed that the switch for selecting the shooting mode included in the operation switch group 132 is operated by the user, and the shooting mode is the subject tracking AF continuous shooting mode. In the photographing mode, when the release switch SW2 included in the operation switch group 132 is turned on, photographing is started, and the process proceeds to step S701.
Step S701 is an accumulation period for a recording image, during which exposure and accumulation are performed with the in-pixel potential set to a potential B1 (second mode), which will be described later. The amount of light is controlled using the aperture/shutter 102 .

In step S702, the image signal obtained from the accumulated charges is read out from the image sensor using the image sensor drive circuit 124 . During readout, the diaphragm/shutter 102 is closed by the diaphragm/shutter actuator 112 so that the imaging device 107 is controlled so that excess electric charge is not accumulated. The charge signals generated by a plurality of photoelectric conversion units in each pixel are finally added by the CPU 121, for example, in units of pixels to form an image signal. The image signal is stored in the flash memory 133 as a recorded image, and is also used as an LV (Live View) display image for the display device 131 and the like.

Step S703 is an accumulation period for phase difference detection, during which exposure and accumulation are performed with the intra-pixel potential set to a potential A1 (first mode), which will be described later. The amount of light is controlled using the aperture/shutter 102 .
In step S<b>704 , signals including parallax information accumulated by the plurality of photoelectric conversion units of each pixel are separately read out to the CPU 121 via the image sensor driving circuit 124 . Signals from a plurality of photoelectric conversion units are individually processed by the CPU 121 to generate phase difference signals, which are used for defocus amount calculation and focus detection. Note that in this embodiment, this signal is used not only for defocus amount calculation and the like, but also for the CPU 121 to use the above phase difference signal for the LV display image and the like so that it can be displayed on the display unit. Note that an image processing circuit 125 provided separately from the CPU 121 may perform the above addition processing, image signal formation, and the like.

In step S705, the lens drive amount is calculated based on the defocus amount calculated in step S704.
In step S706, the lens is driven based on the lens driving amount calculated in step S705.
In step S707, it is determined whether the release switch (SW2) included in the operation switch group 132 is on or off. If it is on, the process returns to step S701 to continue continuous shooting. If it is off, the continuous shooting is stopped and the shooting ends.

Next, the pixel structure and the accumulation operations in steps S701 and S703 will be described.
FIG. 8 is an equivalent circuit diagram of a pixel. The pixel has a _PDA 301, a _PDB 302, and a transfer transistor (T _A ) 801 as a transfer gate for transferring charges accumulated in the _PDA . It also has a transfer transistor (T _B ) 802 as a transfer gate for transferring charges accumulated in PD _B. 803 is a first storage unit (FD _A ) to which charges accumulated in PD _A are transferred, and 804 is a first storage unit (FD _B ) to which charges accumulated in PD _B are transferred. That is, it has two first accumulation units for respectively accumulating the charges of the two photoelectric conversion units.

Correspondingly, 805 is a storage capacity (CS _A ) as a second storage section selectively connected to FD _A , and 806 is a storage capacity (CS A ) as a second storage section selectively connected to FD _{B. B} ). 807 is a storage transistor (S _A ) as a storage gate for connecting FD _A and CS _A , and 808 is a storage transistor (S _B ) as a storage gate for connecting FD _B and CS _B. 809 is a reset transistor (R _A ) for resetting CS _A , FD _A and _PDA , and 810 is a reset transistor (R _B ) for resetting CS _B , FD _B and PD _B. 811 is an amplification transistor (SF _A ) that outputs the voltage of the node connected to FD _A from the pixel, and 812 is an amplification transistor (SF _B ) that outputs the voltage of the node connected to FD _B from the pixel.

813 is a selection transistor (X _A ) for connecting SF _A and an output line (OUT _A ) 815 , and 814 is a selection transistor (X _B ) for connecting SF _B and an output line (OUT _B ) 816 . Note that this embodiment shows an example in which two photoelectric conversion units are provided, and correspondingly, two first storage units and two second storage units are provided. However, the number of photoelectric conversion units may be three or more, and in that case, the same number of first storage units, second storage units, transfer transistors, storage transistors, selection transistors, etc. are provided according to the number of photoelectric conversion units. You should set it.

FIG. 9 is a diagram schematically showing the intra-pixel potential A1 (first mode) used in step S703, which is the accumulation period for phase difference detection. _In the figure, _PDA , _PDB , _TA , _TB , FDA, _FDB , _SA , _SB , _CSA , _CSB , and the potential of the separation section 901 separating _PDA and _PDB with respect to the signal charge is schematically shown. FIG. 10 is a diagram schematically showing an example of charge accumulation at the end of the accumulation period in step S703. As shown in FIGS. 9 and 10, the off-voltages of T _A and T _B are adjusted such that the height 902 of the potential barrier in the isolation portion 901 is higher than the height 903 of the potential barrier in the T _A and T _B portions. It is set (controlled).

Furthermore, at that time, the potential barrier height 903 of the T _A and T _B parts is set higher than the depletion potential of the plurality of photoelectric conversion parts (potential height of PD _A and PD _B in FIG. 9). controlling. As a result, as shown in FIG. 10, when the _PDA is saturated during the accumulation period in step S703, the electric charge exceeding the potential barrier of the _TA portion moves to FD _A and CS _{A and} is accumulated. Then, when the PD _A is saturated, the electric charges exceeding the potential barrier of the _TB portion move to the FD _B and CS _B and are accumulated.

FIG. 11 is a diagram schematically showing the potential B1 (second mode) used during the accumulation period of step S701. FIG. 11 schematically shows potentials of _PDA , _PDB , _TA , TB, FD _A , FD _B , _SA , _SB , CS _A , CS _B , _and the separation section 901 with respect to the signal charge. . FIG. 12 is a diagram schematically showing an example of charge accumulation at the end of the accumulation period in step S701. The off voltages of T _A and T _B are set so that the height 902 of the potential barrier of the separation portion 901 is lower than the height 1101 of the potential barrier of the T _A and T _B portions. As a result, as shown in FIG. 12, it is possible to increase the amount of charge that can be stored in PD _A and PD _B , and if PD _A or PD _B is saturated during the storage period, the amount of charge that exceeds saturation can be increased. is moved to and accumulated in the paired PD.
Next, the method of reading the image signal in steps S702 and S704 will be described.

FIG. 13 is a diagram schematically showing an image signal readout sequence. Image signal readout is performed by repeating the operation of reading pixel signals of a row selected by the vertical selection circuit 202 to the column circuit 203 and then reading them from the image sensor 107 by the horizontal selection circuit 204 .
In a period 1303, a signal _N2A and a signal _N2B , which will be described later, are sequentially read for each row to output lines 815 and 816, respectively. During the accumulation period 1304 after the reading of the signal N2 _A and the signal N2 _B is completed, the diaphragm shutter actuator 112 opens and closes the diaphragm shutter 102 to perform exposure. In a period 1305, a signal _N1A , a signal _N1B , a signal _N1A + _S1A , a signal _N1B + _S1B , a signal _N2A + _S2A , and a signal _N2B + _S2B of each pixel, which will be described later, are read.
Next, a pixel signal readout operation for reading out pixel signals to the column circuit 203 will be described.

FIG. 14 is an example of a pixel driving sequence. t1 to t11 in FIG. 14 represent respective timings in the sequence. In addition, _φXA represents ON/OFF of _XA , and the ON state is indicated by the upper side, and the OFF state by the lower side. For example, _XA is on at timing t1, and _XA is off at timing t3. The same is true for other transistors. In addition, in this sequence, PD _A and PD _B , XA and _{X B} , RA and _RB , S _A and _SB , and _{TA and TB} are driven at the same timing. If not, PD _A and PD _B are denoted by PD, XA _and X _B by X, _RA and _RB by R, _SA and _SB by S, and T _A and _TB by T. Periods 1401 to 1403 correspond to periods 1303 to 1305 in FIG.

First, at timing t1 in period 1401 (period 1303), X, R, S, and T are turned on to reset PD, FD, and CS. After that, after turning off R and T, the voltage of FD+CS is read out from the pixel using SF at timing t2. Among the signals obtained from these voltages, the signal on the OUT _A side is called signal _N2A , and the signal on the OUT _B side is called signal _N2B . These signals N2 _A and N2 _B correspond to fixed pattern noise signals.

In a period 1403 (period 1305) after the accumulation period 1402 (period 1304), X is turned on at timing t5 and the FD voltage is read out from the pixel using SF.
Among the signals obtained from these voltages, the signal on the OUT _A side is called signal _N1A , and the signal on the OUT _B side is called signal _N1B . Signals N1 _A and N1 _B correspond to random noise and fixed pattern noise signals.
Next, T is turned on at timing t6 to transfer the signal charge accumulated in PD to FD, and after T is turned off, the voltage of FD is read out from the pixel using SF at timing t7. Among the signals obtained from these voltages, the signal on the OUT _A side is called signal N1 _A +S1 _A and the signal on the OUT _B side is called signal N1 _B +S1 _B. The signals S1 _A and S1 _B correspond to charge signals accumulated in the PD at the end of the accumulation period.

Next, the storage transistor S is turned on, the FD and CS are connected, and the charges transferred to the FD and the charges accumulated in CS are mixed. Subsequently, when T is turned on at timing t8 and there are charges that could not be transferred at t1 and t6 among the charges accumulated in PD, they are transferred to FD and CS. Subsequently, the voltage of FD+CS after T is turned off at timing t9 is read from the pixel using SF. Among the signals obtained from these voltages, the signal on the OUT _A side is called signal N2 _A +S2 _A and the signal on the OUT _B side is called signal N2 _B +S2 _B. Here, the signal S2 _A and the signal S2 _B correspond to charge signals accumulated in PD, FD, and CS at the end of the accumulation period.

Using the signals read from each pixel, the CPU 121 performs the following arithmetic processing,
(N1 _A + S1 _A )-N1 _A = S1 _A
(N1 _B +S1 _B )−N1 _B =S1 _B
( _N2A + _S2A )-N2A= _S2A
(N2 _B +S2 _B )-N2 _B =S2 _B
, the random noise and the fixed pattern noise are removed to obtain the signal _S1A , the signal _S1B , the signal _S2A , and the signal _S2B . As described above, the signal S1 _A and the signal S1 _B correspond to charge signals accumulated in the PD at the end of the accumulation period.

Signals S2 _A and S2 _B correspond to charge signals accumulated in PD, FD, and CS at the end of the accumulation period. Since the signal S1 _A and the signal S1 _B have a large charge-voltage conversion gain, the signal readout noise is small compared to the signal S2 _A and the signal S2 _B , and the signal-to-noise ratio is high. By selecting and combining suitable signals from these signals, optimum phase difference signals and image signals are obtained.
Next, a method for selecting read pixel signals will be described.

FIG. 15 shows an example of the relationship between the amount of signal charge accumulated in PD _A , _PDB , FD _A , FD _B , CS _A and CS _B and the amount of light at potential A1 (first mode). Signal selection for each light amount range will be described below. Signal selection is performed by the CPU 121 . If the output value of signal _S2A exceeds a predetermined first threshold (1802 in FIG. 18), signal _S2A is selected, and if it is less than the threshold, signal _S1A is selected. Similarly, if the output value of the signal _S2B exceeds the first threshold value, the signal _S2B is selected, and if the output value is equal to or less than the threshold value, the signal _S1B is selected. By doing so, the charge generated in the _PDA selects a signal _S1A with a high SN ratio and low readout noise when the light amount is in the range of 1501 and 1502 in FIG. 15, for example.

On the other hand, the signal _S2A , which is a high saturation charge amount, is selected in the range where the amount of light is 1503 or more so that the generated charges overflow and accumulate in FD _A and CS _A . Similarly, regarding the selection of the signal _S1B and the signal _S2B , the signal _S1B is selected when the light amount is in the range 1501 in FIG. 15, and the signal _S2B is selected when the light amount is in the range of 1502 or 1503 or more. The selected signal is processed as a phase difference signal by the CPU 121 and used for focus detection. With this configuration, it is possible to increase the saturation charge amount in pixels under high illuminance while maintaining the SN ratio in pixels under low illuminance. However, since the amount of light at the switching point is determined by the amount of saturated charge of the signal S1, the amount of light is small when the amount of saturated charge of the PD is small, and the amount of light is large when the amount of saturated charge of the PD is large.

Since the saturation charge amount of the PD is relatively small in the accumulation at the potential A1, it is disadvantageous when the image signal is accumulated in the PD. That is, if the saturation charge amount of the PD is relatively small, it is necessary to reduce the additional capacitance CS and increase the SN ratio of the signal S2 in order to suppress the deterioration of the image quality at the switching point. Difficult to increase.
Therefore, in this embodiment, control is performed as shown in FIGS. 16 and 17 when generating image signals for display/recording. That is, FIG. 16 is a diagram showing an example of the relationship between the amount of signal charge accumulated in _PDA , PDB, FD _A , FD _B , CS _{A and} CS _B and the amount of light. , 1603. FIG.

When the amount of light is in the range of 1601, charges generated by PD _A are accumulated in PD _A , and charges generated by PD _B are accumulated in PD _B , as shown in FIG. 17A. At this time, in the range where the amount of light is 1602, as shown in FIG. 17B, charges generated in PD _A are accumulated in _PDA , and charges generated in PD _B are accumulated in PD _B and _PDA . In the range where the amount of light is 1603, charges generated in PD _A and PD _B are accumulated in PD _A and PD _B as shown in FIG. 17(C). For signal selection, if the sum of the output values of signal _S2A and signal _S2B exceeds a predetermined second threshold value (1902 in FIG. 19), signal _S2A and signal _S2B are selected, and the second is equal to or less than the threshold, the signal S1-- _A and the signal S1-- _B are selected.

In this way, the signals _S1A and S1B are selected in the ranges of light amounts 1601, 1602, and ₁₆₀₃ where charges generated in PD _A and PD _B are accumulated in PD _A and PD _B. Further, in the range where the amount of light is 1604 and the generated charge is also accumulated in the FD and CS, the signal S2A and the signal S2B are selected. These charge signals are finally added by the CPU 121 to form an image signal.
The relationship between the output signal voltage/SN ratio obtained in this way and the amount of light will be described. FIG. 18 is a diagram schematically showing the relationship between the output signal voltage and the SN ratio in the case of the potential A1 for the phase difference signal, and FIG. 19 is the relationship in the case of the potential B1 for the image signal.

In the image signal, the signal S1 has a larger saturated charge number than the phase difference signal. Therefore, the second threshold 1902 for the image signal is a larger value than the first threshold 1802 for the phase difference signal. can do. By doing so, the threshold value of the image signal becomes independent of the PD saturation charge amount determined by the potential barrier 902 of the separation section 901 between PD _A and PD _B. FIG. Therefore, in an imaging device that outputs an image signal/phase difference signal for which it is difficult to increase the potential barrier, it is possible to increase the saturation charge amount while suppressing deterioration in image quality at the switching point of the image signal.

In addition, since the SN ratio of the phase difference signal can be improved by performing pixel addition after the signal is read out from the image sensor, the image quality can be improved even if the switching point output from the pixel is set lower than the image signal. Deterioration can be suppressed. For example, when adding N pixels, the SN ratio improves by 10×log(N). Also, image quality deterioration can be suppressed.
By configuring as described above, it is possible to increase the saturation charge amount without deteriorating the image quality in the image pickup device in which each pixel outputs an image pickup/phase difference signal.
(Example 2)

The second embodiment of the present invention has the same overall configuration, camera operation sequence, etc. as the first embodiment, and the pixel potential in the image signal accumulation period of step S701 is different from the potential B1 of the first embodiment.
FIG. 20 is a diagram schematically showing the minimum potential of pixels for image signal accumulation according to Example 2, which is referred to as potential B2 in this example. 11, the potential B2 has a potential difference between the potential barrier 2001 of the transfer gate _TA and the potential barrier 2002 of _TB . With such a configuration, charges generated when the PD is saturated preferentially move to the FD _A.

FIG. 21 is a diagram schematically showing the relationship between the accumulated charge and the amount of light at each node of potential B2. In the light intensity ranges of 1601, 1602, and 1603, the generated charges are accumulated only in _PDA and _PDB . In the range 2101, light is accumulated in PD _A , _PDB , FD _A , and CS _A , and in the range 2102, light is accumulated in _PDA , _PDB , FD _A , CS _A , FD _B , and CS _B.

If it is determined in advance that the light amount range can be predicted and it is not necessary to read out the light amount range 2102, the signal _S2B is not read out from the image pickup device. and speed up.
(Example 3)

The third embodiment of the present invention is the same as the first embodiment in terms of the overall configuration, the camera operation sequence, etc., but differs in the pixel potential during the phase difference signal accumulation period in step S703.
FIG. 22 is a diagram schematically showing the minimum potential of a pixel for accumulating a phase difference signal according to Example 3, which is referred to as potential A2 in this example. Potential A2 is a kind of first mode. However, compared to FIG. 9, the potential barrier 2203 in the region 2201 between the storage capacitor CS _A and the storage capacitor CS _B in the same pixel has a potential set lower than the fully depleted potential of PD _A and PD _B. there is That is, the height of the potential barrier between the second accumulation portions in the same pixel is made lower than the fully depleted potential of the photoelectric conversion portion. With such a configuration, when CS _A is saturated, charges move to CS _B and are accumulated, and when CSB is saturated, charges move to CS _A and are accumulated.

FIG. 23 is a diagram schematically showing the relationship between the accumulated charge and the amount of light at each node of the potential A2. Charges generated in the _PDA are accumulated in the _PDA when the light amounts are in the range of 1501 and 1502, and are accumulated in the _PDA , FD _A and CS _A when the light amounts are in the range of 1503 and 2301. FIG. The charges generated in the PD _B are accumulated in the PD _B when the light amount is in the range of 1501, in the PD _B , FD _B and CS _B when the light amount is in the range of 1502 and 1503, and in the PD _B and FD when the light amount is in the range of 2301. _B , CS _B and CS _A are stored.

With the potential A1, charges generated _{in the PD B can} be accumulated within the light amount range until the CS _B is saturated. For example, it is possible to accumulate charges generated in the _PDB . Therefore, with the potential A1, phase difference signals and image signals are obtained when the light amount range is 1503 or less. An image signal is obtained when Therefore, the light amount range in which the phase difference signal can be obtained remains unchanged, and the light amount range when the phase difference signal is used for LV display can handle a higher illuminance with the potential A2 than with the potential A1.

A potential barrier between CS _A and CS _B may be formed by a PN junction in advance at the time of design. Alternatively, as shown in FIG. 24, a capacitive coupling transistor (MCS) 2401 is provided as a connection gate connecting CS _A and CS _B , and the potential barrier height between CS _A and CS _B is increased by the bias voltage of the connection gate. may be set. That is, the height of the potential barrier between the second accumulation portions within the same pixel may be controlled by the bias voltage of the connection gate.

Although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to the above embodiments, and various modifications are possible based on the gist of the present invention. They are not excluded from the scope of the invention.
For example, in the embodiment, signals from PD _A and PD _B are used for phase difference detection, but they may be displayed as parallax signals and used for displaying a stereoscopic image. Alternatively, it can be used to create a distance map corresponding to each part in the image.

Further, a computer program that implements the functions of the above-described embodiments may be supplied to the imaging apparatus via a network or various storage media for part or all of the control in this embodiment. Then, the computer (or CPU, MPU, etc.) in the imaging apparatus may read and execute the program. In that case, the program and the storage medium storing the program constitute the present invention.

111 Zoom actuator 112 Aperture shutter actuator 113 Focus actuator 114 Electronic flash 115 AF auxiliary light emitting unit 122 Flash control circuit 123 Auxiliary light drive circuit 124 Image sensor drive circuit 125 Image processing circuit 126 Focus drive circuit 127 Aperture shutter drive circuit 128 Zoom drive circuit 131 display 132 operation switch group 133 flash memory

Claims (13)

having a plurality of pixels, each pixel respectively
a plurality of photoelectric conversion units for receiving and photoelectrically converting a plurality of light beams passing through different exit pupil regions of the optical system to generate electric charges;
a separation region that separates the plurality of photoelectric conversion units;
an imaging device having a plurality of first storage units in the same number as the plurality of photoelectric conversion units for respectively storing charges of the plurality of photoelectric conversion units;
a first potential barrier between the plurality of photoelectric conversion units and the plurality of first storage units that is lower than the potential barrier of the isolation region and higher than the depletion potential of the plurality of photoelectric conversion units; Control means for selectively switching between a mode and a second mode in which a potential barrier between the plurality of photoelectric conversion units and the plurality of first accumulation units is higher than the potential barrier of the isolation region. and an imaging device. The control means is characterized in that, in the second mode, the charges respectively generated by the plurality of photoelectric conversion units are added through the plurality of first storage units and read out to form an image signal. The imaging device according to claim 1. The control means is characterized in that, in the first mode, the charges respectively generated by the plurality of photoelectric conversion units are read through the plurality of first storage units to form phase difference signals. Item 1. The imaging device according to item 1. 3. The control means is characterized in that, in the first mode, the charges respectively generated by the plurality of photoelectric conversion units are read through the plurality of first storage units to form image signals. 4. The imaging device according to 3. a plurality of second storage units, which are the same in number as the plurality of first storage units, for respectively storing a portion of the charges accumulated in the plurality of storage units;
2. The imaging device according to claim 1, further comprising a plurality of storage gates for controlling potential barriers between said plurality of first storage units and said plurality of second storage units. 3. The control means, in the second mode, makes different heights of potential barriers between the plurality of photoelectric conversion units and the plurality of first accumulation units in the same pixel. 1. The imaging device according to 1. In the first mode, the control means makes the height of the potential barrier between the second storage portions in the same pixel lower than the fully depleted potential of the photoelectric conversion portion. 6. The imaging device according to claim 5. 6. The imaging device according to claim 5, further comprising a connection gate for controlling the height of a potential barrier between said second accumulation portions within the same pixel. The control means includes a first pixel signal obtained by accumulating charges generated in the photoelectric conversion section in the first accumulation section;
reading out a second pixel signal obtained by accumulating electric charges generated in the photoelectric conversion unit with the first accumulation unit and the second accumulation unit being connected;
selecting the second pixel signal when the magnitude of the second pixel signal is greater than a predetermined threshold, and selecting the first pixel signal when the magnitude of the second pixel signal is less than the threshold; 6. The image pickup apparatus according to claim 5, wherein the image pickup device selects the image. 10. The imaging apparatus according to claim 9, wherein said control means changes the magnitude of said threshold between said first mode and said second mode. 2. The imaging device according to claim 1, further comprising a transfer gate for controlling potential barriers between said plurality of photoelectric conversion units and said plurality of first storage units. A computer program for causing a computer to function as the control means of the imaging apparatus according to any one of claims 1 to 11. 13. A computer readable storage medium storing the computer program according to claim 12.

Images