JP2021013147A - Imaging element and imaging apparatus - Google Patents

Abstract

To achieve both miniaturization of pixels and improvement of readout speed in an imaging element that contains multiple photoelectric conversion elements in a single pixel.SOLUTION: An imaging element 100 has a pixel unit consisting of a plurality of unit pixels 103 arranged in a matrix on a semiconductor chip 101. The unit pixel has two photoelectric conversion elements that convert light passing through different exit pupils of the photographing lens. One AD converter is provided for each unit pixel. A pixel control unit 105 of a semiconductor chip 102 controls the readout from the unit pixel using a plurality of AD converters. To read out pixel signals of a frame, the pixel control unit performs time-division on the frame into a plurality of subframes and performs a readout operation from the unit pixel for each subframe. In the readout operation of a first subframe, a first signal based on the signal from one photoelectric conversion element is acquired. In the readout operation of a second subframe, a second signal based on signals from all of the photoelectric conversion elements is acquired.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image pickup device and an image pickup apparatus.

In recent years, an image sensor called SPAD (Single Photon Avalanche Diode) has been studied. This is an image sensor that measures the number of incident photons itself by utilizing the avalanche phenomenon that occurs when an avalanche photodiode (APD) is operated in Geiger mode, that is, treats the incident light as a digital value.

When the APD is operated in Geiger mode, for example, when one photon is incident on the APD, an observable level of current is generated by the avalanche phenomenon. By converting this current into pulses and counting the number of pulses, it is possible to directly measure the number of incident photons, so RTS noise does not occur and S / N is expected to improve. .. Such an operation is called photon counting.

In photon counting, since the incident light is treated as a digital value, it is a basic condition that one pixel has one AD converter. This is a type of image sensor called pixel AD. Since the AD converter only needs to count the number of incident photons, it can be realized if there is a comparator and a counter that convert the current generated by the avalanche phenomenon into a pulse.

As an example of a sensing device using photon counting, Patent Document 1 discloses a distance measuring sensor composed of a plurality of pixels of SPAD.

On the other hand, there is a technique of performing a phase difference type focus detection at the same time as imaging by configuring the pixels in the image sensor to receive light from different pupil surfaces of the imaging lens. For example, in Patent Document 2, by dividing a photodiode (hereinafter referred to as “PD”) that is focused by one microlens in one pixel, each PD has a different pupil surface of a photographing lens. It is configured to receive light.

As a result, the image shift amount is detected, that is, the phase difference detection is performed from the signal of each PD that receives the light passing through the different pupil surfaces of the photographing lens, and the focus shift amount (defocus amount) is correlated with the image shift amount. The focal state is detected by obtaining it by calculation. It is also possible to generate a shooting signal for viewing by adding the signals of the PDs that have received the light of the different pupil surfaces of the imaging lens.

Japanese Unexamined Patent Publication No. 2014-081253 Japanese Unexamined Patent Publication No. 2001-083407

In order to realize a configuration having a plurality of photoelectric conversion elements in order to obtain phase difference information in pixel AD represented by photon counting, it is conceivable to provide as many AD converters as there are photoelectric conversion elements. However, since the AD converter requires a large area only with the comparator and the counter, the miniaturization of the pixels is hindered if the AD converters are provided as many as the number of photoelectric conversion elements. Therefore, in a configuration having a plurality of photoelectric conversion elements in one pixel, it is necessary to have a configuration in which even one AD converter per pixel can efficiently perform reading.

The present invention provides a technique advantageous for achieving both miniaturization of pixels and improvement of readout speed in an image pickup device including a plurality of photoelectric conversion elements in one pixel.

According to one aspect of the present invention, there is a pixel portion in which a plurality of unit pixels are arranged in a matrix, and each of the plurality of unit pixels photoelectrically converts light passing through different ejection pupils of a photographing lens. Reading from the plurality of unit pixels using a pixel unit having a photoelectric conversion element, a plurality of AD converters provided for each of the plurality of unit pixels, and the plurality of AD converters. An image pickup element having a pixel control unit for controlling, wherein the pixel control unit time-divides the frame into a plurality of subframes in order to read out a pixel signal of the frame, and the plurality of units for each subframe. The first photoelectric conversion element obtained in each of the plurality of first subframes is configured to perform the reading operation from the pixels, and in the reading operation of the plurality of first subframes among the plurality of subframes. In the read operation of the plurality of second subframes among the plurality of subframes, the first signal based on the signal from is acquired, and the second obtained in each of the plurality of second subframes is obtained. Provided is an image pickup element characterized in that a second signal based on a signal from a photoelectric conversion element is acquired.

According to the present invention, it is possible to provide a technique advantageous for achieving both miniaturization of pixels and improvement of readout speed in an image pickup device including a plurality of photoelectric conversion elements in one pixel.

The figure which shows the structure of the image pickup device in embodiment. The figure which shows the structure of 1 pixel of the image pickup device in embodiment. It is a figure which shows the structure of a pixel and the structure of a circuit which reads a signal from a pixel. The figure which shows the operation of an avalanche photodiode schematically. The timing diagram which shows typically the example of driving an image sensor. The figure which shows the structure of the image pickup device in embodiment. The timing diagram which shows typically the example of driving an image sensor. The timing diagram which shows typically the example of driving an image sensor. The timing diagram which shows typically the example of driving an image sensor. The figure which shows the structure of the image pickup apparatus which concerns on embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate description is omitted.

<First Embodiment>
FIG. 1 is a diagram showing a configuration of an image pickup device according to an embodiment. The image pickup device 100 includes a first semiconductor chip 101 that forms a sensor substrate laminated with each other, and a second semiconductor chip 102 that forms a circuit board. A pixel array (pixel portion) in which a plurality of unit pixels 103 are arranged in a matrix is formed on the first semiconductor chip 101 forming the sensor substrate. The detailed configuration of the unit pixel 103 will be described later. The second semiconductor chip 102 forming the circuit board may include a main control unit 104, a pixel control unit 105, and a signal processing circuit 106.
The main control unit 104 receives an operation instruction or serial communication from the outside, and supplies a timing signal or the like for operating the pixel control unit 105 and the signal processing circuit 106. The pixel control unit 105 is electrically connected to each of the plurality of unit pixels 103 of the first semiconductor chip 101 by bumps or the like, outputs a control signal for driving each of the plurality of unit pixels 103, and is a buffer from the pixels. Receive the output.

The pixel control unit 105 may include a plurality of AD converters provided for each of the plurality of unit pixels 103. The pixel control unit 105 controls reading from a plurality of unit pixels 103 by using a plurality of AD converters. Specifically, the pixel control unit 105 is provided with a corresponding counter for each unit pixel. Then, the pixel control unit 105 has a circuit that uses this counter to measure the number of pulses from the buffer output that is output according to the incident photons. The count value measured by the pixel control unit 105 can be output to the outside by the signal processing circuit 106.

FIG. 2 is a diagram showing a configuration of one pixel (unit pixel 103) of the image sensor 100. The unit pixel 103 has one microlens 202. Further, the unit pixel 103 has a plurality of photoelectric conversion elements that photoelectrically convert light passing through different exit pupils of the photographing lens. For example, the unit pixel 103 has two avalanche photodiodes (hereinafter referred to as "APDs") that operate in an avalanche state, and are shown as APD203 and APD204 in FIG.

In this way, by sharing one microlens 202 with a plurality of APDs, among the light from the photographing lens reaching the image sensor 100, the light passing through different exit pupils is divided and received, and the phase difference information is received. It is possible to obtain. The unit pixel 103 may include a plurality of components described later in addition to the components shown in the figure. Further, although the unit pixel 103 is described here as having two APDs, the number of APDs may be any number as long as it is two or more.

FIG. 3 is a diagram showing a pixel configuration and a circuit configuration for reading a signal from the pixel. The unit pixel 103 in the embodiment has two APDs 203 and 204 as described above. By sharing the microlens 202, the APDs 203 and 204 receive the light from the photographing lens that passes through different exit pupils.
The quench transistors 301 and 302 operate as quench resistors when turned on, and are driven by drive pulses φQA and φQB, respectively. The switches 303 and 304 are switches for selecting which pixel to output, and are driven by drive pulses φTA and φTB, respectively. The AD converter 310 may include a comparator 305 and a counter 306. The comparator 305 can function as a pulse shaping circuit.

The APD 203, 204, the quench transistors 301, 302, and the comparator 305 are electrically connected via switches 303, 304. A voltage H _VDD is applied to the APDs 203 and 204 via the quench transistors 301 and 302, respectively. The voltage H _VDD is set so that the APDs 203 and 204 are voltages that operate in Geiger mode.

In the comparator 305, one of the output voltages V _APD of _APD 203 and 204 is used as the first input, and the predetermined comparison voltage V _comp is used as the second input. By outputting the comparison result V _out as “H” or “L”, the comparator 305 functions as a pulse shaping circuit. The counter 306 counts the output of the comparator 305, so that the signals of the APDs 203 and 204 are AD-converted and recorded as digital data.

Next, the specific operation of the pixel will be described with reference to FIG. FIG. 4 is a diagram schematically showing the operation of APD. Since the operating principles of APD203 and APD204 are the same, only the circuit that reads the signals of APD203 and APD203 will be described here. First, the quench transistor 301 is turned on, and the switch 303 is turned on to enter the read standby state.

In the state where the APD 203 is on standby, no current is flowing through the _APD 203 and the quench transistor 301, and the V _APD indicates H _VDD . That is, in the state where the APD 203 is on standby, the APD 203 is in a state in which a voltage capable of causing an avalanche phenomenon due to the incident of photons is applied to the APD 203. At this time, the comparator 305 outputs "L".

After that, when photons are incident on the APD 203, an avalanche phenomenon occurs, and a current starts to flow through the APD 203 and the quench transistor 301. When a current starts to flow in the quench transistor 301, a voltage drop occurs and the voltage of the V _APD drops. When the V _APD falls below the comparison voltage V _comp , the output V _out of the comparator 305 becomes “H”.

After that, when the V _APD drops to a voltage at which the avalanche phenomenon cannot occur, the avalanche phenomenon stops. As a result, V _APD returns to H _VDD, and the output V _out of the comparator 305 becomes “L”.

By these operations, the incident of photons is extracted as a pulse signal. When the counter 306 counts the pulse output of the comparator 305, the signal of the APD 203 is AD-converted and recorded as digital data.

The pulse signal is taken out by operating the APD204 in the same manner. Further, the comparator 305 may be of a simple inverter type as long as the V _APD can be shaped as a pulse.

In the embodiment, one AD converter is provided for each of the plurality of unit pixels. That is, a comparator 305 and a counter 306 for reading the signals of the APDs 203 and 204 are mounted on each pixel. Therefore, it is possible to read the signal at the same time for all pixels. However, since the comparator 305 and the counter 306 are provided for each pixel and are not provided for each of the APD 203 and 204, the signals of the APD 203 and 204 cannot be taken out as separate signals at the same time.

The read operation in the embodiment for dealing with this will be described with reference to FIG. FIG. 5 is a timing diagram schematically showing the driving of the image pickup device in the embodiment. It shows how the time elapses from the left to the right of the drawing. In the present embodiment, in order to read the pixel signal of one frame, the pixel control unit 105 time-divides one frame into a plurality of subframes and performs a read operation from a plurality of unit pixels 103 for each subframe. It is composed. The pixel control unit 105 reads out all the pixels at the same time for each subframe, synthesizes the information of each subframe, and generates the information of one frame.

FIG. 5 shows how the information of one frame is read out by dividing it into a plurality of subframes 1-1-1 to 1_1 and performing the reading operation 10 times. Here, in the plurality of first subframes 1_1, 1_3, 1_5, 1_7, 1_9, only APD203 is in the operating state. Therefore, the quench transistor 301 and the switch 303 are turned on by setting φQA and φTA to high levels. On the other hand, by setting φQB and φTB to a low level, the quench transistor 302 and the switch 304 are turned off, and as a result, the counter 306 counts by the number of photons incident on the APD 203.

On the other hand, in the plurality of second subframes 1_2, 1_4, 1_6, 1_8, 1_10, both APD203 and APD204 are in the operating state. Therefore, the quench transistors 301 and 302 and the switches 303 and 304 are turned on by setting all of φQA, φTA, φQB, and φTB to high levels. At this time, since both APD 203 and 204 are in the operating state, the counter 306 performs a counting operation with respect to the photon signal incident on both of the APD 203 and 204. In this way, the division of one frame into a plurality of subframes can be performed by changing the drive frequencies of φQA, φTA, φQB, and φTB.

Here, all the count values obtained in the plurality of first subframes 1_1, 1_3, 1_5, 1_7, 1_9 are combined. Then, the signals of the photons incident on the APD203 (first photoelectric conversion element) for the time of a plurality of first subframes 1_1, 1_3, 1_5, 1_7, 1_9 can be obtained. The signal generated in this way is called an "A signal" (first signal). However, in the time of one frame, the period during which the A signal obtained by the plurality of first subframes 1_1, 1_3, 1_5, 1_7, 1_9 is obtained is half of the signal obtained in one frame in this example. Therefore, when the A signal is doubled in gain, a signal close to the signal that was planned to be obtained in one frame time can be obtained. This is called an "A2 signal".

Similarly, all the count values obtained in the plurality of second subframes 1_2, 1_4, 1_6, 1_8, 1_10 are combined. Then, the signals of the photons incident on the APD203 and APD204, which are all the photoelectric conversion elements (second photoelectric conversion elements), can be obtained for the time of the plurality of second subframes 1-4, 1_4, 1_6, 1_8, 1_10. The signal generated in this way is called an "A + B signal" (second signal). However, in the time of one frame, the period during which the A + B signal obtained by the plurality of second subframes 1_2, 1-4, 1_6, 1_8, 1_10 is obtained is half of the signal obtained in one frame in this example. Therefore, when the A + B signal is doubled in gain, a signal close to the signal that was planned to be obtained in one frame time can be obtained. This is called "(A + B) 2 signals".

Here, the signal as the phase signal is acquired by subtracting the A2 signal from the (A + B) 2 signal. This signal is called a "B2 signal" (third signal). Since the (A + B) 2 signals correspond to the signals of the light that has passed through the exit pupil of the photographing lens obtained by photoelectric conversion of the APD 203 and 204 obtained in one frame time, they are used as an imaging signal for display and recording. Will be done. On the other hand, since the A2 signal and the B2 signal are signals of light that have passed through different exit pupils of the photographing lenses that have been photoelectrically converted by the APD203 and APD204, respectively, the A2 signal and the B2 signal are phase signals having phase difference information.

By performing a known correlation calculation from the A2 signal and the B2 signal, it is possible to extract the phase difference information, and it is possible to calculate the defocus amount of the photographing lens. The A2 signal and the B2 signal are information necessary for obtaining phase difference information, and are not used for display or the like. Therefore, as long as the signals required for the calculation are obtained, it is not always necessary to synthesize the signals of each subframe, the correlation calculation may be performed using the information of a single subframe, and the gain is doubled. You don't have to call it.

Similarly, in the second and subsequent frames, an imaging signal and a phase signal can be obtained by dividing the inside of one frame, reading it out, and synthesizing it.

In the present embodiment, since a SPAD type image sensor capable of photon counting is used, even if signals of a plurality of subframes are combined to form one frame, there is almost no deterioration of noise due to the combination. The above-mentioned synthesis operation, difference operation, correlation calculation, and the like may be performed by the signal processing circuit 106, or may be performed by a signal processing unit outside the image pickup device described later.

In FIG. 5, one frame is divided into 10 subframes, but the present invention is not limited to this, and one frame is divided and read, and the imaging signal and the phase signal are divided and read. It is characterized by. Further, the (A + B) 2 signals are used as information in one frame in a pseudo manner by doubling half of the information in one frame. Therefore, if the number of divisions in one frame is small, when a subject moving at high speed or the like is photographed, the deviation from the imaging signal containing all the information in one frame becomes large. Therefore, one frame may be divided into, for example, 30 or more subframes.

Further, in the above example, the gain multiplied by the A signal and the A + B signal is set to 2 as an example, but this is because the ratio of the plurality of first subframes and the plurality of second subframes among the plurality of subframes is Because it is half each. More generally, the A signal is multiplied by a gain corresponding to the proportion occupied by the plurality of first subframes in the plurality of subframes, and the A + B signal is multiplied by the plurality of second subframes in the plurality of subframes. It is good to multiply by the gain according to the ratio occupied by the frame.

As described above, even if one pixel is provided with one AD converter, that is, even if the number of AD converters is not the same as the number of photoelectric conversion elements, a high-speed read operation can be realized. it can.

<Second Embodiment>
FIG. 6 is a diagram showing a configuration of an image pickup device according to a second embodiment. In this embodiment, a general photodiode (hereinafter referred to as “PD”) that does not perform an avalanche operation is used. Similar to the APD203 and 204, the PD601 and 602 are photoelectric conversion elements that photoelectrically convert light that has passed through different exit pupils of a photographing lens. The transfer switches 603 and 604 transfer the electric charges generated by the PDs 601 and 602 to a floating diffusion unit (hereinafter referred to as "FD"), which will be described later. The FD605 has a role as a memory for temporarily storing electric charges. The reset switch 606 resets the FD605 by VDD. The AD converter 210 may include a comparator 205 and a counter 206.

Even with the configuration of FIG. 6, if the FD605 is sufficiently small, it is possible to read out the signal for the incident photon. Therefore, if the drive is performed in the same manner as shown in FIG. 5, even if one AD converter is used for one pixel, it is possible to realize a read operation in which the image pickup signal and the phase signal are individually acquired as separate signals. It will be possible. These are the same even if the photodiode is changed to a photoelectric conversion element such as a photoelectric conversion film, and therefore, the present invention does not limit the type of the photoelectric conversion element.

Further, even if the photodiode signal is a weak voltage that cannot be separated by the comparator 205, the comparator 205 is operated as a slope type AD converter by changing the V _REF over time, and the signal is transmitted by the counter 206. It is possible to count. However, in this case, since reading noise is generated, the noise increases when subframes are combined, unlike the form of photon counting that counts photons. However, even with one AD converter for one pixel, it is possible to realize a read-out operation in which the imaging signal and the phase signal are individually acquired as separate signals.

<Third Embodiment>
FIG. 7 is a timing diagram schematically showing the driving of the image pickup device according to the third embodiment. In the example of FIG. 7, only the APD 203 is in the operating state in the plurality of first subframes 1_1, 1_3, 1_5, 1_7, 1_9. Therefore, the quench transistor 301 and the switch 303 are turned on by setting φQA and φTA to a high level. On the other hand, by setting φQB and φTB to a low level, the quench transistor 302 and the switch 304 are turned off, and as a result, the counter 306 counts by the number of photons incident on the APD 203.

On the other hand, in the plurality of second subframes 1_2, 1_4, 1_6, 1_8, 1_10, only APD204 is in the operating state. Therefore, the quench transistor 301 and the switch 303 are turned off by setting φQA and φTA to the low level. On the other hand, by setting φQB and φTB to a high level, the quench transistor 302 and the switch 304 are turned on, and as a result, the counter 306 counts by the number of photons incident on the APD 204.

Here, all the count values obtained in the plurality of first subframes 1_1, 1_3, 1_5, 1_7, 1_9 are combined. Then, the signals of the photons incident on the APD203 (first photoelectric conversion element) for the time of a plurality of first subframes 1_1, 1_3, 1_5, 1_7, 1_9 can be obtained. As a result, the A signal (first signal) is generated.

Similarly, all the count values obtained in the plurality of second subframes 1_2, 1_4, 1_6, 1_8, 1_10 are combined. Then, a photon signal incident on APD204, which is one photoelectric conversion element (second photoelectric conversion element) different from APD203, can be obtained for a plurality of hours of the second subframes 1_2, 1_4, 1_6, 1_8, 1_10. As a result, the B signal (second signal) is generated.

By such driving, the A signal and the B signal can be directly obtained. The pixel control unit 105 can generate an image pickup signal (fourth signal) by adding the A signal and the B signal. Even in such a drive, as in the first embodiment, even in a configuration in which one AD converter is provided for one pixel, the imaging signal and the phase difference detection signal are individually acquired as separate signals. It is possible to realize a read operation.

<Fourth Embodiment>
FIG. 8 is a timing diagram schematically showing another example of driving the image sensor. In FIG. 8, in the first frame, similarly to FIG. 5, the subframes belonging to the plurality of first subframes and the subframes belonging to the plurality of second subframes are set to alternate. On the other hand, in the second frame (other frame), the number of the plurality of second subframes (subframes represented by A + B) is larger than that of the plurality of first subframes (subframes represented by A).

By driving in this way, it is possible to increase the signal ratio of the A + B signal in one frame by extracting only the required amount of phase difference information and devoting the remaining subframes to reading the A + B signal. As a matter of course, the gain when used for the imaging signal is not doubled, and the gain should be applied by the amount of the insufficient subframe.

FIG. 9 is a timing diagram schematically showing still another example of driving the image sensor. In FIG. 9, in the first frame, as in FIG. 8, the subframes belonging to the plurality of first subframes and the subframes belonging to the plurality of second subframes are set to alternate. On the other hand, the second frame (other frame) is composed of only a plurality of second subframes. That is, in the second frame, only the A + B signal is read out.

Note that this is an example in which the reading is changed between the first frame and the second frame, but the reading is not limited to this, and the frame that independently acquires the A signal and the A + B signal and the frame that reads only the A + B signal are All you need is. By driving in this way, it is possible to retrieve the information only in the frames that require the phase difference information, while when the phase difference information is not required, the whole signal is not the pseudo one-frame information. Can be obtained.

The ratio of the frames for reading the A signal described with reference to FIGS. 8 and 9 may be changed according to the subject brightness and the defocus amount of the photographing lens. Even in such a drive, as in the first embodiment, even in a configuration in which one AD converter is provided for one pixel, a read operation for individually acquiring the image pickup signal and the phase signal as separate signals is realized. It is possible to do.

Next, an embodiment when the above-mentioned image pickup device is applied to a digital camera which is an image pickup device will be described with reference to FIG. In FIG. 10, the image sensor unit 1005 is a unit for capturing the subject imaged by the photographing lens 1001 as an image signal, and the image sensor of FIG. 1 described above is included in the image sensor unit 1005. The photographing lens 1001 forms an optical image of the subject on the image pickup element 100 of the image pickup element unit 1005.

The lens driving device 1002 performs zoom control, focus control, aperture control, and the like on the photographing lens 1001. The mechanical shutter 1003 is controlled by the shutter drive device 1004. The image sensor unit 1005 includes the image sensor 100 as described above, and is also provided with an optical low-pass filter or the like. The signal processing unit 1006 performs various corrections, data compression, and the like on the image signal output from the image sensor unit 1005. The signal processing unit 1006 can also perform the above-mentioned A signal and A + B signal synthesis, subtraction, correlation calculation, and the like.

The timing generation unit 1007 outputs various timing signals to the image sensor 100 and the signal processing unit 1006. The control unit 1009 controls the entire image pickup apparatus. The control unit 1009 can also perform various other calculations. Memory 1008 temporarily stores image data. The recording medium I / F 1010 is an interface for recording or reading from the recording medium 1011. The recording medium 1011 is a detachable recording medium such as a semiconductor memory for recording image data. The display unit 1012 displays various information and captured images.

Next, the operation of the digital camera at the time of shooting in the above configuration will be described. When the main power supply is turned on, the power supply of the control system is turned on, and the power supply of the imaging system circuit such as the signal processing unit 1006 is further turned on. After that, when the release button (not shown) is pressed, the signal processing unit 1006 performs the correlation calculation based on the data from the image sensor.

The control unit 1009 determines whether or not the photographing lens 1001 is driven by the lens driving device 1002 based on the result of the correlation calculation and is in focus, and when it is determined that the photographing lens 1001 is not in focus, the photographing lens 1001 is again determined. Is driven to perform correlation calculation. The correlation calculation may be performed by the signal processing unit 1006 using the data from the image sensor, or by a device dedicated to the calculation (not shown).

Then, after the focusing is confirmed, the shooting operation is started. When the shooting operation is completed, the signal processing unit 1006 performs image processing on the image signal output from the image sensor 100 to generate a shot image. The captured image data is written in the memory 1008 by the control unit 1009. The signal processing unit 1006 also performs sorting processing, addition processing, selection processing thereof, and the like. The data stored in the memory 1008 is recorded on the recording medium 1011 via the recording medium I / F 1010 under the control of the control unit 1009. At this time, data may be transferred to an external device (computer or the like) via an external I / F unit (not shown), and the image may be processed by the external device.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

100: Image sensor, 104: Main control unit, 105: Pixel control unit, 106: Signal processing circuit

Claims (13)

A pixel unit in which a plurality of unit pixels are arranged in a matrix, and each of the plurality of unit pixels has a plurality of photoelectric conversion elements that photoelectrically convert light passing through different exit pupils of a photographing lens. When,
A plurality of AD converters provided for each of the plurality of unit pixels, and
A pixel control unit that controls reading from the plurality of unit pixels by using the plurality of AD converters.
It is an image sensor having
The pixel control unit
In order to read the pixel signal of the frame, the frame is time-divisioned into a plurality of subframes, and the reading operation from the plurality of unit pixels is performed for each subframe.
In the reading operation of the plurality of first subframes among the plurality of subframes, the first signal based on the signal from the first photoelectric conversion element obtained in each of the plurality of first subframes is generated.
In the reading operation of the plurality of second subframes among the plurality of subframes, the second signal based on the signal from the second photoelectric conversion element obtained by each of the plurality of second subframes is used. Generate,
An image sensor characterized by this. The image pickup device according to claim 1, wherein each of the plurality of photoelectric conversion elements is an avalanche photodiode that operates in an avalanche state. The image pickup device according to claim 1, wherein each of the plurality of photoelectric conversion elements is a photodiode. The first photoelectric conversion element is one of the plurality of photoelectric conversion elements, and the second photoelectric conversion element is all the photoelectric conversion elements of the plurality of photoelectric conversion elements. The image pickup device according to any one of claims 1 to 3. The image pickup device according to claim 4, wherein the pixel control unit generates a third signal as a phase signal by subtracting the first signal from the second signal. The first photoelectric conversion element is one photoelectric conversion element among the plurality of photoelectric conversion elements, and the second photoelectric conversion element is the first photoelectric conversion element among the plurality of photoelectric conversion elements. The image pickup device according to any one of claims 1 to 3, wherein the image pickup device is one different photoelectric conversion element. The image pickup device according to claim 6, wherein the pixel control unit generates a fourth signal as an image pickup signal by adding the first signal and the second signal. In the frame, the subframes belonging to the plurality of first subframes and the subframes belonging to the plurality of second subframes are set to alternate, and in other frames different from the frame, the said The image pickup device according to claim 4 or 5, wherein the number of the plurality of second subframes is larger than that of the plurality of first subframes. The image pickup device according to claim 8, wherein the other frame is composed of only the plurality of second subframes. The pixel control unit multiplies the first signal by a gain corresponding to the ratio occupied by the plurality of first subframes among the plurality of subframes, and the plurality of second subframes among the plurality of subframes The image pickup device according to any one of claims 1 to 9, wherein a gain corresponding to the occupied ratio is multiplied by the second signal. The image pickup device according to any one of claims 1 to 10, wherein the plurality of AD converters are included in the pixel control unit. It has a first semiconductor chip and a second semiconductor chip stacked on each other,
The image pickup device according to claim 10, wherein the pixel unit is formed on the first semiconductor chip, and the pixel control unit is formed on the second semiconductor chip. The image sensor according to any one of claims 1 to 12, and the image sensor.
A signal processing unit that generates a captured image based on the signal output from the image sensor, and
An imaging device characterized by having.

Images