JP6884590B2 - Image pickup device, control method of image pickup device, and program - Google Patents

Description

The present invention relates to an image pickup device, a control method of the image pickup device, and a program, and is particularly suitable for being used for capturing an image by using an image pickup means including pixels including a plurality of photoelectric conversion units.

There is an image pickup device including an image pickup device having a plurality of pixels having a plurality of photoelectric conversion units and having a plurality of pixels having an intra-pixel pupil division function by the plurality of photoelectric conversion units. In such an imaging device, focus detection by a phase difference detection method is performed. As an example of an image pickup device that outputs a signal that can be used in a focus detection method, there is one in which pixels having a pair of photoelectric conversion units are provided for each microlens of a microlens array arranged in two dimensions.

As a technique related to such an image pickup apparatus, there is a technique described in Patent Document 1. In the technique described in Patent Document 1, the image pickup apparatus first receives an output signal of the photoelectric conversion unit A from a pixel having a photoelectric conversion unit A and a photoelectric conversion unit B as pixels whose pupils are divided by a microlens. The image and the A + B image which is the addition signal of the photoelectric conversion unit A and the photoelectric conversion unit B are read out. Next, the image pickup apparatus subtracts the A image from the A + B image (calculates (A + B image) − (A image)) to obtain the B image which is the output signal of the photoelectric conversion unit B. Then, the image pickup apparatus performs focus detection of the phase difference detection method using the A image and the B image, and creates a subject image using the A + B image.

Japanese Unexamined Patent Publication No. 2013-106194

However, in the technique described in Patent Document 1, it is necessary to read the A image and the A + B image, and the amount of read data is doubled. Therefore, there is a problem that the read time is also doubled. As a result, there is a problem that the frame rate as a desired moving image performance may be lowered, and there may be a problem that the rolling distortion may be deteriorated due to the difference in the reading time of the pixels for each row.

On the other hand, when the desired frame rate is maintained and the A image and the A + B image are read out, the data rate (the amount of data transferred per unit time) is doubled. Therefore, there is a problem that the data rate when reading the A image and the A + B image may exceed the data rate that can be output by the image sensor.

Further, even when the A image and the A + B image are read out at a data rate that can be output by the image sensor, the signal processing unit included in the image sensor can output the data that the image sensor can output to create the subject image using the A + B image. Must be done at a rate. Therefore, there is a problem that the limit of the signal processing rate (processing amount per unit time) of the signal processing unit may be reached and the power consumption of the signal processing unit may increase.

Further, when reading the A image and the A + B image from the image sensor, the signal processing unit included in the image pickup device calculates the B image from the A + B image and the A image during the creation of the subject image using the A + B image, and the A image. And it is necessary to carry out the focus detection of the phase difference detection method using the B image. Therefore, there is a problem that the power consumption of the signal processing unit may increase.
As described above, in the conventional technique, when the focus detection of the phase difference detection method and the image creation are performed by the intra-pixel pupil division function by using the imaging means including a plurality of pixels each having a plurality of photoelectric conversion units. It is not easy to suppress the decrease in frame rate and reduce power consumption.

The present invention has been made in view of these points, and when performing focus detection and image creation using an imaging means including a plurality of pixels, each of which has a plurality of photoelectric conversion units, the frame rate is determined. The purpose is to suppress the decrease and reduce the power consumption.

The image pickup apparatus of the present invention includes an imaging means including a plurality of pixels each having a first photoelectric conversion unit and a second photoelectric conversion unit, and a first signal output from the first photoelectric conversion unit. A first reading means for reading the second signal output from the second photoelectric conversion unit from the imaging means at a first reading speed, and the first reading means read by the first reading means . first storage means for storing a signal and the second signal to the first storage medium, as a signal used to create the image captured by the pre-Symbol imaging unit, the first signal and the second second reading means for reading the signal from the first storage unit in the second read speed, as a signal used to detect the focal point, the said first signal and said second signal first a third reading means for continuously reading out from the storage means, and wherein the slow towards the second reading speed than the first readout speed.

According to the present invention, when focusing detection and image creation are performed by using an imaging means including a plurality of pixels, each of which has a plurality of photoelectric conversion units, it is possible to suppress a decrease in frame rate and reduce power consumption. It can be realized.

It is a figure which shows the structure of the image pickup apparatus. It is a figure which shows the substrate structure of the image sensor. It is a figure which shows the structure of the image pickup substrate. It is a figure which shows the circuit structure of a pixel. It is a figure which shows the schematic structure of the pixel 200. It is a figure which shows the circuit structure of the column signal processing part. It is a figure explaining the 1st example of signal processing. It is a figure which shows the 1st example of the operation timing of an image pickup apparatus. It is a figure explaining the 2nd example of signal processing. It is a figure which shows the 2nd example of the operation timing of an image pickup apparatus.

Hereinafter, embodiments will be described in detail with reference to the drawings.
(First Embodiment)
First, the first embodiment will be described.
FIG. 1 is a diagram showing an example of the configuration of the image pickup apparatus of the present embodiment. The imaging device of the present embodiment can be applied to, for example, a digital still camera, a digital video camera, an industrial camera, an in-vehicle camera, a medical camera, or the like.
The image pickup apparatus shown in FIG. 1 includes an image pickup optical system 11, an image sensor 12, a signal processing unit 13, a compression / expansion unit 14, a synchronization control unit 15, an operation unit 16, an image display unit 17, an image recording unit 18, and a vibration detection unit 19. Has.

The imaging optical system 11 includes a lens for forming a subject image, a lens driving mechanism for zooming, focusing, or vibration isolation, a mechanical shutter mechanism, an aperture mechanism, and the like. The movable portion of these is driven based on the control signal from the synchronous control unit 15.
The image sensor 12 includes, for example, an XY address type CMOS (Complementary Metal Oxide Semiconductor) sensor. The image sensor 12 performs imaging operations such as exposure, signal reading, and reset in response to a control signal from the synchronous control unit 15. The image sensor 12 has an analog-to-digital conversion circuit (hereinafter referred to as "AD conversion circuit"). The signal read from the CMOS sensor is converted to analog-digital (hereinafter referred to as "AD conversion") by an analog-to-digital conversion circuit and digitized. The digitized image signal is output to the signal processing unit 13. The pixels provided in the image pickup device 12 output a signal for first focus detection for focus detection by the phase difference detection method and a signal for a captured image for creating a captured image. In the present embodiment, the first focus detection signal corresponds to the A signal described later, and the captured image signal corresponds to the A + B signal described later. Then, the image sensor 12 obtains a second focus detection signal by using the first focus detection signal and the captured image signal output from one pixel. In the present embodiment, the second focus detection signal corresponds to a signal (B signal) obtained by subtracting the A signal from the A + B signal described later.

Under the control of the synchronous control unit 15, the signal processing unit 13 performs signal processing and detection of control information on the digitized image signal input from the image sensor 12. Signal processing includes, for example, white balance adjustment, color correction, gamma correction, and the like. The control information includes, for example, control information such as AF (Auto Focus) and AE (Auto Exposure). Then, the signal processing unit 13 outputs the signal-processed image signal and control information to the synchronous control unit 15.

In the present embodiment, the signal processing unit 13 performs the focus detection process by the phase difference detection method as the AF detection process. As a focus detection method by the phase difference detection method, for example, there are the following methods using the intra-pixel pupil division function. The signal processing unit 13 detects a waveform shift (phase difference) between the output signal of one photoelectric conversion element and the output signal of the other photoelectric conversion element of a plurality of pixels in a pixel including two photoelectric conversion elements. That is, the signal processing unit 13 detects the phase difference of the incident light with respect to the two photoelectric conversion elements whose intra-pixel pupils are divided, based on the above-mentioned two focus detection signals related to the plurality of pixels. Then, the signal processing unit 13 obtains the distance to the subject based on the detected phase difference, performs focus detection, and drives the focusing lens so that the subject is in focus. In addition, the signal processing unit 13 creates a captured image using the captured image signal obtained from the pixels in the pixel region.

The compression / decompression unit 14 operates under the control of the synchronization control unit 15. The compression / decompression unit 14 receives the image signal signal-processed by the signal processing unit 13 from the synchronization control unit 15 to perform compression coding processing, or to the coded data of the still image supplied from the synchronization control unit 15. On the other hand, decompression / decoding processing is performed. Then, the compression / decompression unit 14 outputs the compression-encoded coded data and the decompression-decoding-processed image signal to the synchronization control unit 15. Further, the compression / decompression unit 14 may execute the compression coding / decompression / decoding process of the moving image.

The synchronization control unit 15 has, for example, a microcontroller. The microcontroller has, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The synchronous control unit 15 comprehensively controls each unit of the image pickup apparatus by executing a program stored in a ROM or the like. Further, the transfer of image data and the transfer of control data between each unit are also executed via the synchronous control unit 15.

The operation unit 16 has, for example, various operation keys such as a shutter release button, a lever, a dial, and the like. The operation unit 16 outputs a control signal corresponding to an input operation by the user to the synchronous control unit 15.
The image display unit 17 includes a display device such as an LCD (Liquid Crystal Display), an interface circuit for the display device, and the like. The image display unit 17 generates a signal for displaying the image signal supplied from the synchronization control unit 15 on the display device, and supplies the generated signal to the display device to display the image.
A recording medium made of, for example, a portable semiconductor memory or the like is connected to the image recording unit 18. The image recording unit 18 receives, for example, a file of coded data (image data file) compressed and coded by the compression / decompression unit 14 from the synchronization control unit 15 and stores it in a recording medium. Further, the image recording unit 18 reads the data designated based on the control signal from the synchronization control unit 15 from the recording medium and outputs the data to the synchronization control unit 15.

The vibration detection unit 19 has, for example, a gyro sensor and an acceleration sensor. The vibration detection unit 19 detects a signal indicating the direction and speed (motion vector) of vibration with respect to the image pickup device due to camera shake or the like and outputs the signal to the synchronization control unit 15.
The synchronization control unit 15 calculates the amount of movement of the image pickup apparatus based on the motion vector and implements image vibration isolation. Image stabilization methods include optical image stabilization and electronic image stabilization.
When performing optical image stabilization, a shift lens arranged in the imaging optical system 11 is used. Optical image stabilization cancels image shake by moving the shift lens in the imaging optical system 11 so as to shoot in the direction opposite to the shake with respect to the image pickup device, based on the amount of movement calculated from the motion vector. How to do it.

When performing electronic image stabilization, the image sensor 12 is used in which the imageable area is set wider than the image pickup area of the recorded image. Electronic image stabilization is a method of canceling image shake by cutting out a shooting area in the direction opposite to the shake with respect to the image pickup device and using it as a recorded image based on the amount of movement calculated from the motion vector.
In this embodiment, either one of the image stabilizers may be used, or both image stabilizers may be used. Further, the synchronous control unit 15 can also obtain a motion vector from changes in continuously captured images instead of the direction and intensity of vibration detected by the vibration detection unit 19.

Next, an example of the basic operation of the image pickup apparatus in this embodiment will be described.
Before capturing a still image, the image signal output from the image sensor 12 is sequentially supplied to the signal processing unit 13.
The signal processing unit 13 performs signal processing on the image signal from the image sensor 12, and supplies the processed image signal as a camera-through image signal to the image display unit 17 through the synchronization control unit 15. As a result, the camera-through image supplied from the signal processing unit 13 is displayed on the image display unit 17. The user can adjust the angle of view by looking at the image displayed by the image display unit 17.

When the shutter release button of the operation unit 16 is pressed in this state, the image signal for one frame from the image sensor 12 is taken into the signal processing unit 13 under the control of the synchronization control unit 15. The signal processing unit 13 performs signal processing on the captured image signal for one frame, and supplies the processed image signal to the compression / decompression unit 14.
The compression / decompression unit 14 compresses and encodes the image signal supplied from the signal processing unit 13 to generate coded data, and supplies the generated coded data to the image recording unit 18 through the synchronization control unit 15. As a result, the captured still image data file is recorded in the image recording unit 18.

When playing back a still image data file recorded in the image recording unit 18, the synchronization control unit 15 reads the data file selected in response to the operation input from the operation unit 16 from the image recording unit 18 and compresses / decompresses the data file. It is supplied to No. 14 to execute the decompression / decoding process. The decoded image signal is supplied to the image display unit 17 via the synchronization control unit 15. As a result, the still image is displayed (reproduced) by the image display unit 17.
When recording a moving image, the compression / decompression unit 14 compresses and encodes the image signal sequentially signal-processed by the signal processing unit 13 to generate coded data, and synchronizes the coded data of the generated moving image. The data is sequentially transferred to the image recording unit 18 through the control unit 15. As a result, the captured moving image data file is recorded in the image recording unit 18.
When playing back a moving image data file recorded in the image recording unit 18, the synchronization control unit 15 reads the data file selected in response to the operation input from the operation unit 16 from the image recording unit 18, and compresses and decompresses the data file. It is supplied to No. 14 to execute the decompression / decoding process. The decoded image signal is supplied to the image display unit 17 via the synchronization control unit 15. As a result, the moving image is displayed (reproduced) by the image display unit 17.

FIG. 2 is a diagram showing an example of the substrate configuration of the image pickup device 12 of the present embodiment.
FIG. 2A shows an example of a planar configuration of each substrate, and FIG. 2B shows an example of a laminated configuration of each substrate.
The image pickup device 12 has a structure (a stacked structure) in which an image pickup board 100, a memory board 110, and a circuit board 120 are stacked in this order. The image pickup board 100, the memory board 110, and the circuit board 120 are electrically connected to each other. Circuit elements (photoelectric conversion elements, transistors, capacitances, etc.) constituting pixels are arranged on the image pickup substrate 100. A circuit element (a storage circuit such as a memory) constituting a storage unit is arranged on the memory board 110. Circuit elements (transistors, capacitances, etc.) constituting the signal preprocessing unit are arranged on the circuit board 120. In the present embodiment, for example, by using the image pickup board 100, an example of the first board having a plurality of pixels is realized. Further, for example, by using the memory substrate 110, an example of a second substrate including the first storage medium is realized. Further, for example, by using the circuit board 120, an example of a third substrate provided with an output means for outputting a signal read by the second reading means and the third reading means to the outside of the imaging means is realized. To.

In the planar configuration of FIG. 2A, the image pickup substrate 100 has a back-illuminated CMOS sensor. An imaging unit 101 composed of pixels having a photoelectric conversion element is arranged on the surface of the imaging substrate 100 on the side where the subject light is incident. A column signal processing unit 102 that performs pixel wiring and AD conversion is arranged on the surface of the image pickup substrate 100 on the memory substrate 110 side opposite to the surface on the side on which the subject light is incident. Further, the image pickup board 100 is formed with a micropad for electrically connecting to the memory control unit 111 of the memory board 110.

The memory board 110 has a memory control unit 111 and a recording unit 112. A micropad for electrically connecting to the column signal processing unit 102 is formed on the surface of the memory board 110 on the image pickup board 100 side. A micropad for electrically connecting to the bus control unit 121 is formed on the surface of the memory board 110 on the circuit board 120 side. In the present embodiment, for example, by using the recording unit 112, an example of the first storage medium is realized.
The circuit board 120 includes a bus control unit 121, an image signal preprocessing unit 122, an image signal output unit 123, an AF signal preprocessing unit 124, and an AF signal output unit 125. A micropad for electrically connecting to the memory control unit 111 is formed on the surface of the circuit board 120 on the memory board 110 side. Here, the image signal preprocessing unit 122 and the AF signal preprocessing unit 124 are combined to form the signal preprocessing unit 126. In the present embodiment, for example, an example of the output means is realized by using the image signal output unit 123 and the AF signal output unit 125.

By connecting the micro pad of the imaging board 100 and the micro pad of the memory board 110 corresponding to the micro pad with a micro bump, it is possible to transfer the signal data of the AD-converted pixel to the memory control unit 111. It becomes. Similarly, by connecting the micro pad of the memory board 110 and the micro pad of the circuit board 120 corresponding to the micro pad with a micro bump, signal data is transmitted from the memory control unit 111 to the bus control unit 121. It becomes possible to transfer.

In the example shown in FIG. 2A, the memory control unit 111 transfers the signal data of the pixels transferred from the imaging board 100 to the recording unit 112 and records the signal data, and the signal data read from the recording unit 112 is bused. It controls the operation of transferring to the control unit 121. Further, the bus control unit 121 controls an operation of distributing and transferring the signal data transferred from the memory control unit 111 to the image signal preprocessing unit 122 and the AF signal preprocessing unit 124.

When the memory control unit 111 and the recording unit 112 are formed on the surface of the memory board 110 on the image pickup board 100 side, the micropad and the memory control on the surface of the memory board 110 on the circuit board 120 side are used by using the through silicon vias. It is electrically connected to the unit 111. On the other hand, when the memory control unit 111 and the recording unit 112 are formed on the surface of the memory board 110 on the circuit board 120 side, a vertical through electrode is used to form a micropad on the surface of the memory board 110 on the image pickup board 100 side. The memory control unit 111 is electrically connected.

Similarly, when the bus control unit 121, the signal preprocessing unit 126, the image signal output unit 123, and the AF signal output unit 125 are formed on the surface of the circuit board 120 on the memory board 110 side, the memory board 110 of the circuit board 120 is formed. A pad electrode for signal output is formed on the surface opposite to the side. A signal output terminal can be provided on the lower surface of the image sensor 12 by electrically connecting the image signal output unit 123 and the AF signal output unit 125 and the signal output pad electrode using the through silicon via. .. On the other hand, when the bus control unit 121, the signal preprocessing unit 126, the image signal output unit 123, and the AF signal output unit 125 are formed on the surface of the circuit board 120 opposite to the surface on the memory substrate 110 side, the following You can do it like this. That is, using the through silicon via, the micropad on the surface of the circuit board 120 on the memory board 110 side and the bus control unit 121 are electrically connected, and signals are sent to the image signal output unit 123 and the AF signal output unit 125. A pad electrode for output may be formed.

As described above, FIG. 2B shows an example of the laminated structure of the image sensor 12. In the example shown in FIG. 2B, the image pickup device 12 is configured by stacking the image pickup board 100, the memory board 110, and the circuit board 120 in this order. Subject light (Light) is incident from the upper surface of the image sensor 12 (the surface on which the image pickup board 100 is arranged), and a signal is output from the lower surface (the surface on which the circuit board 120 is arranged). As a result, the area of the image pickup device 12 can be reduced, so that the effect of increasing the mounting density of the image pickup device can be expected. Further, the opposing micropads between the substrates may be directly connected without providing the microbumps. As a result, the thickness of the image pickup device 12 can be reduced, so that the effect of increasing the mounting density of the image pickup device can be expected.

FIG. 3 is a diagram showing an example of the configuration of the imaging board 100 of the present embodiment.
The imaging board 100 shown in FIG. 3 has a pixel region 201 composed of pixels 200, a vertical scanning unit 202, a column signal processing unit 203, a horizontal scanning unit 207, an output unit 209, and a timing unit 211.
The pixel region 201 is configured using pixels 200 having a CMOS sensor. As shown by P11 to P46, each pixel 200 is arranged in a two-dimensional matrix in the horizontal and vertical directions. In FIG. 3, the pixels in the first row are represented by P11 to P16, and the pixels in the fourth row are represented by P41 to P46. In FIG. 3, a case where the pixel arrangement in the pixel area 201 is a 6 × 4 arrangement (4 rows and 6 columns) will be described as an example. However, the pixel arrangement in the pixel region 201 is not limited to this number. In the present embodiment, for example, the pixel region 201 realizes an example of a first region corresponding to an image captured by the imaging means.
Further, in the present embodiment, it is assumed that a 2 × 2 array of color filters is arranged for the plurality of pixels 200. In the present embodiment, the 2 × 2 array is an array in which the odd-numbered rows are the repetition of the R (red) filter and the G (green) filter, and the even-numbered rows are the repetition of the G (green) filter and the B (blue) filter. It shall be.

The vertical scanning unit 202 selects the pixel array of the pixel area 201 line by line, and controls the reset operation and the read operation of the selected pixel line. The pixel control line 221 is commonly connected for each pixel line, and transmits a drive control signal for each line by the vertical scanning unit 202. The vertical signal line 231 is commonly connected for each pixel column, and the pixel signals of the line selected by the pixel control line 221 are read out to the corresponding vertical signal lines 231. The column signal processing unit 203 is provided for each corresponding vertical signal line 231 and performs signal processing described later for each row-based pixel signal transmitted through the vertical signal line 231.

The horizontal scanning unit 207 selects the column signal processing unit 203 for each column via the column selection line 251 and transmits the digitized pixel signal stored in the selected column signal processing unit 203 via the horizontal output line 261. Is transferred to the output unit 209. The output unit 209 outputs a digitized line-by-line pixel signal to the signal processing unit 13. The timing unit 211 outputs various clock signals, control signals, and the like necessary for the operation of each unit of the image pickup device 12 based on the control signal from the synchronization control unit 15. Here, the control lines 271, 281, and 285 are control lines that send clock signals, control signals, and the like from the timing unit 211 to the vertical scanning unit 202, the column signal processing unit 203, and the horizontal scanning unit 207, respectively.
In FIG. 3, a case where the timing unit 211 is arranged on the image pickup substrate 100 has been described as an example. However, it is not always necessary to do this. For example, the timing unit 211 may be arranged on the memory board 110 or the circuit board 120. Further, the timing unit 211 may be arranged on each of the image pickup board 100, the memory board 110, and the circuit board 120.

FIG. 4 is a diagram showing an example of a circuit configuration of pixels 200 of the imaging board 100 of the present embodiment.
The pixel 200 surrounded by a broken line in FIG. 4 represents one of the pixels 200 constituting the pixel region 201. Further, the pixel 200 is electrically connected to another circuit by the pixel control line 221 and the vertical signal line 231.
The vertical signal line 231 is connected to the load circuit and the column signal processing unit 203, and is also commonly connected to each pixel in the vertical column to output a pixel signal.

The pixel control line 221 is connected to the vertical scanning unit 202 and is commonly connected to the pixels in one horizontal line. By simultaneously controlling one horizontal line of pixels using the pixel control line 221, resetting and signal reading become possible. The reset control line pR, the transfer control lines pTa, pTb, and the vertical selection line pSEL are included in the pixel control line 221.

The photoelectric conversion elements D1a and D1b are photodiodes that convert light into electric charges and store the electric charges. In the present embodiment, for example, an example of a plurality of photoelectric conversion units is realized by the photoelectric conversion elements D1a and D1b. Further, for example, the photoelectric conversion element D1a realizes an example of the first photoelectric conversion unit, and the photoelectric conversion element D1b realizes an example of the second photoelectric conversion unit. The photoelectric conversion elements D1a and D1b are grounded on the P side of the PN junction, and the N side is connected to the sources of the transfer transistors (transfer switches) T1a and T1b, respectively. The gates of the transfer transistors (transfer switches) T1a and T1b are connected to the transfer control lines pTa and pTb, respectively, and the drain thereof is connected to the FD capacitance Cfd. The transfer transistors (transfer switches) T1a and T1b control the transfer of electric charges from the photoelectric conversion elements D1a and D1b to the FD capacitance Cfd, respectively.

One end of the FD (Floating Diffusion) capacitance Cfd is grounded, and the electric charge is accumulated when the electric charge transferred from the photoelectric conversion elements D1a and D1b is converted into a voltage. Here, the connection point between the drain of the transfer transistors (transfer switches) T1a and T1b and the other end of the FD capacitance Cfd is referred to as an FD node 401, if necessary.
The gate of the reset transistor (reset switch) T2 is connected to the reset control line pR, its drain is connected to the power supply voltage Vdd, and its source is connected to the FD capacitance Cfd. The reset transistor (reset switch) T2 resets the potential of the FD node 401 to the power supply voltage Vdd.

The drive transistor (amplification unit) Tdrv is a transistor that constitutes an in-pixel amplifier. The gate of the drive transistor (amplification unit) Tdrv is connected to the other end (FD node 401) of the FD capacitance Cfd, the drain is connected to the power supply voltage Vdd, and the source is the drain of the selection transistor (selection switch) T3. Be connected. The drive transistor (amplification unit) Tdrv outputs a voltage corresponding to the voltage of the FD capacitance Cfd.
The gate of the selection transistor (selection switch) T3 is connected to the vertical selection line pSEL, and its source is connected to the vertical signal line 231. The selection transistor (selection switch) T3 outputs the output of the drive transistor Tdrv as an output signal of the pixel 200 to the vertical signal line 231.

The load transistor Trod of the load circuit provided for each vertical signal line 231 has its source and gate grounded, and its drain is connected to the vertical signal line 231. The load transistor Trod constitutes a source follower circuit that serves as an intra-pixel amplifier together with the drive transistor Tdrv arranged in the pixel 200 of the row connected by the vertical signal line 231. Normally, when the signal of the pixel 200 is output, the load transistor Trod is operated as a constant current source for grounding the gate.
In the description of the present embodiment, the transistors other than the drive transistor Tdrv and the load transistor Trod act as switches, and the control line connected to the gate conducts (turns on) when it is High and shuts off (turns off) when it is Low. ).

Here, as the first exposure control, an example of the exposure control of the photoelectric conversion element D1a will be described.
It is assumed that the subject light is incident on the image sensor 12.
First, at the timing of the start of exposure, the reset transistor T2 is turned on to reset the FD node 401 side of the FD capacitance Cfd, and at the same time, the transfer transistor T1a is turned on to reset the charge of the photoelectric conversion element D1a. Then, by sequentially turning off the transfer transistor T1a and the reset transistor T2 in the order of the transfer transistor T1a and the reset transistor T2, the exposure of the photoelectric conversion element D1a is started.

Next, after the elapse of a predetermined exposure time, the reset transistor T2 is turned on to reset the FD node 401 side of the FD capacitance Cfd. Then, the reset transistor T2 is turned off. After that, the transfer transistor T1a is turned on, and the signal charge photoelectrically converted by the photoelectric conversion element D1a by exposure is transferred to the FD node 401. Then, the transfer transistor T1a is turned off. At this point, the exposure of the photoelectric conversion element D1a is completed. At this time, a signal voltage corresponding to the signal charge is generated in the FD node 401 having the FD capacitance Cfd.

Then, by turning on the selection transistor T3, a source follower circuit including the drive transistor Tdrv and the load transistor Trod is configured. As a result, the signal corresponding to the signal voltage of the FD node 401 is output to the vertical signal line 231 as the signal of the photoelectric conversion element D1a. The signal of the photoelectric conversion element D1a output to the vertical signal line 231 is input to the column signal processing unit 203, and the column signal processing described later is performed.

Next, as a second exposure control, an example of exposure control of the photoelectric conversion element D1b will be described.
Similar to the photoelectric conversion element D1a, the exposure of the photoelectric conversion element D1b is also controlled. However, in the present embodiment, the transfer transistors T1a and T1b are controlled by different transfer control lines pTa and pTb, respectively. Therefore, the reset timing of the exposure start and the transfer timing of the exposure end can be set separately by the photoelectric conversion elements D1a and D1b.

Therefore, the timing at which the exposure of the photoelectric conversion element D1b is started is set to be the same as the exposure time of the photoelectric conversion element D1a by calculating back from the timing of transferring the signal charge of the photoelectric conversion element D1b to the FD capacitance Cfd. .. Further, in the present embodiment, after the string signal processing (first exposure control) of the signal of the photoelectric conversion element D1a is performed, the signal charge of the photoelectric conversion element D1b is transferred to the FD capacitance Cfd to perform the first exposure. It is added to the signal charge of the photoelectric conversion element D1a previously transferred to the FD capacitance Cfd by control. As described above, in the present embodiment, after the exposure of the photoelectric conversion element D1a is started, the exposure of the photoelectric conversion element D1b is started during the exposure of the photoelectric conversion element D1a, and the exposure times thereof are set. Make sure they are the same.

To explain a specific example of the second exposure control, first, the transfer transistor T1b is turned on in a state where the signal charge of the photoelectric conversion element D1a is accumulated in the FD node 401 of the FD capacitance Cfd. As a result, the signal charge photoelectrically converted by the photoelectric conversion element D1b by the exposure is transferred to the FD node 401. Then, the transfer transistor T1b is turned off. At this point, the exposure of the photoelectric conversion element D1b is completed.

At this time, a signal voltage corresponding to the signal charge to which the signal charges of the photoelectric conversion elements D1a and D1b are added is generated in the FD node 401 having the FD capacitance Cfd. Then, by turning on the selection transistor T3, a source follower circuit including the drive transistor Tdrv and the load transistor Trod is configured. As a result, the signal corresponding to the signal voltage of the FD node 401 is output to the vertical signal line 231 as an addition signal of the photoelectric conversion elements D1a and D1b. The addition signals of the photoelectric conversion elements D1a and D1b output to the vertical signal line 231 are input to the column signal processing unit 203, and the column signal processing described later is performed.

Here, when the exposure control of the photoelectric conversion elements D1a and D1b is performed, the exposure start reset timing may be set simultaneously by the photoelectric conversion elements D1a and D1b, and the exposure to each of the photoelectric conversion elements D1a and D1b may be started at the same time. .. Then, at the timing of transferring the signal charge of the photoelectric conversion element D1b, the signal charge of the photoelectric conversion element D1a is transferred for the second time. As a result, with respect to the addition signals of the photoelectric conversion elements D1a and D1b, it is possible to perform shooting in which the deviation of the exposure time is eliminated.
Here, the output signal of the photoelectric conversion element D1a is referred to as an A signal for focus detection or simply an A signal, and the output signal to which the signal charges of the photoelectric conversion elements D1a and D1b are added is an A + B signal for a captured image or simply an A + B signal. It will be referred to as. Further, a signal obtained by subtracting the focus detection A signal from the captured image A + B signal is referred to as a focus detection B signal or simply a B signal. The B signal for focus detection is a signal corresponding to the output signal of the photoelectric conversion element D1b.

FIG. 5 is a diagram showing an example of a schematic configuration of pixels 200 of the image pickup substrate 100 of the present embodiment.
5 (a) shows an example of a plan view in which pixels 200 are arranged in 2 × 2, and FIG. 5 (b) shows a cross-sectional view of xx ′ of FIG. 5 (a).
The regions 501a and 501b of the photoelectric conversion elements D1a and D1b correspond to the N side of the PN junction of the photoelectric conversion elements D1a and D1b, respectively, and the imaging board 100 corresponds to the P side. The position 502 indicates the position of the circuit portion other than the photoelectric conversion elements D1a and D1b of the pixel 200. In FIG. 5, for convenience of notation, the pixel control line 221 and the vertical signal line 231 are not shown.

As described above, the microlens 503 is provided for each pixel 200. In the present embodiment, the microlens 503 is arranged so as to evenly cover both the photoelectric conversion elements D1a and D1b so as to be offset below the center of the pixel 200 in FIG. 5A. A color filter 504 is also provided for each pixel 200. The color filter 504 evenly covers both the photoelectric conversion elements D1a and D1b. As described above, any one of the R (red) filter, the G (green) filter, and the B (blue) filter is arranged for each pixel 200.
As shown in FIG. 5, in the present embodiment, one microlens 503 is shared by two photoelectric conversion elements D1a and D1b. Therefore, focus detection can be performed based on the image signal (A signal for focus detection) obtained from the photoelectric conversion element D1a and the image signal (B signal for focus detection) obtained from the photoelectric conversion element D1b.

FIG. 6 is a diagram showing an example of the circuit configuration of the column signal processing unit 203 of the imaging board 100 of the present embodiment.
The sample hold circuit (S / H) 611 is electrically connected to the signal selection control line pSH1. The sample hold circuit (S / H) 611 holds the pixel signal received from the vertical signal line 231 under the control of the timing unit 211 via the signal selection control line pSH1.

The AD conversion circuit (AD) 621 is connected to the AD control line pAD1. The AD conversion circuit (AD) 621 performs AD conversion of the pixel signal held by the sample hold circuit (S / H) 611 by control from the timing unit 211 via the AD control line pAD1. The memory circuit 631 is connected to the memory control line pMEM1 and stores the digital signal of the pixel output by the AD conversion circuit 621 under control via the memory control line pMEM1.
Further, the memory circuit 631 outputs the digitized pixel signal stored in the memory circuit 631 to the digital output line DSig1 by the control via the horizontal selection line pH1.

The digital output line DSig1 is also commonly connected to the memory circuit 631 of the other column signal processing unit 203. Each column signal processing unit 203 digitally outputs a digitized pixel signal stored in the memory circuit 631 by controlling the horizontal scanning unit 207 via the horizontal selection line pH 1 corresponding to the column signal processing unit 203. Output to line DSig1 in a predetermined order.
Here, the control line 281 for controlling the column signal processing unit 203 from the timing unit 211 includes a signal selection control line pSH1, an AD control line pAD1, and a memory control line pMEM1. The column selection line 251 for selecting the column signal processing unit 203 from the horizontal scanning unit 207 includes the horizontal selection line pH1. The horizontal output line 261 connected to the output unit 209 includes a digital output line DSig1.
As described above, the column signal processing unit 203 shown in FIG. 6 has a circuit configuration capable of AD conversion.

Next, an example of the first read operation and the second read operation of the image pickup substrate 100 of the present embodiment will be described.
In the first read-out operation, column signal processing is performed on the signal read from the photoelectric conversion element D1a and the signal of the pixel 200 for one row for which the first exposure control is performed.
The signal of the photoelectric conversion element D1a output to the vertical signal line 231 is input to the column signal processing unit 203, and is a sample of the column signal processing unit 203 under the control of the signal selection control line pSH1 input to the column signal processing unit 203. It is held by the hold circuit (S / H) 611.

Then, the signal held in the sample hold circuit (S / H) 611 is converted into a digital signal by the AD conversion circuit (AD) 621 of the column signal processing unit 203, and stored in the memory circuit 631 of the column signal processing unit 203. To.
The digital signal of the photoelectric conversion element D1a stored in the memory circuit 631 is read out from the memory circuit 631 under the control of the horizontal scanning unit 207 via the horizontal selection line pH 1 with respect to the column signal processing unit 203. The digital signal of the photoelectric conversion element D1a read from the memory circuit 631 is output to the digital output line DSig1.
Then, the digital signal of the photoelectric conversion element D1a is transferred as an A signal to the memory board 110 via the output unit 209.

In the second reading operation, column signal processing is performed on the signals read from the pixels 200 for one row for which the second exposure control is performed (that is, the addition signals of the photoelectric conversion elements D1a and D1b).
The addition signal of the photoelectric conversion elements D1a and D1b is converted into a digital signal by the AD conversion circuit (AD) 621 of the column signal processing unit 203, and is converted into a digital signal as the digital addition signal of the photoelectric conversion elements D1a and D1b. It is stored in the memory circuit 631 of. Since the column signal processing for the addition signals of the photoelectric conversion elements D1a and D1b and the output to the digital output line DSig1 are the same as the first read operation, detailed description thereof will be omitted.

The digital addition signals of the photoelectric conversion elements D1a and D1b are transferred as A + B signals to the memory board 110 via the output unit 209.
As a result, the A signal and the A + B signal are read from the pixels for one line and transferred to the memory board 110.
By performing the first read operation and the second read operation row by row on the pixel 200 of the pixel area 201, one shooting operation can be performed.

In FIG. 6, a case where the A signal and the A + B signal of the pixel 200 are transferred to the memory board 110 via the output unit 209 is shown as an example. However, instead of outputting the A signal and the A + B signal to the digital output line DSig1, the following may be used. That is, a vertical through electrode may be provided for each output of the memory circuit 631 to directly connect the memory control unit 111 of the memory board 110 and the memory circuit 631.
In the present embodiment, as described above, the A signal or the A + B signal of the pixel 200 can be transferred to the memory board 110 at the same time for one line, so that the transfer time can be shortened.

FIG. 7 is a diagram illustrating an example of signal processing of the present embodiment. FIG. 7A shows an A signal and an A + B signal read from the pixel 200 of the pixel region 201 in the imaging unit 101. In the present embodiment, the A signal or the A + B signal is read out from the pixel 200 of the pixel area 201 in a predetermined order for each row. In addition, in FIG. 3, the case where the pixel arrangement is a 6 × 4 arrangement (4 rows and 6 columns) has been described as an example, but in FIG. 7 (a), the pixel arrangement is a 10 × 8 arrangement (8 rows and 10 columns). ) Will be described as an example.

In FIG. 7, each signal is shown in parentheses, and a color filter is shown before it. As described above, in the present embodiment, the odd-numbered rows are the repetition of the R (red) filter and the G (green) filter, and the even-numbered rows are the repetition of the G (green) filter and the B (blue) filter. Use an array color filter. Therefore, the signal (A signal and A + B signal) of the pixel 200 corresponding to the R (red) filter is referred to as R. Further, the signal (A signal and A + B signal) of the pixel 200 corresponding to the G (green) filter is referred to as G. Further, the signal (A signal and A + B signal) of the pixel 200 corresponding to B (blue) is referred to as B. For example, the signal 701 is the A signal of the pixel 200 corresponding to the R (red) filter in the first row and the first column of the pixel area 201, and is referred to as R (A). The signal 702 is an A + B signal of the pixel 200 corresponding to the R (red) filter in the first row and the first column of the pixel area 201, and is expressed as R (A + B). Further, the area 703 surrounding the diagonal pixels 200 in the 2nd row and 2nd column to the pixel 200 in the 7th row and 9th column is defined as a focus detection area.

FIG. 7B shows the A + B signal of the pixel 200 used for creating the captured image in the signal processing unit 13 in FIG. 7A.
In the present embodiment, when creating the captured image, the signal processing unit 13 uses the A + B signals in all the pixels 200 of the pixel area 201. For example, for the pixel 200 in the first row and the first column, the signal processing unit 13 uses a signal 702 (R (A + B)) which is an A + B signal. At this time, the signal processing unit 13 does not use the signal 701 (R (A)) which is the A signal.

FIG. 7 (c) shows the signal of the pixel 200 involved in the focus detection process (AF process) in the signal processing unit 13 in FIG. 7 (a).
In the present embodiment, a case where the signal processing unit 13 uses the signal of the pixel 200 corresponding to the even-numbered row G (green) filter in the focus detection area 703 during the focus detection process (AF process) will be described as an example. To do. Therefore, as shown in FIG. 7C, the pixels 200 involved in the focus detection process (AF process) are the third column, the fifth column, and the seventh column, respectively, in the second row, the fourth row, and the sixth row. The pixel 200 corresponds to the G (green) filter in the ninth column.

Further, in the present embodiment, the signal processing unit 13 performs focus detection of a phase difference detection method based on the A signal obtained from the photoelectric conversion element D1a and the signal (B signal) obtained from the photoelectric conversion element D1b. .. Therefore, the signal processing unit 13 needs to calculate the B signal (by calculating (A + B signal) − (A signal)) by subtracting the value of the A signal from the value of the A + B signal. Therefore, the A signal and the A + B signal read from the pixel 200 are required. Specifically, as shown in FIG. 7C, the signal processing unit 13 has G (A) and G (A + B) from the pixel 200 in the second row and the third column to the pixel 200 in the sixth row and the ninth column. Signal will be used. For example, the signal processing unit 13 subtracts the A signal signal 704 (G (A)) from the A + B signal signal 705 (G (A + B)) for the pixel 200 in the second row and third column (G (A + B)). ) -G (A) = G (B)) B signal may be obtained. In the present embodiment, for example, an example of the first signal is realized by the A signal, and an example of the second signal is realized by the A + B signal.
From the above, in the present embodiment, the column signal processing unit 102 reads the A signal and the A + B signal from the pixels of the focus detection area 703 of the pixel area 201 during the period of the imaging operation 801 and reads the A signal and the A + B signal of the pixel area 201. The A + B signal is read from the other pixels.

FIG. 8 is a diagram showing an example of the operation timing of the image pickup apparatus of the present embodiment. 8 (a) shows an image pickup operation and a moving image display, FIG. 8 (b) shows an image processing operation, and FIG. 8 (c) shows an AF processing operation.

In FIG. 8A, the operating state of the imaging unit 101 is referred to as an imaging operation 801 and the operation of displaying the image-processed captured image on the image display unit 17 is referred to as a moving image display 802. The L in the vertical direction of the graph of FIG. 8A indicates the order of the lines for reading the captured image from the imaging unit 101 and the order of the rows for displaying the image-processed captured image as a moving image. T in the horizontal direction indicates the time lapse of the timing of reading the line and the timing of the line displaying the moving image. In FIG. 8A, by defining the vertical and horizontal axes in this way, the state of the line read from the imaging unit 101 and the state of the line displaying the moving image are the imaging operation 801 and the moving image display, respectively. It is represented by 802.

In the example shown in FIG. 8A, the tilt of the imaging operation 801 is larger than the tilt of the moving image display 802. Therefore, as the operation speed per unit time, the image pickup operation 801 is faster than the moving image display 802.
Here, the amount of data processing per unit time of the imaging operation 801 from the start to the completion of reading from the imaging unit 101 is referred to as an imaging rate, if necessary. Further, the amount of data processing per unit time of the moving image display 802 from the start to the completion of the processing for displaying the image-processed captured image is referred to as a moving image rate, if necessary.

In the image processing operation shown in FIG. 8B, items to be executed mainly as operations related to image processing are shown in order from the top in the order of execution. In FIG. 8B, the horizontal direction is a period corresponding to FIG. 8A, and indicates an operation period of each item. In the present embodiment, the column signal processing unit 102 reads the A signal and the A + B signal from the pixel 200 of the imaging unit 101 during the imaging operation 801. The signal of the pixel 200 read out at this time is the signal of the pixel 200 of the pixel region 201 described with reference to FIG. 7A.
Therefore, in FIG. 8B, the period of the imaging operation 801 is defined as the reading period of the A signal and the A + B signal, and is expressed as “A / A + B reading”. In the present embodiment, for example, the column signal processing unit 203 reads the A signal and the A + B signal from the pixels 200 of the image pickup unit 101 during the period of the image pickup operation 801 to realize an example of the first reading means. .. Further, for example, an example of the first read speed is realized by the read speed corresponding to the image pickup rate.

At the same time as the A signal and A + B signal reading operation, the column signal processing unit 102 transfers the A signal and the A + B signal from the image pickup board 100 to the memory control unit 111 of the memory board 110. Then, the memory control unit 111 records the A signal and the A + B signal in the recording unit 112. In FIG. 8B, the period of these operations is referred to as “memory writing”. In the present embodiment, for example, the memory control unit 111 records the A signal and the A + B signal in the recording unit 112, so that an example of the first storage means is realized. Further, an example of the second writing speed is realized by the writing speed (writing speed corresponding to the imaging rate) when the memory control unit 111 records the A signal and the A + B signal in the recording unit 112.

Here, in FIG. 8B, the period required for signal processing on the captured image by the signal processing unit 13 is referred to as “image processing”. The period of image processing is determined from the operating specifications of the image pickup apparatus based on the processing capacity of the signal processing unit 13, the magnitude of power consumption, and the amount of heat generated. Further, the amount of data processing per unit time required for signal processing on the captured image by the signal processing unit 13 is referred to as a processing rate, if necessary.

In the present embodiment, in order to read the signal of the pixel 200 from the image pickup unit 101 at high speed, a case where the processing rate, which is the processing capacity of the signal processing unit 13, is lower than the image pickup rate will be described.
Therefore, the memory control unit 111 reads out the signals of all the pixels 200 of the pixel area 201 recorded in the recording unit 112 of the memory board 110 at a processing rate lower than the imaging rate, and reads the signals from the memory board 110 to the circuit board 120. Transfer to the bus control unit 121. In FIG. 8B, this period is referred to as “memory read”.
In the present embodiment, as described above, conversion of the amount of data processing per unit time (rate conversion) from a high-speed imaging rate to a low-speed processing rate is performed using the recording unit 112.

The signal of the pixel 200 transferred to the circuit board 120 at the processing rate is subjected to necessary signal preprocessing in the image signal preprocessing unit 122, and the signal processing unit from the image sensor 12 via the image signal output unit 123. It is output to 13.
Here, the A + B signal, which is an addition signal of the photoelectric conversion elements D1a and D1b of the pixel 200, is used to create the captured image in the signal processing unit 13. Therefore, as described in FIG. 7B, the signal of the pixel 200 read from the recording unit 112 may be only the A + B signal. This eliminates the need to output the two signals, the A signal and the A + B signal shown in FIG. 7B, from the image sensor 12 to the signal processing unit 13. Therefore, it is possible to reduce the data transfer capacity when the image sensor 12 outputs the data to the signal processing unit 13.

Further, the signal preprocessing performed by the image signal preprocessing unit 122 includes scratch correction, noise reduction processing, and the like. However, since the A + B signal is a signal capable of creating a captured image, the image signal output unit 123 may output the A + B signal to the signal processing unit 13 as it is.
In the present embodiment, for example, the memory control unit 111 reads the signals of all the pixels 200 of the pixel area 201 recorded in the recording unit 112 of the memory board 110 at the processing rate, so that the second reading means is used. An example is realized. Further, for example, an example of the second read speed is realized by the speed corresponding to the processing rate. Further, for example, a signal of the pixel stored by the first storage means by the A + B signals of all the pixels 200 of the pixel area 201, which is a signal used to create an image captured by the imaging means. An example is realized.

The signal of the captured image processed at the processing rate by the signal processing unit 13 is temporarily stored in a buffer (not shown) in the signal processing unit 13 or the synchronization control unit 15. In FIG. 8B, this period is referred to as “buffer write”.
Then, the signal processing unit 13 or the synchronization control unit 15 reads the signal of the captured image processed during the "image processing" period from the buffer and transfers it to the image display unit 17. In FIG. 8B, this period is referred to as “buffer read”.
The signal of the captured image read from the buffer is displayed on the image display unit 17 during the period of the moving image display 802. Therefore, as shown in FIGS. 8A and 8B, the period of "buffer read" is the same as the period of the moving image display 802.

At this time, if the image processing is slower than the image display, a display problem occurs, so the processing rate must be higher than the moving image rate. Therefore, in FIG. 8B, the period of "buffer writing", which is the same as the period of "image processing", is shorter than the period of "buffer reading".
In the present embodiment, for example, the signal processing unit 13 or the synchronization control unit 15 stores the signal of the image-processed captured image in the buffer, thereby realizing an example of the second storage means. Further, for example, an example of the first writing speed is realized by the speed at which the signal processing unit 13 or the synchronization control unit 15 writes the signal of the image-processed captured image to the buffer (writing speed corresponding to the processing rate). .. Further, for example, an example of a second storage medium is realized by a buffer. Further, for example, when the signal processing unit 13 or the synchronization control unit 15 reads the signal of the image-processed captured image from the buffer, an example of the fourth reading means is realized. Further, for example, an example of the third reading speed is realized by the speed at which the signal processing unit 13 or the synchronization control unit 15 reads the signal of the image-processed captured image from the buffer (reading speed corresponding to the moving image rate). ..

In the present embodiment, as described above, rate conversion from a high-speed processing rate to a low-speed moving image rate is performed using the buffer of the signal processing unit 13 or the synchronization control unit 15.
Further, the created captured image can be recorded in the image recording unit 18 according to the instruction of the synchronization control unit 15.

Here, in the description of FIG. 8B, it is assumed that the signal processing unit 13 or the synchronization control unit 15 has a buffer, but the image display unit 17 may be provided with a buffer. In this case, the signal of the image-processed captured image is transferred to the buffer in the image display unit 17 at the processing rate, read out at the moving image rate, and displayed on the image display unit 17.

In the AF processing operation shown in FIG. 8C, items that are mainly performed as operations related to the focus detection processing (AF processing) are shown in order from the top. In FIG. 8C, the horizontal direction is a period corresponding to FIG. 8A, and indicates an operation period of each item.
The “memory writing” to the recording unit 112, which is performed at the same time as the “A / A + B reading” and the “A / A + B reading”, which is the imaging operation 801 is the same item as that shown in FIG. 8 (b). Therefore, detailed description thereof will be omitted here.

As described with reference to FIG. 7A, the focus detection region 703 is provided in the pixel region 201 in the imaging unit 101. The signal processing unit 13 performs the focus detection process based on the A signal and the A + B signal obtained from the pixels 200 in the focus detection region 703.
At this time, the pixels 200 used for focus detection do not have to be all the pixels in the focus detection region 703. In the present embodiment, as described with reference to FIG. 7C, the signal processing unit 13 uses pixels 200 corresponding to the even-numbered rows of G (green) filters in the focus detection region 703 during the focus detection process.

Here, in order to show that the signal of the pixel 200 corresponding to the even-numbered G (green) filter in the focus detection area 703 is at the position of the discrete pixel 200, the time interval is set in FIG. 8 (c). One open rectangle is represented as a signal of one pixel 200. Further, in FIG. 8C, the signal (rectangle) of the pixel 200 corresponding to the G (green) filter in the even number of rows in the focus detection area 703 is represented as “AF information on the memory”. In the following description, the pixel 200 used for the focus detection process (in the present embodiment, the pixel 200 corresponding to the even-numbered G (green) filter in the focus detection area 703) is referred to as a focus detection pixel, if necessary. .. In the present embodiment, for example, the focus detection region 703 realizes an example of a second region that is narrower than the first region and includes pixels that output signals used for detecting the focus. Will be done. Further, for example, the focus detection pixel realizes an example of a pixel that outputs a signal used for detecting the focus.

It can be seen that in the "A / A + B read" operation, it is necessary to read all the pixels 200 in the focus detection area 703 in order to read the focus detection pixels. Therefore, when the focus detection process is performed at the same time as this reading using the signals of the focus detection pixels sequentially read from the focus detection area 703, the focus detection process is performed over a period of reading all the pixels 200 in the focus detection area 703. The circuit involved in must continue to operate.
However, in the present embodiment, the memory control unit 111 continuously reads only the signal of the focus detection pixel recorded in the recording unit 112 of the memory board 110 from the memory board 110 as shown in FIG. 7C. Transfer to the circuit board 120. At this time, the signals of the focus detection pixels read from the recording unit 112 are the A signal and the A + B signal. In FIG. 8C, this period is referred to as “continuous reading”. In the present embodiment, for example, the memory control unit 111 continuously reads only the signal of the focus detection pixel recorded in the recording unit 112 of the memory board 110, thereby realizing an example of the third reading means. .. Further, for example, the A signal and the A + B signal of the focus detection pixel realize an example of a signal used for detecting the focus, which is a part of the signal of the pixel stored by the first storage means. ..

The signal of the focus detection pixel transferred to the circuit board 120 is subjected to necessary signal preprocessing in the AF signal preprocessing unit 124, and is transmitted from the image sensor 12 to the signal processing unit 13 via the AF signal output unit 125. It is output. For the signal preprocessing performed by the AF signal preprocessing unit 124, for example, the calculation formula “(A + B signal) − (A signal)” for calculating the B signal is required. The AF signal preprocessing unit 124 may perform scratch correction, noise reduction processing, and the like before executing this calculation formula. As a result, the AF signal preprocessing unit 124 outputs the A signal and the B signal as the signals of the focus detection pixels. In FIG. 8C, this period is referred to as “A / B processing”.

Then, the focus detection processing unit (not shown) included in the signal processing unit 13 is a different photoelectric conversion element Ta1 whose intra-pixel pupil is divided based on the A signal and the B signal for focus detection relating to a plurality of pixels in the horizontal direction. The phase difference of the incident light with respect to Ta2 is detected.
Next, the focus detection processing unit calculates an AF evaluation value, which is an evaluation value corresponding to the distance information from the image pickup apparatus to the subject, and is an evaluation value used for AF, based on the detected phase difference, and synchronizes. Output to the control unit 15. In the present embodiment, for example, an example of the detection means is realized by using the focus detection processing unit.

Up to this point, the focus detection process executed by the signal processing unit 13 is performed, and in FIG. 8C, this period is referred to as “AF evaluation value calculation”.
Based on the AF evaluation value, the synchronization control unit 15 drives the focusing lens included in the imaging optical system 11 to control the subject so that it is in focus. In FIG. 8C, shifting to control by the synchronous control unit 15 is referred to as “AF control”.

The focus detection processing unit of the signal processing unit 13 continuously performs focus detection processing based on the signals of the focus detection pixels that have been “continuously read”. Here, the amount of data processing per unit time required for the focus detection process of the focus detection pixel is referred to as an AF rate, if necessary.
In the AF processing operation of FIG. 8C, the AF signal preprocessing unit 124 converts the A signal and the A + B signal of the focus detection pixel continuously read from the recording unit 112 at the AF rate into the A signal of the focus detection pixel. And convert to B signal. Then, the focus detection processing unit of the signal processing unit 13 executes the focus detection processing at the AF rate, calculates the AF evaluation value, and outputs it to the synchronization control unit 15. Therefore, in FIG. 8C, the period of "continuous reading", the period of "A / B processing", and the period of "AF evaluation value calculation" are the same period. Therefore, at this time, rate conversion from a low-speed imaging rate to a high-speed AF rate is performed using the recording unit 112.

As described above, in the present embodiment, it is possible to simultaneously acquire the focus detection signal and the captured image signal in one imaging operation. Further, in the imaging operation, the processing rate for image processing of the captured image is slower than the imaging rate for reading the signal of the pixel 200 of the imaging unit 101 and transferring it to the recording unit 112. Therefore, the period of image processing can be determined based on the processing capacity of the signal processing unit 13, the magnitude of power consumption, and the amount of heat generated. Further, only the signals of the focus detection pixels in the focus detection region 703 are continuously read out. Therefore, in the imaging operation, the focus detection processing unit, the AF signal preprocessing unit 124, and the AF of the signal processing unit 13 are compared with the case where the focus detection processing is performed simultaneously while reading all the pixels in the focus detection area 703. The operating period of the signal output unit 125 can be shortened. Therefore, the power consumption of the image pickup apparatus can be reduced. Further, since the transfer period of the signal of the focus detection pixel from the image sensor 12 to the signal processing unit 13 is shortened, the time required for the focus detection process can be significantly shortened. Further, since the signal for the captured image output from the image sensor 12 to the signal processing unit 13 is only the A + B signal, the processing rate can be further reduced and the data transfer capacity can be reduced. Further, by recording the signal of the pixel of the imaging unit 101 in the recording unit 112, the restriction of the processing rate required for the image processing is eliminated, and the imaging operation can be performed at high speed, so that the rolling distortion can be reduced. As described above, in the present embodiment, when the image pickup apparatus including the pixel having the intra-pixel pupil division function performs the focus detection of the phase difference detection method and the creation of the subject image, the reduction in the frame rate is suppressed and the power consumption is reduced. Reduction and can be possible.

In the present embodiment, a case where a signal is read from the recording unit 112 at a processing rate, image processing is performed, and a signal is read from a buffer at a moving image rate has been described as an example. However, it is not always necessary to do this. For example, a signal may be read from the recording unit 112 at a moving image rate, image processing may be performed, and the signal may be displayed on the image display unit 17. By doing so, the image processing can be performed at a moving image rate slower than the processing rate, so that the data transfer rate can be further reduced. It is also possible to omit the buffer for displaying an image. In the present embodiment, for example, an example of a second read-out speed (a speed at which an image based on a signal stored by the first storage means is output to its output destination) is realized by a moving image rate. The image display unit 17 is an example of an output destination. However, the output destination of the image-processed signal is not limited to the image display unit 17. For example, the output destination of the image-processed signal may be the image recording unit 18.

Further, as in the present embodiment, the A signal and the A + B signal are read from the pixels of the focus detection area 703 of the pixel area 201, and the A + B signal is read from the other pixels of the pixel area 201. It is preferable because the signal can be reduced. However, it is not always necessary to do this. For example, the A signal and the A + B signal may be read out from all the pixels in the pixel area 201.

(Second Embodiment)
Next, a second embodiment will be described with reference to FIGS. 9 to 10 in addition to FIGS. 1 to 6. In the first embodiment, the A signal and the A + B signal (the focus detection signal of one of the two photoelectric conversion elements and the captured image signal) are read from the pixel 200 of the imaging unit 101, and the phase difference detection method is used. The case where the focus detection and the creation of the captured image are performed at the same time has been described as an example. On the other hand, in the present embodiment, the A signal and the B signal (signals for focusing detection of both of the two photoelectric conversion elements) are read from the pixel 200 of the imaging unit 101, and the focus detection and imaging of the phase difference detection method are performed. An example will be described in which the image is created at the same time. As described above, the processing due to the difference in the signal read from the pixel 200 of the imaging unit 101 is mainly different between the present embodiment and the first embodiment. Therefore, in the description of the present embodiment, detailed description of the same parts as those of the first embodiment will be omitted by adding the same reference numerals as those given in FIGS. 1 to 8.

In the present embodiment, after performing the same first exposure control (exposure control of the photoelectric conversion element D1a) as in the first embodiment, a second exposure control (photoelectric conversion element D1b) different from the first embodiment is performed. Exposure control) is carried out.
In the second exposure control of the present embodiment, after the lapse of a predetermined exposure time, only the signal charge of the photoelectric conversion element D1b is read out by turning on and off the reset transistor T2 and then turning on and off the transfer transistor T1b. Is done. The output signal of the photoelectric conversion element D1b at this time is referred to as a B signal for focus detection. Further, in the second reading operation of the present embodiment, column signal processing is performed on the signal of the photoelectric conversion element D1b read from the pixel 200 of one row for which the second exposure control is performed, that is, the B signal. ..

FIG. 9 is a diagram illustrating an example of signal processing of the present embodiment. FIG. 9A shows an A signal and a B signal read from the pixel 200 of the pixel region 201 in the imaging unit 101. In the present embodiment, the A signal or the B signal is read out from the pixel 200 of the pixel area 201 in a predetermined order for each row.
For example, the signal 901 is the A signal of the pixel 200 corresponding to the R (red) filter in the first row and the first column of the pixel area 201, and is represented as R (A). The signal 902 is a B signal of the pixel 200 corresponding to the R (red) filter in the first row and the first column of the pixel area 201, and is represented as R (B). Further, the area 903 surrounding the diagonal pixels 200 in the 2nd row and 2nd column to the pixel 200 in the 7th row and 9th column is defined as a focus detection area. In FIG. 9A, since the A + B signal is only replaced with the B signal in FIG. 7A, the detailed description of FIG. 9A will be omitted.

9 (b) shows the A signal and the B signal of the pixel 200 used for creating the captured image in the signal processing unit 13 in FIG. 9 (a).
In the present embodiment, when creating the captured image, the signal processing unit 13 uses all the signals of all the pixels 200 of the pixel area 201. That is, the signal processing unit 13 creates a captured image using the A + B signal which is the addition signal of the photoelectric conversion elements D1a and D1b. Therefore, the signal processing unit 13 adds the A signal and the B signal to calculate the A + B signal (by executing the calculation formula of (A signal) + (B signal)), so that the A signal in each pixel 200 is calculated. And B signal is required. For example, the signal processing unit 13 adds (R) the signal 901 (R (A)) which is the A signal and the signal 902 (R (B)) which is the B signal for the pixel 200 in the first row and the first column. The A + B signal may be obtained (as (A) + R (B) = R (A + B)).

FIG. 9 (c) shows the signal of the pixel 200 involved in the focus detection process (AF process) in the signal processing unit 13 in FIG. 9 (a).
In the present embodiment as well, as in the first embodiment, during the focus detection process (AF process), the signal processing unit 13 signals the pixels 200 corresponding to the even-numbered G (green) filter in the focus detection area 903. Will be described by taking as an example the case of using. Since FIG. 9 (c) merely replaces the A + B signal with the B signal in FIG. 7 (c), the detailed description of FIG. 9 (c) will be omitted.

In the present embodiment, the signal processing unit 13 carries out the focus detection of the phase difference detection method based on the A signal obtained from the photoelectric conversion element D1a and the B signal obtained from the photoelectric conversion element D1b. For example, in FIG. 9C, the signal processing unit 13 has a signal 904 (G (A)) which is an A signal and a signal 905 (G (B)) which is a B signal for the pixels in the second row and the third column. Focus detection of the phase difference detection method is performed using the signal as it is. In the present embodiment, for example, an example of the first signal is realized by the A signal, and an example of the second signal is realized by the B signal.

FIG. 10 is a diagram showing an example of the operation timing of the image pickup apparatus of the present embodiment. FIG. 10A shows an imaging operation and a moving image display, FIG. 10B shows an image processing operation, and FIG. 10C shows an AF processing operation. Since FIG. 10 (a) is the same as the imaging operation and the moving image display shown in FIG. 8 (a), the detailed description of FIG. 10 (a) will be omitted.

In the image processing operation shown in FIG. 10B, items to be executed mainly as operations related to image processing are shown in order from the top in the order of execution. In FIG. 10B, the horizontal direction is a period corresponding to FIG. 10A, and indicates an operation period of each item. In the present embodiment, the A signal and the B signal are read out from the pixel 200 of the imaging unit 101 during the imaging operation 801. The signal of the pixel 200 read out at this time is a signal of all the pixels 200 of the pixel area 201 described with reference to FIG. 9A.
Therefore, in FIG. 10B, the period of the imaging operation 801 is defined as the reading period of the A signal and the B signal, and is expressed as “A / B reading”. In the present embodiment, for example, the column signal processing unit 203 reads the A signal and the B signal from the pixels 200 of the image pickup unit 101 during the period of the image pickup operation 801 to realize an example of the first reading means. .. Further, for example, an example of the first read speed is realized by the read speed corresponding to the image pickup rate.

At the same time as the A signal and B signal reading operation, the column signal processing unit 102 transfers the A signal and the B signal from the image pickup board 100 to the memory board 110. Then, the memory control unit 111 records the A signal and the B signal in the recording unit 112. In FIG. 10B, the period of these operations is referred to as “memory writing”. In the present embodiment, for example, the memory control unit 111 records the A signal and the B signal in the recording unit 112, so that an example of the first storage means is realized. Further, an example of the second writing speed is realized by the writing speed (writing speed corresponding to the imaging rate) when the memory control unit 111 records the A signal and the B signal in the recording unit 112.

In this embodiment as well, as in the first embodiment, since the signal of the pixel 200 is read out from the imaging unit 101 at high speed, a case where the processing rate, which is the processing capacity of the signal processing unit 13, is lower than the imaging rate will be described.
Therefore, the memory control unit 111 reads out the signals of all the pixels 200 of the pixel area 201 recorded in the recording unit 112 of the memory board 110 at a processing rate lower than the imaging rate, and reads the signals from the memory board 110 to the circuit board 120. Transfer to the bus control unit 121. In FIG. 10B, this period is referred to as “memory read”.
In the present embodiment, as described above, rate conversion from a high-speed imaging rate to a low-speed processing rate is performed using the recording unit 112.

The signal of the pixel 200 transferred to the circuit board 120 at the processing rate is subjected to necessary signal preprocessing in the image signal preprocessing unit 122, and the signal processing unit from the image sensor 12 via the image signal output unit 123. It is output to 13.
Here, the A + B signal, which is an addition signal of the photoelectric conversion elements D1a and D1b of the pixel 200, is used to create the captured image in the signal processing unit 13. Therefore, the signals of the pixel 200 read from the recording unit 112 are the A signal and the B signal as described with reference to FIG. 9B. The signal preprocessing performed by the image signal preprocessing unit 122 requires the calculation formula "(A signal) + (B signal)" for calculating the A + B signal. The image signal preprocessing unit 122 may perform scratch correction, noise reduction processing, or the like before executing this calculation formula. As a result, the image signal preprocessing unit 122 outputs an A + B signal as a signal for creating a captured image. In FIG. 10B, this period is referred to as “A + B processing”. In the present embodiment, an example of the addition means is realized by the image signal preprocessing unit 122 calculating the A + B signal.

As a result, it is not necessary to output the two signals of the A signal and the B signal shown in FIG. 9B from the image sensor 12 to the signal processing unit 13, so that the data when the image sensor 12 outputs the two signals to the signal processing unit 13. It is possible to reduce the transfer capacity of.
In the present embodiment, for example, the memory control unit 111 reads the signals of all the pixels 200 of the pixel area 201 recorded in the recording unit 112 of the memory board 110 at the processing rate, so that the second reading means is used. An example is realized. Further, for example, an example of the second read speed is realized by the speed corresponding to the processing rate. Further, for example, an example of a signal of the pixel stored by the first storage means by the signals of all the pixels 200 of the pixel area 201 and used for creating an image captured by the imaging means. Is realized.

The signal of the captured image processed at the processing rate by the signal processing unit 13 is temporarily stored in a buffer (not shown) in the signal processing unit 13 or the synchronization control unit 15. In FIG. 10B, this period is referred to as “buffer write”.
Then, the signal processing unit 13 or the synchronization control unit 15 reads the signal of the captured image processed during the "image processing" period from the buffer and transfers it to the image display unit 17. In FIG. 10B, this period is referred to as “buffer read”.
The signal of the captured image read from the buffer is displayed on the image display unit 17 during the period of the moving image display 802. Therefore, as shown in FIGS. 10A and 10B, the period of "buffer read" is the same as the period of the moving image display 802.

At this time, if the image processing is slower than the image display, a display problem occurs, so the processing rate must be higher than the moving image rate. Therefore, in FIG. 10B, the period of "buffer writing", which is the same as the period of "image processing", is shorter than the period of "buffer reading".
In the present embodiment, for example, the signal processing unit 13 or the synchronization control unit 15 stores the signal of the image-processed captured image in the buffer, thereby realizing an example of the second storage means. Further, for example, an example of the first writing speed is realized by the speed at which the signal processing unit 13 or the synchronization control unit 15 writes the signal of the image-processed captured image to the buffer (writing speed corresponding to the processing rate). .. Further, for example, an example of a second storage medium is realized by a buffer. Further, for example, when the signal processing unit 13 or the synchronization control unit 15 reads the signal of the image-processed captured image from the buffer, an example of the fourth reading means is realized. Further, for example, an example of the third reading speed is realized by the speed at which the signal processing unit 13 or the synchronization control unit 15 reads the signal of the image-processed captured image from the buffer (reading speed corresponding to the moving image rate). ..
In the present embodiment, as described above, rate conversion from a high-speed processing rate to a low-speed moving image rate is performed using the buffer of the signal processing unit 13 or the synchronization control unit 15.
Further, the created captured image can be recorded in the image recording unit 18 according to the instruction of the synchronization control unit 15.
Here, in the description of FIG. 10B, it is assumed that the signal processing unit 13 or the synchronization control unit 15 has a buffer, but the image display unit 17 may be provided with a buffer. In this case, the signal of the image-processed captured image is transferred to the buffer in the image display unit 17 at the processing rate, read out at the moving image rate, and displayed on the image display unit 17.

In the AF processing operation shown in FIG. 10C, items that are mainly performed as operations related to the focus detection processing (AF processing) are shown in order from the top. In FIG. 10 (c), the horizontal direction is a period corresponding to FIG. 10 (a) and indicates an operation period of each item.
The “memory writing” to the recording unit 112, which is performed at the same time as the “A / B reading” that is the imaging operation 801 is the same item as that shown in FIG. 10 (b). Therefore, detailed description thereof will be omitted here.
As shown in FIG. 9A, the focus detection region 903 is provided in the pixel region 201 in the imaging unit 101. The signal processing unit 13 performs the focus detection process based on the A signal and the B signal obtained from the pixels 200 in the focus detection region 903.
At this time, the pixels 200 (focus detection pixels) used for focus detection do not have to be all the pixels in the focus detection region 903. In this embodiment, as shown in FIG. 9C, the signal processing unit 13 uses pixels 200 corresponding to the even-numbered rows of G (green) filters in the focus detection region 903.

Here, in order to show that the signal of the focus detection pixel (pixel 200 corresponding to the even-numbered G (green) filter in the focus detection area 903) is at the position of the discrete pixel 200, FIG. 10 (c) is shown. Then, one rectangle with a time interval is expressed as a signal of one pixel 200. Further, in FIG. 10C, the signal (rectangle) of the focus detection pixel is represented as “AF information on memory”. In the present embodiment, for example, the focus detection region 903 realizes an example of a second region that is narrower than the first region and includes pixels that output signals used for detecting the focus. Will be done. Further, for example, the focus detection pixel realizes an example of a pixel that outputs a signal used for detecting the focus.

It can be seen that in the "A / B read" operation, it is necessary to read all the pixels in the focus detection area 903 in order to read the focus detection pixels. Therefore, when the focus detection process is performed at the same time as this reading using the signals of the focus detection pixels sequentially read from the focus detection area 903, the focus detection process is performed over a period of reading all the pixels 200 in the focus detection area 903. The circuit involved in must continue to operate.
However, in the present embodiment as well as in the first embodiment, the memory control unit 111 continuously transmits only the signals of the focus detection pixels recorded in the recording unit 112 of the memory board 110 as shown in FIG. 9 (c). And read it out, and transfer it from the memory board 110 to the circuit board 120. At this time, the signals of the focus detection pixels read from the recording unit 112 are the A signal and the B signal. In FIG. 10 (c), this period is referred to as "continuous reading". In the present embodiment, for example, the memory control unit 111 continuously reads only the signal of the focus detection pixel recorded in the recording unit 112 of the memory board 110, thereby realizing an example of the third reading means. .. Further, for example, the A signal and the B signal of the focus detection pixel realize an example of a signal used for detecting the focus, which is a part of the signal of the pixel stored by the first storage means. ..

The signal of the focus detection pixel transferred to the circuit board 120 is subjected to necessary signal preprocessing in the AF signal preprocessing unit 124, and is transmitted from the image sensor 12 to the signal processing unit 13 via the AF signal output unit 125. It is output. The signal preprocessing performed by the AF signal preprocessing unit 124 includes scratch correction, noise reduction processing, and the like. However, since the A signal and the B signal are signals capable of focus detection processing, the AF signal output unit 125 may output the A signal and the B signal to the signal processing unit 13 as they are. As a result, the AF signal preprocessing unit 124 outputs the A signal and the B signal as the signals of the focus detection pixels.

Then, the focus detection processing unit (not shown) included in the signal processing unit 13 is a different photoelectric conversion element Ta1 whose intra-pixel pupil is divided based on the A signal and the B signal for focus detection relating to a plurality of pixels in the horizontal direction. The phase difference of the incident light with respect to Ta2 is detected.
Next, the focus detection processing unit calculates an AF evaluation value, which is an evaluation value corresponding to the distance information from the image pickup apparatus to the subject, and is an evaluation value used for AF, based on the detected phase difference, and synchronizes. Output to the control unit 15. In the present embodiment, for example, an example of the detection means is realized by using the focus detection processing unit.

Up to this point, the focus detection process executed by the signal processing unit 13 is performed, and in FIG. 10C, this period is referred to as “AF evaluation value calculation”.
Based on the AF evaluation value, the synchronization control unit 15 drives the focusing lens included in the imaging optical system 11 to control the subject so that it is in focus. In FIG. 10C, shifting to control by the synchronous control unit 15 is referred to as “AF control”.

The focus detection processing unit of the signal processing unit 13 continuously performs focus detection processing based on the signals of the focus detection pixels that have been “continuously read”. Here, the amount of data processing per unit time required for the focus detection process of the focus detection pixel is referred to as an AF rate, if necessary.
In the AF processing operation of FIG. 10C, the focus detection processing unit of the signal processing unit 13 uses the A signal and the B signal of the focus detection pixels continuously read from the recording unit 112 at the AF rate to perform AF. The focus detection process is performed at the rate to calculate the AF evaluation value. Then, the focus detection processing unit outputs the AF evaluation value to the synchronization control unit 15. Therefore, in FIG. 10 (c), the period of "continuous reading" and the period of "AF evaluation value calculation" are the same period. Therefore, at this time, rate conversion from a low-speed imaging rate to a high-speed AF rate is performed using the recording unit 112.

As described above, in the present embodiment, it is possible to simultaneously acquire the focus detection signal and the captured image signal in one imaging operation. Further, in the imaging operation, the processing rate for image processing of the captured image is slower than the imaging rate for reading the signal of the pixel 200 of the imaging unit 101 and transferring it to the recording unit 112. Therefore, the period of image processing can be determined based on the processing capacity of the signal processing unit 13, the magnitude of power consumption, and the amount of heat generated. Further, only the signals of the focus detection pixels in the focus detection area 903 are continuously read out. Therefore, in the imaging operation, the operation of the focus detection processing unit of the signal processing unit 13 and the AF signal preprocessing unit 124 is compared with the case where all the pixels in the focus detection region 903 are read out and the focus detection processing is performed at the same time. The period can be shortened. Therefore, the power consumption of the image pickup apparatus can be reduced. Further, since the transfer period of the focus detection signal from the image sensor 12 to the signal processing unit 13 is shortened, the time required for the focus detection process can be significantly shortened. Further, since the signal for the captured image output from the image sensor 12 to the signal processing unit 13 is only the A + B signal, the processing rate can be further reduced and the data transfer capacity can be reduced. Further, by recording the signal of the pixel of the imaging unit 101 in the recording unit 112, the restriction of the processing rate required for the image processing is eliminated, and the imaging operation can be performed at high speed, so that the rolling distortion can be reduced. As described above, also in the present embodiment, as in the first embodiment, when the imaging device including the pixel having the intra-pixel pupil division function performs the focus detection of the phase difference detection method and the creation of the subject image, the frame It is possible to suppress the decrease in the rate and reduce the power consumption.

It should be noted that the above-described embodiments are merely examples of embodiment of the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

11: Imaging optical system, 12: Image sensor, 13; Signal processing unit, 15: Synchronous control unit, 17: Image display unit

Claims (13)

An imaging means including a plurality of pixels, each having a first photoelectric conversion unit and a second photoelectric conversion unit.
A first reading means for reading a first signal output from the first photoelectric conversion unit and a second signal output from the second photoelectric conversion unit from the imaging means at a first reading speed. When,
A first storage means for storing the first signal and the second signal read by the first reading means in the first storage medium, and
As a signal used to create the image captured by the pre-Symbol image pickup means, and a second reading means for reading said first signal and said second signal from said first memory means at a second readout speed ,
As a signal used to detect focal point, has a third reading means for continuously reading from said first signal and said second signal to said first memory means,
Imaging and wherein the slow towards the second reading speed than the first readout speed. The imaging apparatus according to claim 1, further comprising a summing means for adding the first signal and the second signal read by the pre-Symbol second reading means. Provide Reinforced predetermined image processing on the signal of the first signal and the second signal are added by the adding means, further comprising an image processing means for generating a signal of the image captured by the image pickup means 2. The image pickup apparatus according to claim 2. The second read speed is a speed corresponding to a speed at which an image based on the signal stored by the first storage means is output to the image processing means which is an output destination thereof. Item 3. The imaging device according to item 3. A second storage means for storing the signal generated by the image processing means in the second storage medium at a first writing speed slower than the first reading speed.
3 or 4 according to claim 3, further comprising a fourth reading means for reading the signal stored by the second storage means at a third reading speed slower than the first reading speed. The imaging apparatus according to. The imaging device according to claim 5 , wherein the third read speed is slower than the second read speed and the first write speed. The imaging device according to claim 5 or 6 , wherein the second read speed and the first write speed are the same. The first read speed is the same as the second write speed, which is the write speed of the first signal and the second signal to the first storage medium by the first storage means. The imaging device according to any one of claims 1 to 7. To detect the focal point, according to any one of claims 1-8, characterized by further comprising a detection means for detecting a phase difference between the first signal and the second issue signal Imaging device. The imaging means includes the first substrate including the plurality of pixels and
A third substrate including a second substrate including the first storage medium, the second reading means, and an output means for outputting the signal read by the third reading means to the outside of the imaging means. And have
The first substrate, the second substrate, and the third substrate are laminated so as to sandwich the second substrate between the first substrate and the third substrate , and are electrically connected to each other. The imaging device according to any one of claims 1 to 9, wherein the image pickup apparatus is connected to the image device. The imaging device according to claim 10 , wherein the plurality of pixels are arranged on the surface of the first substrate on the side on which the subject light is incident. Each is a control method of an image pickup apparatus having an image pickup means including a plurality of pixels having a first photoelectric conversion unit and a second photoelectric conversion unit.
A first readout step of reading out a first signal output from the first photoelectric conversion unit and a second signal output from the second photoelectric conversion unit from the imaging means at a first readout speed. When,
A first storage step of storing the first signal and the second signal read by the first reading step in the first storage medium, and
As a signal used to create the image captured by the pre-Symbol imaging means, a second read step of reading the first signal and the second signal from the first storage medium at a second readout speed ,
As a signal used to detect focal point, has a third reading step of reading sequentially the first signal and the second signal from the first storage medium,
Control method for an imaging apparatus characterized by slow towards the second reading speed than the first readout speed. A program for causing a computer to control an image pickup apparatus having an image pickup means including a plurality of pixels, each of which has a first photoelectric conversion unit and a second photoelectric conversion unit.
A first readout step of reading out a first signal output from the first photoelectric conversion unit and a second signal output from the second photoelectric conversion unit from the imaging means at a first readout speed. When,
A first storage step of storing the first signal and the second signal read by the first reading step in the first storage medium, and
As a signal used to create the image captured by the pre-Symbol imaging means, a second read step of reading the first signal and the second signal from the first storage medium at a second readout speed ,
As a signal used to detect focal point, by executing the third read step of reading the first signal and the second signal continuously from the first storage medium, to a computer,
A program characterized by slow towards the second reading speed than the first readout speed.

Images