JP2018133600A - Imaging apparatus, imaging apparatus control method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve suppression of degradation in a frame rate and reduction in power consumption when performing focus detection and image preparation, using imaging means with a plurality of pixels, each having a plurality of photoelectric conversion parts.SOLUTION: An imaging apparatus concurrently acquires a signal for focus detection and a signal for a picked-up image by one imaging operation and stores the signals in a recording unit 112. At this time, the imaging apparatus delays a processing rate at which image processing of picked-up images is executed more than an imaging rate at which signals of pixels 200 of an imaging unit 101 are read and transferred to the recording unit 112. The imaging apparatus continuously reads only signals of focus detection pixels in a focus detection region.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image pickup apparatus, an image pickup apparatus control method, and a program, and is particularly suitable for taking an image using an image pickup unit including a pixel including a plurality of photoelectric conversion units.

  There is an image pickup apparatus including a pixel having a plurality of photoelectric conversion units and an image pickup element having a plurality of pixels each having an in-pixel pupil division function by the plurality of photoelectric conversion units. In such an imaging apparatus, focus detection by a phase difference detection method is performed. As an example of an image sensor that outputs a signal that can be used for a focus detection method, there is one in which pixels having a pair of photoelectric conversion units are provided for each microlens of a microlens array that is two-dimensionally arranged.

  There exists a technique of patent document 1 as a technique regarding such an imaging device. In the technique disclosed in Patent Document 1, the imaging apparatus first outputs A which is an output signal of the photoelectric conversion unit A from a pixel having the photoelectric conversion unit A and the photoelectric conversion unit B as pixels divided by a microlens. An A + B image that is an addition signal of the image, the photoelectric conversion unit A, and the photoelectric conversion unit B is read out. Next, the imaging apparatus subtracts the A image from the A + B image (calculates (A + B image) − (A image)) to obtain a B image that is an output signal of the photoelectric conversion unit B. The imaging apparatus performs focus detection by the phase difference detection method using the A image and the B image, and creates a subject image using the A + B image.

JP 2013-106194 A

  However, in the technique described in Patent Document 1, it is necessary to read the A image and the A + B image, and the read data amount is doubled. For this reason, there is a problem that the readout time is also doubled. As a result, there is a problem that a frame rate as a desired moving image performance may be lowered, and a problem that a rolling distortion is deteriorated due to a pixel readout time difference for each row may occur.

  On the other hand, when reading out the A image and the A + B image while maintaining the desired frame rate, the data rate (transfer data amount per unit time) is doubled. For this reason, there exists a problem that the data rate at the time of reading A image and A + B image may exceed the data rate which an image sensor can output.

  In addition, even when the A image and the A + B image are read out at a data rate that can be output by the image sensor, the signal processing unit included in the image capturing apparatus creates a subject image using the A + B image, and the data that the image sensor can output. Need to be implemented at a rate. For this reason, there exists a problem that there exists a possibility that the limit of the signal processing rate (processing amount per unit time) of a signal processing part may be reached, or the power consumption of a signal processing part may become high.

Further, when the A image and the A + B image are read from the image sensor, the signal processing unit included in the imaging device calculates the B image from the A + B image and the A image during the creation of the subject image using the A + B image, and the A image It is necessary to perform focus detection by the phase difference detection method using the B image and the B image. For this reason, there exists a problem that there exists a possibility that the power consumption of a signal processing part may become large.
As described above, in the conventional technology, when performing the focus detection and the image creation of the phase difference detection method by the intra-pixel pupil division function using the imaging unit including a plurality of pixels each having a plurality of photoelectric conversion units, It is not easy to suppress the decrease in frame rate and reduce power consumption.

  The present invention has been made in view of such a point, and when performing focus detection and image creation using an imaging unit including a plurality of pixels each having a plurality of photoelectric conversion units, the frame rate is adjusted. The purpose is to realize a reduction in power consumption and a reduction in power consumption.

  An imaging apparatus according to the present invention includes: an imaging unit including a plurality of pixels each having a plurality of photoelectric conversion units; and a first signal that reads a signal of the pixel in a first region corresponding to an image captured by the imaging unit. Reading means, first storage means for storing the pixel signals read by the first reading means in a first storage medium, and pixel signals stored by the first storage means A second readout unit that reads out a signal used to create an image captured by the imaging unit, and a signal that is a part of the pixel signal stored in the first storage unit. And a third readout means for continuously reading out the signal used for detecting the focal point, and a first readout speed which is a readout speed of the signal of the pixel by the first readout means. , Wherein said that the direction of the second reading speed is read speed of the signal is low by the second reading means.

  According to the present invention, when performing focus detection and image creation using an imaging unit including a plurality of pixels each having a plurality of photoelectric conversion units, suppression of a decrease in frame rate and reduction in power consumption are achieved. Can be realized.

It is a figure which shows the structure of an imaging device. It is a figure which shows the board | substrate structure of an image pick-up element. It is a figure which shows the structure of an imaging substrate. It is a figure which shows the circuit structure of a pixel. 2 is a diagram illustrating a schematic configuration of a pixel 200. FIG. It is a figure which shows the circuit structure of a column signal processing part. It is a figure explaining the 1st example of signal processing. It is a figure showing the 1st example of operation timing of an imaging device. It is a figure explaining the 2nd example of signal processing. It is a figure showing the 2nd example of operation timing of an imaging device.

Hereinafter, embodiments will be described in detail with reference to the drawings.
(First embodiment)
First, the first embodiment will be described.
FIG. 1 is a diagram illustrating an example of the configuration of the imaging apparatus according to the present embodiment. The imaging apparatus of the present embodiment can be applied to, for example, a digital still camera, a digital video camera, an industrial camera, an in-vehicle camera, or a medical camera.
An imaging apparatus shown in FIG. 1 includes an imaging optical system 11, an imaging element 12, a signal processing unit 13, a compression / decompression unit 14, a synchronization control unit 15, an operation unit 16, an image display unit 17, an image recording unit 18, and a vibration detection unit 19. Have

The imaging optical system 11 includes a lens for forming a subject image, a lens driving mechanism for performing zooming, focusing, or image stabilization, a mechanical shutter mechanism, a diaphragm mechanism, and the like. Among these, the movable part is driven based on a control signal from the synchronization control part 15.
The imaging device 12 includes, for example, an XY address type CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging element 12 performs an imaging operation such as exposure, signal readout, and reset in accordance with a control signal from the synchronization control unit 15. The image sensor 12 has an analog-digital conversion circuit (hereinafter referred to as “AD conversion circuit”). A signal read from the CMOS sensor is subjected to analog-digital conversion (hereinafter referred to as “AD conversion”) by an analog-digital conversion circuit and digitized. The digitized image signal is output to the signal processing unit 13. The pixels provided in the image sensor 12 output a first focus detection signal for focus detection by a phase difference detection method and a captured image signal for creating a captured image. In the present embodiment, the first focus detection signal corresponds to an A signal described later, and the captured image signal corresponds to an A + B signal described later. Then, the image pickup device 12 obtains a second focus detection signal using the first focus detection signal and the captured image signal output from one pixel. In the present embodiment, the second focus detection signal corresponds to a signal (B signal) obtained by subtracting the A signal from the A + B signal described later.

  The signal processing unit 13 performs signal processing and control information detection on the digitized image signal input from the image sensor 12 under the control of the synchronization control unit 15. The signal processing includes, for example, white balance adjustment, color correction, and gamma correction. The control information includes control information such as AF (Auto Focus) and AE (Auto Exposure). Then, the signal processing unit 13 outputs the image signal and control information subjected to signal processing to the synchronization control unit 15.

  In the present embodiment, the signal processing unit 13 performs focus detection processing by a phase difference detection method as AF detection processing. As a focus detection method by the phase difference detection method, for example, there is the following method using the intra-pixel pupil division function. The signal processing unit 13 detects, in a pixel including two photoelectric conversion elements, a waveform shift (phase difference) between the output signal of one of the plurality of pixels and the output signal of the other photoelectric conversion element. That is, the signal processing unit 13 detects the phase difference of the incident light with respect to the two photoelectric conversion elements obtained by dividing the intra-pixel pupil, based on the above-described two focus detection signals related to a plurality of pixels. Then, the signal processing unit 13 obtains the distance to the subject based on the detected phase difference, performs focus detection, and drives the focusing lens so that the subject is in focus. The signal processing unit 13 creates a photographed image using a photographed image signal obtained from the pixels in the pixel region.

  The compression / decompression unit 14 operates under the control of the synchronization control unit 15. The compression / decompression unit 14 receives the image signal signal-processed by the signal processing unit 13 from the synchronization control unit 15, performs compression encoding processing, or converts the still image encoded data supplied from the synchronization control unit 15 into encoded data of the still image. For example, decompression decoding processing is performed. Then, the compression / decompression unit 14 outputs the encoded data subjected to the compression encoding process and the image signal subjected to the expansion decoding process to the synchronization control unit 15. Further, the compression / decompression unit 14 may execute a compression encoding / decompression decoding process of a moving image.

  The synchronization control unit 15 includes, for example, a microcontroller. The microcontroller includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The synchronization control unit 15 comprehensively controls each unit of the imaging apparatus by executing a program stored in a ROM or the like. Further, image data transfer and control data transfer between the respective units are also executed via the synchronization control unit 15.

The operation unit 16 includes, for example, various operation keys such as a shutter release button, a lever, a dial, and the like. The operation unit 16 outputs a control signal corresponding to the input operation by the user to the synchronization control unit 15.
The image display unit 17 includes a display device such as an LCD (Liquid Crystal Display), an interface circuit for the display device, and the like. The image display unit 17 generates a signal for causing the display device to display the image signal supplied from the synchronization control unit 15, and supplies the generated signal to the display device to display an image.
For example, a recording medium composed of a portable semiconductor memory or the like is connected to the image recording unit 18. For example, the image recording unit 18 receives a file of encoded data (image data file) that has been compression-encoded by the compression / decompression unit 14 from the synchronization control unit 15 and stores the file in a recording medium. Further, the image recording unit 18 reads data designated based on the control signal from the synchronization control unit 15 from the recording medium and outputs the data to the synchronization control unit 15.

The vibration detection unit 19 includes, for example, a gyro sensor and an acceleration sensor. The vibration detection unit 19 detects a signal indicating the direction and speed (motion vector) of vibration with respect to the imaging apparatus due to camera shake or the like, and outputs the signal to the synchronization control unit 15.
The synchronization control unit 15 calculates image movement based on the motion vector and performs image stabilization. Image stabilization methods include optical image stabilization and electronic image stabilization.
When performing optical camera shake correction, a shift lens arranged in the imaging optical system 11 is used. Optical image stabilization cancels image blur by moving the shift lens in the imaging optical system 11 so as to capture the direction opposite to the camera shake based on the amount of movement calculated from the motion vector. It is a method to do.

When electronic camera shake correction is performed, the imaging element 12 is used in which the area that can be captured is set wider than the captured area of the recorded image. Electronic camera shake correction is a method of canceling image blur by cutting out a shooting region in the opposite direction to the camera shake based on the amount of movement calculated from a motion vector to form a recorded image.
In the present embodiment, either one of the camera shake corrections may be used, or both of the camera shake corrections may be used. Further, the synchronization control unit 15 can also obtain a motion vector from changes in continuously captured images, instead of the direction and intensity of vibration detected by the vibration detection unit 19.

Next, an example of a basic operation of the imaging apparatus according to the present embodiment will be described.
Prior to capturing a still image, image signals output from the image sensor 12 are sequentially supplied to the signal processing unit 13.
The signal processing unit 13 performs signal processing on the image signal from the image sensor 12 and supplies the image signal subjected to the signal processing to the image display unit 17 through the synchronization control unit 15 as a camera-through image signal. As a result, the camera through image supplied from the signal processing unit 13 is displayed on the image display unit 17. The user can adjust the angle of view by looking at the image displayed by the image display unit 17.

When the shutter release button of the operation unit 16 is pressed in this state, an image signal for one frame from the image sensor 12 is taken into the signal processing unit 13 under the control of the synchronization control unit 15. The signal processing unit 13 performs signal processing on the captured image signal for one frame, and supplies the image signal subjected to the signal processing to the compression / decompression unit 14.
The compression / decompression unit 14 compresses and encodes the image signal supplied from the signal processing unit 13 to generate encoded data, and supplies the generated encoded data to the image recording unit 18 through the synchronization control unit 15. As a result, the data file of the captured still image is recorded in the image recording unit 18.

When reproducing a still image data file recorded in the image recording unit 18, the synchronization control unit 15 reads the data file selected according to the operation input from the operation unit 16 from the image recording unit 18, and compresses and decompresses the data file. 14 to execute the decompression decoding process. The decoded image signal is supplied to the image display unit 17 via the synchronization control unit 15. As a result, the still image is displayed (reproduced) by the image display unit 17.
When recording a moving image, the compression / decompression unit 14 compresses and encodes the image signal sequentially processed by the signal processing unit 13 to generate encoded data, and synchronizes the generated encoded data of the moving image with synchronization. The images are sequentially transferred to the image recording unit 18 through the control unit 15. As a result, the data file of the captured moving image is recorded in the image recording unit 18.
When playing back a moving image data file recorded in the image recording unit 18, the synchronization control unit 15 reads the data file selected according to the operation input from the operation unit 16 from the image recording unit 18, and compresses and decompresses the data file. 14 to execute the decompression decoding process. The decoded image signal is supplied to the image display unit 17 via the synchronization control unit 15. As a result, the moving image is displayed (reproduced) by the image display unit 17.

FIG. 2 is a diagram illustrating an example of the substrate configuration of the image sensor 12 of the present embodiment.
FIG. 2A shows an example of a planar configuration of each substrate, and FIG. 2B shows an example of a stacked configuration of each substrate.
The imaging element 12 has a structure (stacked structure) in which the imaging substrate 100, the memory substrate 110, and the circuit substrate 120 are stacked in this order. The imaging board 100, the memory board 110, and the circuit board 120 are electrically connected to each other. On the imaging substrate 100, circuit elements (photoelectric conversion elements, transistors, capacitors, and the like) constituting pixels are arranged. On the memory substrate 110, circuit elements (storage circuits such as a memory) constituting a storage unit are arranged. Circuit elements (transistors, capacitors, etc.) constituting the signal preprocessing unit are arranged on the circuit board 120. In the present embodiment, for example, by using the imaging substrate 100, an example of a first substrate including a plurality of pixels is realized. For example, by using the memory substrate 110, an example of a second substrate including the first storage medium is realized. In addition, for example, by using the circuit board 120, an example of a third substrate including an output unit that outputs the signals read by the second reading unit and the third reading unit to the outside of the imaging unit is realized. The

  In the planar configuration of FIG. 2A, the imaging substrate 100 has a backside illumination type CMOS sensor. An imaging unit 101 including pixels having photoelectric conversion elements is disposed on the surface of the imaging substrate 100 on which subject light is incident. A column signal processing unit 102 that performs pixel wiring and AD conversion is disposed on the surface on the memory substrate 110 side opposite to the surface on which the subject light is incident on the imaging substrate 100. Further, the imaging substrate 100 is formed with a micropad for electrical connection with the memory control unit 111 of the memory substrate 110.

The memory substrate 110 includes a memory control unit 111 and a recording unit 112. A micropad for electrical connection with the column signal processing unit 102 is formed on the surface of the memory substrate 110 on the imaging substrate 100 side. On the surface of the memory substrate 110 on the circuit board 120 side, a micropad for electrical connection with the bus control unit 121 is formed. In the present embodiment, for example, an example of the first storage medium is realized by using the recording unit 112.
The circuit board 120 includes a bus control unit 121, an image signal preprocessing unit 122, an image signal output unit 123, an AF signal preprocessing unit 124, and an AF signal output unit 125. A micropad for electrical connection with the memory control unit 111 is formed on the surface of the circuit board 120 on the memory board 110 side. Here, the image signal preprocessing unit 122 and the AF signal preprocessing unit 124 are combined to form a signal preprocessing unit 126. In the present embodiment, for example, an example of an output unit is realized by using the image signal output unit 123 and the AF signal output unit 125.

  By connecting the micropads of the imaging substrate 100 and the micropads of the memory substrate 110 corresponding to the micropads by micro bumps, it is possible to transfer AD converted pixel signal data to the memory control unit 111. It becomes. Similarly, signal data is sent from the memory control unit 111 to the bus control unit 121 by connecting the micropads of the memory substrate 110 and the micropads of the circuit board 120 corresponding to the micropads by micro bumps. It becomes possible to transfer.

  In the example shown in FIG. 2A, the memory control unit 111 transfers the signal data of the pixels transferred from the imaging substrate 100 to the recording unit 112 and records it, and the signal data read from the recording unit 112 as a bus. The operation to be transferred to the control unit 121 is controlled. In addition, the bus control unit 121 controls the operation of distributing and transferring the signal data transferred from the memory control unit 111 to the image signal preprocessing unit 122 and the AF signal preprocessing unit 124.

  When the memory control unit 111 and the recording unit 112 are formed on the surface of the memory substrate 110 on the imaging substrate 100 side, the micropad and the memory control on the surface of the memory substrate 110 on the circuit substrate 120 side are used using the vertical through electrode. The part 111 is electrically connected. On the other hand, when the memory control unit 111 and the recording unit 112 are formed on the surface of the memory substrate 110 on the circuit board 120 side, the micropads on the surface of the memory substrate 110 on the imaging substrate 100 side are formed using the vertical through electrodes. The memory control unit 111 is electrically connected.

  Similarly, when the bus control unit 121, the signal preprocessing unit 126, the image signal output unit 123, and the AF signal output unit 125 are formed on the surface of the circuit board 120 on the memory board 110 side, the memory board 110 of the circuit board 120 is formed. A pad electrode for signal output is formed on the opposite surface. A signal output terminal can be provided on the lower surface of the image sensor 12 by electrically connecting the image signal output unit 123 and the AF signal output unit 125 to the signal output pad electrode using the vertical through electrode. . On the other hand, when the bus control unit 121, the signal preprocessing unit 126, the image signal output unit 123, and the AF signal output unit 125 are formed on the surface of the circuit board 120 opposite to the surface on the memory substrate 110 side, Like this. In other words, the micropad on the surface of the circuit board 120 on the memory board 110 side is electrically connected to the bus control unit 121 using the vertical through electrode, and the signal is sent to the image signal output unit 123 and the AF signal output unit 125. An output pad electrode may be formed.

  As described above, FIG. 2B shows an example of the laminated structure of the image sensor 12. In the example shown in FIG. 2B, the imaging element 12 is configured by stacking an imaging substrate 100, a memory substrate 110, and a circuit substrate 120 in this order. Subject light (Light) is incident from the upper surface (the surface on the side where the imaging substrate 100 is disposed) of the image sensor 12, and a signal is output from the lower surface (the surface on which the circuit substrate 120 is disposed). Thereby, since the area of the image pick-up element 12 can be reduced, the effect of raising the mounting density of an image pick-up device can be expected. Moreover, you may connect directly the micropads which oppose between each board | substrate, without providing a micro bump. Thereby, since the thickness of the image pick-up element 12 can be reduced, the effect of raising the mounting density of an image pick-up device can be expected.

FIG. 3 is a diagram illustrating an example of the configuration of the imaging substrate 100 of the present embodiment.
The imaging substrate 100 illustrated in FIG. 3 includes a pixel region 201 including pixels 200, a vertical scanning unit 202, a column signal processing unit 203, a horizontal scanning unit 207, an output unit 209, and a timing unit 211.
The pixel region 201 is configured using a pixel 200 having a CMOS sensor. Each pixel 200 is arranged in a two-dimensional matrix in the horizontal and vertical directions, as indicated by P11 to P46. In FIG. 3, the pixels in the first row are represented as P11 to P16, and the pixels in the fourth row are represented as P41 to P46. In FIG. 3, a case where the pixel array in the pixel area 201 is a 6 × 4 array (4 rows and 6 columns) will be described as an example. However, the pixel arrangement in the pixel region 201 is not limited to this number. In the present embodiment, for example, the pixel region 201 realizes an example of a first region corresponding to an image captured by the imaging unit.
In the present embodiment, it is assumed that a 2 × 2 array of color filters is arranged for the plurality of pixels 200. In this embodiment, the 2 × 2 array is an array in which odd rows are repeated R (red) filters and G (green) filters, and even rows are repeated G (green) filters and B (blue) filters. Shall.

  The vertical scanning unit 202 selects the pixel array of the pixel region 201 row by row, and controls the reset operation and readout operation of the selected pixel row. The pixel control line 221 is commonly connected to each pixel row, and transmits a drive control signal in units of rows by the vertical scanning unit 202. The vertical signal lines 231 are commonly connected to the respective pixel columns, and the pixel signals in the row selected by the pixel control line 221 are read out to the corresponding vertical signal lines 231, respectively. The column signal processing unit 203 is provided for each corresponding vertical signal line 231, and performs signal processing to be described later on each pixel signal of the row unit sent through the vertical signal line 231.

The horizontal scanning unit 207 selects the column signal processing unit 203 for each column via the column selection line 251, and outputs the digitized pixel signal stored in the selected column signal processing unit 203 via the horizontal output line 261. To the output unit 209. The output unit 209 outputs the digitized pixel signal in units of rows to the signal processing unit 13. The timing unit 211 outputs various clock signals, control signals, and the like necessary for the operation of each unit of the image sensor 12 based on the control signal from the synchronization control unit 15. Here, the control lines 271, 281, and 285 are control lines that send a clock signal, a control signal, and the like from the timing unit 211 to the vertical scanning unit 202, the column signal processing unit 203, and the horizontal scanning unit 207, respectively.
In FIG. 3, the case where the timing unit 211 is arranged on the imaging substrate 100 has been described as an example. However, this is not always necessary. For example, the timing unit 211 may be disposed on the memory board 110 or the circuit board 120. Further, the timing unit 211 may be disposed on each of the imaging substrate 100, the memory substrate 110, and the circuit substrate 120.

FIG. 4 is a diagram illustrating an example of a circuit configuration of the pixel 200 of the imaging substrate 100 of the present embodiment.
A pixel 200 surrounded by a broken line in FIG. 4 represents one of the pixels 200 constituting the pixel region 201 as a representative. In addition, the pixel 200 is electrically connected to other circuits by a pixel control line 221 and a vertical signal line 231.
The vertical signal line 231 is connected to the load circuit and the column signal processing unit 203 and is connected in common to each pixel in one vertical column, and outputs a pixel signal.

  The pixel control line 221 is connected to the vertical scanning unit 202 and is commonly connected to pixels in one horizontal row. By simultaneously controlling pixels in one horizontal row using the pixel control line 221, resetting and signal reading can be performed. The pixel control line 221 includes a reset control line pR, transfer control lines pTa, pTb, and a vertical selection line pSEL.

  The photoelectric conversion elements D1a and D1b are photodiodes that convert light into electric charges and accumulate the electric charges. In the present embodiment, for example, an example of a plurality of photoelectric conversion units is realized by the photoelectric conversion elements D1a and D1b. For example, an example of the first photoelectric conversion unit is realized by the photoelectric conversion element D1a, and an example of the second photoelectric conversion unit is realized by the photoelectric conversion element D1b. In the photoelectric conversion elements D1a and D1b, the P side of the PN junction is grounded, and the N side is connected to the sources of transfer transistors (transfer switches) T1a and T1b, respectively. The transfer transistors (transfer switches) T1a and T1b have gates connected to transfer control lines pTa and pTb, respectively, and drains connected to the FD capacitor Cfd. The transfer transistors (transfer switches) T1a and T1b control the transfer of charges from the photoelectric conversion elements D1a and D1b to the FD capacitor Cfd, respectively.

An FD (Floating Diffusion) capacitor Cfd is grounded at one end, and accumulates charges when the charges transferred from the photoelectric conversion elements D1a and D1b are converted into voltages. Here, a connection point between the drains of the transfer transistors (transfer switches) T1a and T1b and the other end of the FD capacitor Cfd is referred to as an FD node 401 as necessary.
The reset transistor (reset switch) T2 has a gate connected to the reset control line pR, a drain connected to the power supply voltage Vdd, and a source connected to the FD capacitor Cfd. The reset transistor (reset switch) T2 resets the potential of the FD node 401 to the power supply voltage Vdd.

The driving transistor (amplifying unit) Tdrv is a transistor that forms an in-pixel amplifier. The driving transistor (amplifying unit) Tdrv has a gate connected to the other end (FD node 401) of the FD capacitor Cfd, a drain connected to the power supply voltage Vdd, and a source connected to the drain of the selection transistor (selection switch) T3. Connected. The driving transistor (amplifying unit) Tdrv outputs a voltage corresponding to the voltage of the FD capacitor Cfd.
The selection transistor (selection switch) T3 has a gate connected to the vertical selection line pSEL and a source connected to the vertical signal line 231. The selection transistor (selection switch) T3 outputs the output of the driving transistor Tdrv to the vertical signal line 231 as an output signal of the pixel 200.

The load transistor Tlod of the load circuit provided for each vertical signal line 231 has its source and gate grounded and its drain connected to the vertical signal line 231. The load transistor Tlod constitutes a source follower circuit serving as an in-pixel amplifier together with the drive transistor Tdrv disposed in the pixel 200 of the column connected by the vertical signal line 231. Normally, when outputting the signal of the pixel 200, the load transistor Tlod is operated as a constant current source with a grounded gate.
In the description of the present embodiment, the transistors other than the drive transistor Tdrv and the load transistor Tlod act as switches, and are turned on (turned on) when the control line connected to the gate is High and turned off (turned off) when the control line is Low. )

Here, an example of exposure control of the photoelectric conversion element D1a will be described as the first exposure control.
It is assumed that subject light is incident on the image sensor 12.
First, at the exposure start timing, the reset transistor T2 is turned on to reset the FD node 401 side of the FD capacitor Cfd, and at the same time, the transfer transistor T1a is turned on to reset the charge of the photoelectric conversion element D1a. The exposure of the photoelectric conversion element D1a is started by sequentially turning off the transfer transistor T1a and the reset transistor T2 in the order of the transfer transistor T1a and the reset transistor T2.

  Next, after a predetermined exposure time has elapsed, the reset transistor T2 is turned on to reset the FD node 401 side of the FD capacitor Cfd. Then, the reset transistor T2 is turned off. Thereafter, the transfer transistor T1a is turned on, and the signal charge photoelectrically converted by the photoelectric conversion element D1a by exposure is transferred to the FD node 401. Then, the transfer transistor T1a is turned off. Thus, the exposure of the photoelectric conversion element D1a is completed. At this time, a signal voltage corresponding to the signal charge is generated at the FD node 401 of the FD capacitor Cfd.

  Then, by turning on the selection transistor T3, a source follower circuit including a drive transistor Tdrv and a load transistor Tlod is configured. Accordingly, a signal corresponding to the signal voltage of the FD node 401 is output to the vertical signal line 231 as a signal of the photoelectric conversion element D1a. The signal of the photoelectric conversion element D1a output to the vertical signal line 231 is input to the column signal processing unit 203, and column signal processing described later is performed.

Next, an example of exposure control of the photoelectric conversion element D1b will be described as second exposure control.
Similarly to the photoelectric conversion element D1a, the exposure of the photoelectric conversion element D1b is also controlled. However, in this embodiment, the transfer transistors T1a and T1b are controlled by different transfer control lines pTa and pTb, respectively. Therefore, the reset timing at the start of exposure and the transfer timing at the end of exposure can be set separately by the photoelectric conversion elements D1a and D1b.

  Therefore, the timing for starting exposure of the photoelectric conversion element D1b is set to be the same as the exposure time of the photoelectric conversion element D1a by calculating backward from the timing of transferring the signal charge of the photoelectric conversion element D1b to the FD capacitor Cfd. . In the present embodiment, after the column signal processing (first exposure control) of the signal of the photoelectric conversion element D1a is performed, the signal charge of the photoelectric conversion element D1b is transferred to the FD capacitor Cfd, and the first exposure is performed. This is added to the signal charge of the photoelectric conversion element D1a previously transferred to the FD capacitor Cfd under control. Thus, in this embodiment, after the exposure of the photoelectric conversion element D1a is started, the exposure of the photoelectric conversion element D1b is started during the exposure of the photoelectric conversion element D1a, and the exposure time thereof is set. To be identical.

  A specific example of the second exposure control will be described. First, the transfer transistor T1b is turned on while the signal charge of the photoelectric conversion element D1a is accumulated in the FD node 401 of the FD capacitor Cfd. Thereby, the signal charge photoelectrically converted by the photoelectric conversion element D <b> 1 b by exposure is transferred to the FD node 401. Then, the transfer transistor T1b is turned off. Thus far, the exposure of the photoelectric conversion element D1b is completed.

  At this time, a signal voltage corresponding to the signal charge obtained by adding the signal charges of the photoelectric conversion elements D1a and D1b is generated at the FD node 401 of the FD capacitor Cfd. Then, by turning on the selection transistor T3, a source follower circuit including a drive transistor Tdrv and a load transistor Tlod is configured. As a result, a signal corresponding to the signal voltage of the FD node 401 is output to the vertical signal line 231 as an addition signal of the photoelectric conversion elements D1a and D1b. The addition signals of the photoelectric conversion elements D1a and D1b output to the vertical signal line 231 are input to the column signal processing unit 203, and column signal processing described later is performed.

Here, when performing exposure control of the photoelectric conversion elements D1a and D1b, the reset timing of the start of exposure may be set simultaneously in the photoelectric conversion elements D1a and D1b, and exposure to each of the photoelectric conversion elements D1a and D1b may be started simultaneously. . Then, the signal charge of the photoelectric conversion element D1a is transferred for the second time at the timing of transferring the signal charge of the photoelectric conversion element D1b. As a result, it is possible to perform shooting with the exposure time deviation eliminated for the addition signals of the photoelectric conversion elements D1a and D1b.
Here, the output signal of the photoelectric conversion element D1a is referred to as the focus detection A signal or simply the A signal, and the output signal obtained by adding the signal charges of the photoelectric conversion elements D1a and D1b is used as the A + B signal or simply the A + B signal for the photographed image. It shall be called. A signal obtained by subtracting the A signal for focus detection from the A + B signal for captured images is referred to as a focus detection B signal or simply a B signal. The B signal for focus detection is a signal corresponding to the output signal of the photoelectric conversion element D1b.

FIG. 5 is a diagram illustrating an example of a schematic configuration of the pixel 200 of the imaging substrate 100 of the present embodiment.
FIG. 5A shows an example of a plan view in which the pixels 200 are arranged in 2 × 2, and FIG. 5B shows a cross-sectional view taken along line xx ′ of FIG.
The regions 501a and 501b of the photoelectric conversion elements D1a and D1b correspond to the N side of the PN junction of the photoelectric conversion elements D1a and D1b, respectively, and the imaging substrate 100 corresponds to the P side. A position 502 indicates the position of a circuit portion other than the photoelectric conversion elements D1a and D1b of the pixel 200. In FIG. 5, the pixel control line 221 and the vertical signal line 231 are not shown for the convenience of description.

As described above, the micro lens 503 is provided for each pixel 200. In the present embodiment, the microlens 503 is arranged to be shifted downward from the center of the pixel 200 in FIG. 5A so as to cover both the photoelectric conversion elements D1a and D1b evenly. A color filter 504 is also provided for each pixel 200. The color filter 504 uniformly covers both the photoelectric conversion elements D1a and D1b. As described above, for each pixel 200, any one of an R (red) filter, a G (green) filter, and a B (blue) filter is disposed.
As shown in FIG. 5, in the present embodiment, one photoelectric conversion element D1a and D1b share one microlens 503. Therefore, focus detection can be performed based on the image signal (A signal for focus detection) obtained from the photoelectric conversion element D1a and the image signal (B signal for focus detection) obtained from the photoelectric conversion element D1b.

FIG. 6 is a diagram illustrating an example of a circuit configuration of the column signal processing unit 203 of the imaging substrate 100 according to the present embodiment.
The sample hold circuit (S / H) 611 is electrically connected to the signal selection control line pSH1. The sample hold circuit (S / H) 611 holds the pixel signal received from the vertical signal line 231 under the control of the timing unit 211 via the signal selection control line pSH1.

The AD conversion circuit (AD) 621 is connected to the AD control line pAD1. The AD conversion circuit (AD) 621 AD converts the pixel signal held by the sample hold circuit (S / H) 611 under the control of the timing unit 211 via the AD control line pAD1. The memory circuit 631 is connected to the memory control line pMEM1, and stores a pixel digital signal output from the AD conversion circuit 621 under the control of the memory control line pMEM1.
Further, the memory circuit 631 outputs the digitized pixel signal stored in the memory circuit 631 to the digital output line DSig1 by the control via the horizontal selection line pH1.

The digital output line DSig1 is also commonly connected to the memory circuit 631 of the other column signal processing unit 203. Each column signal processing unit 203 digitally outputs the digitized pixel signal stored in the memory circuit 631 under the control of the horizontal scanning unit 207 via the horizontal selection line pH1 corresponding to the column signal processing unit 203. Output to the line DSig1 in a predetermined order.
Here, the control line 281 for controlling the column signal processing unit 203 from the timing unit 211 includes a signal selection control line pSH1, an AD control line pAD1, and a memory control line pMEM1. The column selection line 251 for selecting the column signal processing unit 203 from the horizontal scanning unit 207 includes a horizontal selection line pH1. The horizontal output line 261 connected to the output unit 209 includes a digital output line DSig1.
As described above, the column signal processing unit 203 illustrated in FIG. 6 has a circuit configuration capable of AD conversion.

Next, an example of the first readout operation and the second readout operation of the imaging substrate 100 of the present embodiment will be described.
In the first reading operation, column signal processing is performed on the signal read from the photoelectric conversion element D1a and the signal of the pixels 200 for one row subjected to the first exposure control.
The signal of the photoelectric conversion element D1a output to the vertical signal line 231 is input to the column signal processing unit 203, and the sample of the column signal processing unit 203 is controlled by the control of the signal selection control line pSH1 input to the column signal processing unit 203. It is held in a hold circuit (S / H) 611.

The signal held in the sample hold circuit (S / H) 611 is converted into a digital signal by the AD conversion circuit (AD) 621 of the column signal processing unit 203 and stored in the memory circuit 631 of the column signal processing unit 203. The
The digital signal of the photoelectric conversion element D1a stored in the memory circuit 631 is read from the memory circuit 631 under the control of the horizontal scanning unit 207 via the horizontal selection line pH1 for the column signal processing unit 203. The digital signal of the photoelectric conversion element D1a read from the memory circuit 631 is output to the digital output line DSig1.
Then, the digital signal of the photoelectric conversion element D1a is transferred to the memory substrate 110 via the output unit 209 as an A signal.

In the second readout operation, column signal processing is performed on the signals read from the pixels 200 for one row subjected to the second exposure control (that is, the addition signals of the photoelectric conversion elements D1a and D1b).
The addition signal of the photoelectric conversion elements D1a and D1b is converted into a digital signal by the AD conversion circuit (AD) 621 of the column signal processing unit 203, and the column signal processing unit 203 as a digital addition signal of the photoelectric conversion elements D1a and D1b. Is stored in the memory circuit 631. Note that the column signal processing for the addition signals of the photoelectric conversion elements D1a and D1b and the output to the digital output line DSig1 are the same as those in the first read operation, and thus detailed description thereof is omitted.

The digital addition signals of the photoelectric conversion elements D1a and D1b are transferred to the memory substrate 110 via the output unit 209 as A + B signals.
As a result, the A signal and the A + B signal are read from the pixels for one row and transferred to the memory substrate 110.
By performing the first readout operation and the second readout operation for each row on the pixel 200 in the pixel region 201, one photographing operation can be performed.

FIG. 6 shows an example in which the A signal and A + B signal of the pixel 200 are transferred to the memory substrate 110 via the output unit 209. However, instead of outputting the A signal and the A + B signal to the digital output line DSig1, the following may be performed. That is, a vertical through electrode may be provided for each output of the memory circuit 631, and the memory control unit 111 of the memory substrate 110 and the memory circuit 631 may be directly connected.
In this embodiment, as described above, the A signal or A + B signal of the pixel 200 can be simultaneously transferred to the memory substrate 110 for one row, so that the transfer time can be shortened.

  FIG. 7 is a diagram illustrating an example of signal processing according to the present embodiment. FIG. 7A shows an A signal and an A + B signal read from the pixel 200 in the pixel area 201 in the imaging unit 101. In the present embodiment, the A signal or the A + B signal is read from the pixels 200 in the pixel region 201 in a predetermined order for each row. In FIG. 3, the case where the pixel array is a 6 × 4 array (4 rows and 6 columns) has been described as an example. However, in FIG. 7A, the pixel array is a 10 × 8 array (8 rows and 10 columns). ) Will be described as an example.

  In FIG. 7, each signal is shown in parentheses, and a color filter is shown before that. As described above, in this embodiment, the odd-numbered rows are repeated with the R (red) filter and the G (green) filter, and the even-numbered rows are repeated with the G (green) filter and the B (blue) filter. Use an array of color filters. Therefore, the signal (A signal and A + B signal) of the pixel 200 corresponding to the R (red) filter is denoted as R. Further, a signal (A signal and A + B signal) of the pixel 200 corresponding to the G (green) filter is denoted as G. Further, a signal (A signal and A + B signal) of the pixel 200 corresponding to B (blue) is denoted as B. For example, the signal 701 is an A signal of the pixel 200 corresponding to the R (red) filter in the first row and the first column of the pixel region 201 and is denoted as R (A). A signal 702 is an A + B signal of the pixel 200 corresponding to the R (red) filter in the first row and the first column of the pixel region 201 and is expressed as R (A + B). Further, a region 703 surrounding the pixel 200 in the seventh row and the ninth column from the pixel 200 in the second row and the second column which is a diagonal is set as a focus detection region.

FIG. 7B shows an A + B signal of the pixel 200 used for creating a captured image in the signal processing unit 13 in FIG.
In the present embodiment, the signal processing unit 13 uses A + B signals in all the pixels 200 in the pixel area 201 when creating a captured image. For example, for the pixel 200 in the first row and the first column, the signal processing unit 13 uses a signal 702 (R (A + B)) that is an A + B signal. At this time, the signal processing unit 13 does not use the signal 701 (R (A)) that is the A signal.

FIG. 7C shows a signal of the pixel 200 related to the focus detection process (AF process) in the signal processing unit 13 in FIG.
In the present embodiment, an example in which the signal processing unit 13 uses the signal of the pixel 200 corresponding to the G (green) filter in the even-numbered row in the focus detection area 703 in the focus detection process (AF process) will be described. To do. Therefore, as shown in FIG. 7C, the pixels 200 related to the focus detection process (AF process) are in the third, fifth, seventh, second, fourth, and sixth rows, respectively. The pixel 200 corresponds to the G (green) filter in the ninth column.

In the present embodiment, the signal processing unit 13 performs focus detection by a phase difference detection method based on the A signal obtained from the photoelectric conversion element D1a and the signal (B signal) obtained from the photoelectric conversion element D1b. . Therefore, the signal processing unit 13 needs to calculate the B signal by subtracting the value of the A signal from the value of the A + B signal (by calculating (A + B signal) − (A signal)). For this reason, the A signal and A + B signal read from the pixel 200 are required. Specifically, as shown in FIG. 7C, the signal processing unit 13 performs G (A) and G (A + B) from the pixel 200 in the second row and the third column to the pixel 200 in the sixth row and the ninth column. This signal is used. For example, the signal processing unit 13 subtracts the signal 704 (G (A)) as the A signal from the signal 705 (G (A + B)) as the A + B signal for the pixel 200 in the second row and the third column (G (A + B ) −G (A) = G (B)) and the B signal may be obtained. In the present embodiment, for example, an example of the first signal is realized by the A signal, and an example of the second signal is realized by the A + B signal.
From the above, in this embodiment, the column signal processing unit 102 reads the A signal and the A + B signal from the pixels in the focus detection region 703 in the pixel region 201 during the imaging operation 801, and A + B signals are read from other pixels.

  FIG. 8 is a diagram illustrating an example of operation timing of the imaging apparatus according to the present embodiment. FIG. 8A shows the imaging operation and moving image display, FIG. 8B shows the image processing operation, and FIG. 8C shows the AF processing operation.

  In FIG. 8A, the operation state of the image capturing unit 101 is represented as an image capturing operation 801, and the operation for displaying a captured image subjected to image processing on the image display unit 17 is represented as a moving image display 802. L in the vertical direction of the graph in FIG. 8A indicates the order of rows in which the captured image is read from the imaging unit 101 and the order of rows in which the imaged captured image is displayed as a moving image. The t in the horizontal direction indicates the passage of time between the timing of reading a row and the timing of a row displaying a moving image. In FIG. 8A, by determining the vertical and horizontal axes in this manner, the state of the row read from the imaging unit 101 and the state of the row for displaying the moving image are the imaging operation 801 and the moving image display, respectively. 802.

In the example shown in FIG. 8A, the inclination of the imaging operation 801 is larger than the inclination of the moving image display 802. Therefore, the imaging operation 801 is faster than the moving image display 802 as the operation speed per unit time.
Here, the data processing amount per unit time of the imaging operation 801 from the start to the completion of reading from the imaging unit 101 is referred to as an imaging rate as necessary. Further, the data processing amount per unit time of the moving image display 802 from the start to the completion of the processing for displaying the photographed image subjected to the image processing is referred to as a moving image rate as necessary.

In the image processing operation shown in FIG. 8B, items that are mainly implemented as operations related to image processing are shown in order from the top in the order of execution. In FIG. 8B, the horizontal direction is a period corresponding to FIG. 8A and indicates the operation period of each item. In the present embodiment, the column signal processing unit 102 reads the A signal and the A + B signal from the pixel 200 of the imaging unit 101 during the period of the imaging operation 801. The signal of the pixel 200 read at this time is the signal of the pixel 200 in the pixel region 201 described with reference to FIG.
Therefore, in FIG. 8B, the period of the imaging operation 801 is a readout period of the A signal and the A + B signal, and is expressed as “A / A + B readout”. In the present embodiment, for example, the column signal processing unit 203 reads out the A signal and the A + B signal from the pixel 200 of the imaging unit 101 during the imaging operation 801, thereby realizing an example of the first reading unit. . Further, for example, an example of the first reading speed is realized by the reading speed corresponding to the imaging rate.

  Simultaneously with the reading operation of the A signal and the A + B signal, the column signal processing unit 102 transfers the A signal and the A + B signal from the imaging substrate 100 to the memory control unit 111 of the memory substrate 110. Then, the memory control unit 111 records the A signal and the A + B signal in the recording unit 112. In FIG. 8B, the period of these operations is represented as “memory writing”. In the present embodiment, for example, the memory control unit 111 records the A signal and the A + B signal in the recording unit 112, thereby realizing an example of the first storage unit. An example of the second writing speed is realized by the writing speed (writing speed corresponding to the imaging rate) when the memory control unit 111 records the A signal and the A + B signal in the recording unit 112.

  Here, in FIG. 8B, the period of signal processing for the captured image in the signal processing unit 13 is represented as “image processing”. The image processing period is determined from the operation specifications of the imaging apparatus based on the processing capability of the signal processing unit 13, the power consumption, and the heat generation amount. In addition, the data processing amount per unit time required for signal processing on a captured image in the signal processing unit 13 is referred to as a processing rate as necessary.

In the present embodiment, a case where the processing rate that is the processing capability of the signal processing unit 13 is lower than the imaging rate in order to read the signal of the pixel 200 from the imaging unit 101 at high speed will be described.
Therefore, the memory control unit 111 reads out the signals of all the pixels 200 in the pixel area 201 recorded in the recording unit 112 of the memory substrate 110 at a processing rate lower than the imaging rate, and reads the signals from the memory substrate 110 to the circuit substrate 120. Transfer to the bus control unit 121. In FIG. 8B, this period is expressed as “memory read”.
In the present embodiment, as described above, the data processing amount conversion (rate conversion) per unit time is performed from the high-speed imaging rate to the low-speed processing rate using the recording unit 112.

The signal of the pixel 200 transferred to the circuit board 120 at the processing rate is subjected to necessary signal preprocessing in the image signal preprocessing unit 122, and the signal processing unit from the image sensor 12 through the image signal output unit 123. 13 is output.
Here, the A + B signal that is an addition signal of the photoelectric conversion elements D1a and D1b of the pixel 200 is used to create a captured image in the signal processing unit 13. Accordingly, the signal of the pixel 200 read from the recording unit 112 may be only the A + B signal as described with reference to FIG. This eliminates the need to output the two signals of the A signal and the A + B signal shown in FIG. 7B from the image sensor 12 to the signal processing unit 13. Therefore, it is possible to reduce the data transfer capacity when outputting from the image sensor 12 to the signal processor 13.

The signal preprocessing performed in the image signal preprocessing unit 122 includes scratch correction and noise reduction processing. However, since the A + B signal is a signal capable of creating a captured image, the image signal output unit 123 may output the A + B signal to the signal processing unit 13 as it is.
In the present embodiment, for example, the memory control unit 111 reads out the signals of all the pixels 200 in the pixel area 201 recorded in the recording unit 112 of the memory substrate 110 at the processing rate, so that the second reading unit An example is realized. Further, for example, an example of the second reading speed is realized by a speed corresponding to the processing rate. In addition, for example, the signal of the pixel stored in the first storage unit based on the A + B signal of all the pixels 200 in the pixel region 201 and used to create an image captured by the imaging unit. An example is realized.

The signal of the captured image that has been image-processed at the processing rate in the signal processing unit 13 is temporarily stored in a buffer (not shown) in the signal processing unit 13 or the synchronization control unit 15. In FIG. 8B, this period is expressed as “buffer writing”.
Then, the signal processing unit 13 or the synchronization control unit 15 reads out the signal of the captured image that has been subjected to image processing during the “image processing” period, and transfers it to the image display unit 17. In FIG. 8B, this period is expressed as “buffer read”.
The captured image signal read from the buffer is displayed on the image display unit 17 during the moving image display 802 period. Therefore, as shown in FIGS. 8A and 8B, the “buffer read” period is the same as the period of the moving image display 802.

At this time, if the image processing is slower than the image display, a problem occurs in the display, and therefore the processing rate must be larger than the moving image rate. Accordingly, in FIG. 8B, the “buffer writing” period, which is the same as the “image processing” period, is also shorter than the “buffer reading” period.
In the present embodiment, for example, the signal processing unit 13 or the synchronization control unit 15 stores an image-processed captured image signal in a buffer, thereby realizing an example of a second storage unit. Further, for example, an example of the first writing speed is realized by the speed at which the signal processing unit 13 or the synchronization control unit 15 writes the signal of the captured image that has undergone image processing into the buffer (the writing speed corresponding to the processing rate). . Further, for example, an example of the second storage medium is realized by a buffer. Further, for example, when the signal processing unit 13 or the synchronization control unit 15 reads out the signal of the captured image subjected to the image processing from the buffer, an example of a fourth reading unit is realized. Further, for example, an example of the third reading speed is realized by the speed at which the signal processing unit 13 or the synchronization control unit 15 reads the signal of the captured image that has undergone image processing from the buffer (reading speed corresponding to the moving image rate). .

In the present embodiment, rate conversion from a high-speed processing rate to a low-speed moving image rate is performed using the buffer of the signal processing unit 13 or the synchronization control unit 15 as described above.
Further, the created captured image can be recorded in the image recording unit 18 according to an instruction from the synchronization control unit 15.

  Here, in the description of FIG. 8B, the signal processing unit 13 or the synchronization control unit 15 has a buffer, but the image display unit 17 may be provided with a buffer. In this case, the image-processed captured image signal is transferred to a buffer in the image display unit 17 at a processing rate, read at a moving image rate, and displayed on the image display unit 17.

In the AF processing operation shown in FIG. 8C, items executed mainly as operations related to the focus detection processing (AF processing) are shown in order from the top in the order of execution. In FIG. 8C, the horizontal direction is a period corresponding to FIG. 8A and indicates the operation period of each item.
“Memory write” to the recording unit 112, which is performed simultaneously with “A / A + B read” and “A / A + B read”, which are the imaging operations 801, is the same item as shown in FIG. Therefore, detailed description thereof is omitted here.

As described with reference to FIG. 7A, the focus detection area 703 is provided in the pixel area 201 in the imaging unit 101. The signal processing unit 13 performs focus detection processing based on the A signal and the A + B signal obtained from the pixels 200 in the focus detection region 703.
At this time, the pixels 200 used for focus detection need not be all the pixels in the focus detection area 703. In the present embodiment, as described with reference to FIG. 7C, the signal processing unit 13 uses the pixels 200 corresponding to the G (green) filters in even-numbered rows in the focus detection region 703 in the focus detection process.

  Here, in order to indicate that the signal of the pixel 200 corresponding to the G (green) filter in the even-numbered row in the focus detection region 703 is at the position of the skipped pixel 200, in FIG. One empty rectangle is expressed as a signal of one pixel 200. In FIG. 8C, the signal (rectangle) of the pixel 200 corresponding to the G (green) filter in the even-numbered row in the focus detection area 703 is expressed as “AF information on memory”. In the following description, the pixels 200 used for the focus detection process (in this embodiment, the pixels 200 corresponding to the G (green) filters in even rows in the focus detection region 703) are referred to as focus detection pixels as necessary. . In this embodiment, for example, the focus detection region 703 realizes an example of a second region that is a region narrower than the first region and includes a pixel that outputs a signal used to detect a focus. Is done. Also, for example, an example of a pixel that outputs a signal used to detect a focus is realized by the focus detection pixel.

In the “A / A + B read” operation, it is understood that all pixels 200 in the focus detection area 703 need to be read in order to read the focus detection pixels. Therefore, when focus detection processing is performed simultaneously with the readout using the signals of the focus detection pixels sequentially read from the focus detection region 703, the focus detection processing is performed over a period in which all the pixels 200 in the focus detection region 703 are read. It is necessary to continue to operate the circuit related to.
However, in the present embodiment, the memory control unit 111 continuously reads out only the focus detection pixel signals recorded in the recording unit 112 of the memory substrate 110 as illustrated in FIG. Transfer to the circuit board 120. At this time, the focus detection pixel signals read from the recording unit 112 are the A signal and the A + B signal. In FIG. 8C, this period is expressed as “continuous reading”. In the present embodiment, for example, the memory control unit 111 continuously reads only the focus detection pixel signal recorded in the recording unit 112 of the memory substrate 110, thereby realizing an example of the third reading unit. . In addition, for example, an A signal and an A + B signal of the focus detection pixel realize an example of a signal that is a part of the pixel signal stored in the first storage unit and is used to detect the focus. .

  The signal of the focus detection pixel transferred to the circuit board 120 is subjected to necessary signal preprocessing in the AF signal preprocessing unit 124, and is sent from the image sensor 12 to the signal processing unit 13 via the AF signal output unit 125. Is output. The signal preprocessing performed in the AF signal preprocessing unit 124 requires, for example, a calculation formula “(A + B signal) − (A signal)” for calculating the B signal. The AF signal preprocessing unit 124 may perform scratch correction, noise reduction processing, and the like before executing this calculation formula. As a result, the AF signal preprocessing unit 124 outputs the A signal and the B signal as the focus detection pixel signals. In FIG. 8C, this period is expressed as “A / B processing”.

A focus detection processing unit (not shown) included in the signal processing unit 13 includes different photoelectric conversion elements Ta1 obtained by dividing the intra-pixel pupil based on the A signal and B signal for focus detection related to a plurality of pixels in the horizontal direction. The phase difference of incident light with respect to Ta2 is detected.
Next, the focus detection processing unit calculates an AF evaluation value that is an evaluation value corresponding to distance information from the imaging device to the subject and used for AF based on the detected phase difference, and performs synchronization. Output to the control unit 15. In the present embodiment, for example, an example of a detection unit is realized by using a focus detection processing unit.

The focus detection processing executed by the signal processing unit 13 has been described so far. In FIG. 8C, this period is represented as “AF evaluation value calculation”.
The synchronization control unit 15 drives the focusing lens included in the imaging optical system 11 based on the AF evaluation value, and controls the subject to focus. In FIG. 8C, shifting to control by the synchronization control unit 15 is expressed as “AF control”.

The focus detection processing unit of the signal processing unit 13 continuously performs focus detection processing based on the signals of the focus detection pixels that have been “continuously read”. Here, the data processing amount per unit time required for the focus detection processing of the focus detection pixels is referred to as an AF rate as necessary.
In the AF processing operation of FIG. 8C, the AF signal preprocessing unit 124 uses the A signal and the A + B signal of the focus detection pixel continuously read from the recording unit 112 at the AF rate, and the A signal of the focus detection pixel. And B signal. Then, the focus detection processing unit of the signal processing unit 13 performs focus detection processing at the AF rate, calculates an AF evaluation value, and outputs the AF evaluation value to the synchronization control unit 15. Therefore, in FIG. 8C, the “continuous reading” period, the “A / B processing” period, and the “AF evaluation value calculation” period are equal periods. Accordingly, at this time, the recording unit 112 is used to convert the rate from a low imaging rate to a high AF rate.

  As described above, in the present embodiment, it is possible to simultaneously acquire a focus detection signal and a captured image signal in one imaging operation. In the imaging operation, the processing rate for the image processing of the captured image is made slower than the imaging rate for reading the signal of the pixel 200 of the imaging unit 101 and transferring it to the recording unit 112. Therefore, the image processing period can be determined based on the processing capability of the signal processing unit 13, the power consumption, and the heat generation amount. Furthermore, only the signal of the focus detection pixel in the focus detection area 703 is read out continuously. Therefore, in the imaging operation, the focus detection processing unit of the signal processing unit 13, the AF signal preprocessing unit 124, and the AF are compared with the case where the focus detection process is performed simultaneously while reading all the pixels in the focus detection region 703. The operation period of the signal output unit 125 can be shortened. Therefore, power consumption of the imaging device can be reduced. Further, since the transfer period of the focus detection pixel signal from the image sensor 12 to the signal processing unit 13 is shortened, the time required for the focus detection process can be greatly shortened. In addition, since only the A + B signal is output from the image pickup device 12 to the signal processing unit 13 for the captured image, the processing rate can be further reduced and the data transfer capacity can be reduced. In addition, by recording the pixel signal of the imaging unit 101 in the recording unit 112, there is no restriction on the processing rate for image processing, and the imaging operation can be performed at high speed, so that rolling distortion can be reduced. As described above, in the present embodiment, when an imaging apparatus including pixels having the intra-pixel pupil division function performs focus detection by the phase difference detection method and creation of a subject image, the reduction of the frame rate and the power consumption are reduced. Reductions can be made possible.

  In this embodiment, the case where a signal is read from the recording unit 112 at a processing rate to perform image processing, and a signal is read from the buffer at a moving image rate has been described as an example. However, this is not always necessary. For example, a signal may be read from the recording unit 112 at a moving image rate and image processing may be performed and displayed on the image display unit 17. In this way, since image processing can be performed at a moving image rate slower than the processing rate, the data transfer rate can be further reduced. It is also possible to omit a buffer for displaying images. In the present embodiment, for example, an example of the second reading speed (speed at which an image based on the signal stored in the first storage unit is output to the output destination) is realized by the moving image rate. The image display unit 17 is an example of an output destination. However, the output destination of the image-processed signal is not limited to the image display unit 17. For example, the output destination of the image-processed signal may be the image recording unit 18.

  Further, as in the present embodiment, if the A signal and the A + B signal are read from the pixels in the focus detection region 703 in the pixel region 201 and the A + B signal is read from the other pixels in the pixel region 201, the readout is performed. This is preferable because the number of signals can be reduced. However, this is not always necessary. For example, the A signal and the A + B signal may be read from all the pixels in the pixel region 201.

(Second Embodiment)
Next, a second embodiment will be described with reference to FIGS. 9 to 10 in addition to FIGS. In the first embodiment, an A signal and an A + B signal (one focus detection signal and a captured image signal of two photoelectric conversion elements) are read from the pixel 200 of the imaging unit 101, and a phase difference detection method is used. The case where the focus detection and the creation of the captured image are performed simultaneously has been described as an example. On the other hand, in the present embodiment, the A signal and the B signal (the signals for detecting the focus of both photoelectric conversion elements) are read from the pixel 200 of the imaging unit 101, and the focus detection and photographing by the phase difference detection method are performed. A case where image creation is performed simultaneously will be described as an example. As described above, the present embodiment and the first embodiment mainly differ in processing due to different signals read from the pixels 200 of the imaging unit 101. Therefore, in the description of the present embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals as those in FIGS.

In the present embodiment, after performing the same first exposure control (exposure control of the photoelectric conversion element D1a) as in the first embodiment, the second exposure control (photoelectric conversion element D1b) different from the first embodiment is performed. Exposure control).
In the second exposure control of the present embodiment, only a signal charge of the photoelectric conversion element D1b is read by turning on and off the reset transistor T2 and then turning on and off the transfer transistor T1b after a predetermined exposure time has elapsed. It is. The output signal of the photoelectric conversion element D1b at this time is referred to as a focus detection B signal. In the second readout operation of the present embodiment, column signal processing is performed on the signal of the photoelectric conversion element D1b read out from the pixels 200 for one row subjected to the second exposure control, that is, the B signal. .

FIG. 9 is a diagram illustrating an example of signal processing according to the present embodiment. FIG. 9A shows the A signal and the B signal read from the pixel 200 in the pixel region 201 in the imaging unit 101. In the present embodiment, the A signal or the B signal is read from the pixels 200 in the pixel area 201 in a predetermined order for each row.
For example, the signal 901 is an A signal of the pixel 200 corresponding to the R (red) filter in the first row and the first column of the pixel region 201 and is represented as R (A). A signal 902 is a B signal of the pixel 200 corresponding to the R (red) filter in the first row and the first column of the pixel region 201 and is represented as R (B). Further, a region 903 that surrounds the pixel 200 in the seventh row and the ninth column from the pixel 200 in the second row and the second column, which is a diagonal, is set as a focus detection region. In FIG. 9A, the A + B signal is simply replaced with the B signal in FIG. 7A, and the detailed description of FIG.

FIG. 9B shows an A signal and a B signal of the pixel 200 used for creating a captured image in the signal processing unit 13 in FIG.
In the present embodiment, the signal processing unit 13 uses all the signals of all the pixels 200 in the pixel region 201 when creating a captured image. That is, the signal processing unit 13 creates a captured image using the A + B signal that is an addition signal of the photoelectric conversion elements D1a and D1b. Therefore, the signal processing unit 13 adds the A signal and the B signal (executes the calculation formula of (A signal) + (B signal)) to calculate the A + B signal. And B signals are required. For example, the signal processing unit 13 adds the signal 901 (R (A)) that is the A signal and the signal 902 (R (B)) that is the B signal to the pixel 200 in the first row and the first column (R (A) + R (B) = R (A + B)) A + B signal may be obtained.

FIG. 9C shows a signal of the pixel 200 related to the focus detection process (AF process) in the signal processing unit 13 in FIG.
Also in the present embodiment, as in the first embodiment, in the focus detection process (AF process), the signal processing unit 13 outputs the signal of the pixel 200 corresponding to the G (green) filter in the even-numbered row in the focus detection area 903. The case where is used will be described as an example. In FIG. 9C, only the A + B signal is replaced with the B signal in FIG. 7C, and the detailed description of FIG. 9C is omitted.

  In the present embodiment, the signal processing unit 13 performs focus detection by a phase difference detection method based on the A signal obtained from the photoelectric conversion element D1a and the B signal obtained from the photoelectric conversion element D1b. For example, in FIG. 9C, the signal processing unit 13 performs the signal 904 (G (A)) that is the A signal and the signal 905 (G (B)) that is the B signal for the pixel in the second row and the third column. The focus detection by the phase difference detection method is performed using the signal as it is. In the present embodiment, for example, an example of the first signal is realized by the A signal, and an example of the second signal is realized by the B signal.

  FIG. 10 is a diagram illustrating an example of operation timing of the imaging apparatus according to the present embodiment. 10A shows the imaging operation and moving image display, FIG. 10B shows the image processing operation, and FIG. 10C shows the AF processing operation. Since FIG. 10A is the same as the imaging operation and the moving image display shown in FIG. 8A, detailed description of FIG. 10A is omitted.

In the image processing operation shown in FIG. 10B, items executed mainly as operations related to image processing are shown in order from the top in the order of execution. In FIG. 10B, the horizontal direction is a period corresponding to FIG. 10A and indicates the operation period of each item. In the present embodiment, the A signal and the B signal are read from the pixel 200 of the imaging unit 101 during the period of the imaging operation 801. The signal of the pixel 200 read at this time is the signal of all the pixels 200 in the pixel area 201 described with reference to FIG.
Therefore, in FIG. 10B, the period of the imaging operation 801 is set as the readout period of the A signal and the B signal, and is expressed as “A / B readout”. In the present embodiment, for example, the column signal processing unit 203 reads an A signal and a B signal from the pixel 200 of the imaging unit 101 during the imaging operation 801, thereby realizing an example of a first reading unit. . Further, for example, an example of the first reading speed is realized by the reading speed corresponding to the imaging rate.

  Simultaneously with the reading operation of the A signal and the B signal, the column signal processing unit 102 transfers the A signal and the B signal from the imaging substrate 100 to the memory substrate 110. Then, the memory control unit 111 records the A signal and the B signal in the recording unit 112. In FIG. 10B, the period of these operations is represented as “memory writing”. In the present embodiment, for example, the memory control unit 111 records the A signal and the B signal in the recording unit 112, thereby realizing an example of the first storage unit. In addition, an example of the second writing speed is realized by the writing speed (writing speed corresponding to the imaging rate) when the memory control unit 111 records the A signal and the B signal in the recording unit 112.

Also in the present embodiment, as in the first embodiment, a case where the processing rate that is the processing capability of the signal processing unit 13 is lower than the imaging rate in order to read the signal of the pixel 200 from the imaging unit 101 at high speed will be described.
Therefore, the memory control unit 111 reads out the signals of all the pixels 200 in the pixel area 201 recorded in the recording unit 112 of the memory substrate 110 at a processing rate lower than the imaging rate, and reads the signals from the memory substrate 110 to the circuit substrate 120. Transfer to the bus control unit 121. In FIG. 10B, this period is expressed as “memory read”.
In the present embodiment, rate conversion from a high imaging rate to a low processing rate is performed using the recording unit 112 as described above.

The signal of the pixel 200 transferred to the circuit board 120 at the processing rate is subjected to necessary signal preprocessing in the image signal preprocessing unit 122, and the signal processing unit from the image sensor 12 through the image signal output unit 123. 13 is output.
Here, the A + B signal that is an addition signal of the photoelectric conversion elements D1a and D1b of the pixel 200 is used to create a captured image in the signal processing unit 13. Therefore, the signals of the pixels 200 read from the recording unit 112 are the A signal and the B signal as described with reference to FIG. The signal preprocessing performed in the image signal preprocessing unit 122 requires a calculation formula “(A signal) + (B signal)” for calculating the A + B signal. The image signal preprocessing unit 122 may perform defect correction, noise reduction processing, and the like before executing this calculation formula. As a result, the A + B signal is output from the image signal preprocessing unit 122 as a signal for creating a captured image. In FIG. 10B, this period is expressed as “A + B processing”. In the present embodiment, the image signal preprocessing unit 122 calculates an A + B signal, thereby realizing an example of an adding unit.

This eliminates the need to output the two signals A and B shown in FIG. 9B from the image sensor 12 to the signal processor 13, and therefore data when the image sensor 12 outputs to the signal processor 13. Transfer capacity can be reduced.
In the present embodiment, for example, the memory control unit 111 reads out the signals of all the pixels 200 in the pixel area 201 recorded in the recording unit 112 of the memory substrate 110 at the processing rate, so that the second reading unit An example is realized. Further, for example, an example of the second reading speed is realized by a speed corresponding to the processing rate. Further, for example, an example of a signal of the pixel stored in the first storage unit based on the signal of all the pixels 200 in the pixel region 201 and used to create an image captured by the imaging unit. Is realized.

The signal of the captured image that has been image-processed at the processing rate in the signal processing unit 13 is temporarily stored in a buffer (not shown) in the signal processing unit 13 or the synchronization control unit 15. In FIG. 10B, this period is expressed as “buffer writing”.
Then, the signal processing unit 13 or the synchronization control unit 15 reads out the signal of the captured image that has been subjected to image processing during the “image processing” period, and transfers it to the image display unit 17. In FIG. 10B, this period is expressed as “buffer read”.
The captured image signal read from the buffer is displayed on the image display unit 17 during the moving image display 802 period. Therefore, as shown in FIGS. 10A and 10B, the “buffer read” period is the same as the period of the moving image display 802.

At this time, if the image processing is slower than the image display, a problem occurs in the display, and therefore the processing rate must be larger than the moving image rate. Accordingly, in FIG. 10B, the “buffer writing” period, which is the same as the “image processing” period, is shorter than the “buffer reading” period.
In the present embodiment, for example, the signal processing unit 13 or the synchronization control unit 15 stores an image-processed captured image signal in a buffer, thereby realizing an example of a second storage unit. Further, for example, an example of the first writing speed is realized by the speed at which the signal processing unit 13 or the synchronization control unit 15 writes the signal of the captured image that has undergone image processing into the buffer (the writing speed corresponding to the processing rate). . Further, for example, an example of the second storage medium is realized by a buffer. Further, for example, when the signal processing unit 13 or the synchronization control unit 15 reads out the signal of the captured image subjected to the image processing from the buffer, an example of a fourth reading unit is realized. Further, for example, an example of the third reading speed is realized by the speed at which the signal processing unit 13 or the synchronization control unit 15 reads the signal of the captured image that has undergone image processing from the buffer (reading speed corresponding to the moving image rate). .
In the present embodiment, rate conversion from a high-speed processing rate to a low-speed moving image rate is performed using the buffer of the signal processing unit 13 or the synchronization control unit 15 as described above.
Further, the created captured image can be recorded in the image recording unit 18 according to an instruction from the synchronization control unit 15.
Here, in the description of FIG. 10B, the signal processing unit 13 or the synchronization control unit 15 has a buffer, but the image display unit 17 may be provided with a buffer. In this case, the image-processed captured image signal is transferred to a buffer in the image display unit 17 at a processing rate, read at a moving image rate, and displayed on the image display unit 17.

In the AF processing operation shown in FIG. 10C, items executed mainly as operations related to the focus detection processing (AF processing) are shown in order from the top in the order of execution. In FIG. 10C, the horizontal direction is a period corresponding to FIG. 10A and indicates the operation period of each item.
“Memory writing” to the recording unit 112 performed simultaneously with “A / B reading” as the imaging operation 801 is the same item as shown in FIG. Therefore, detailed description thereof is omitted here.
As shown in FIG. 9A, a focus detection area 903 is provided in the pixel area 201 in the imaging unit 101. The signal processing unit 13 performs focus detection processing based on the A signal and the B signal obtained from the pixels 200 in the focus detection region 903.
At this time, the pixels 200 (focus detection pixels) used for focus detection need not be all the pixels in the focus detection region 903. In the present embodiment, as illustrated in FIG. 9C, the signal processing unit 13 uses the pixels 200 corresponding to the G (green) filters in even rows in the focus detection region 903.

  Here, in order to indicate that the signal of the focus detection pixel (the pixel 200 corresponding to the G (green) filter of the even-numbered row in the focus detection region 903) is at the position of the skipped pixel 200, FIG. Then, one rectangle with a time interval is expressed as a signal of one pixel 200. In FIG. 10C, the signal (rectangle) of the focus detection pixel is represented as “AF information on memory”. In the present embodiment, for example, the focus detection region 903 realizes an example of a second region that is a region narrower than the first region and includes a pixel that outputs a signal used to detect a focus. Is done. Also, for example, an example of a pixel that outputs a signal used to detect a focus is realized by the focus detection pixel.

In the “A / B read” operation, it is understood that all pixels in the focus detection area 903 need to be read in order to read the focus detection pixels. Therefore, when the focus detection process is performed simultaneously with the readout using the signals of the focus detection pixels sequentially read from the focus detection area 903, the focus detection process is performed over a period of reading all the pixels 200 in the focus detection area 903. It is necessary to continue to operate the circuit related to.
However, in this embodiment as well, as in the first embodiment, the memory control unit 111 continuously outputs only the focus detection pixel signals recorded in the recording unit 112 of the memory substrate 110 as shown in FIG. The data is read and transferred from the memory board 110 to the circuit board 120. At this time, the focus detection pixel signals read from the recording unit 112 are the A signal and the B signal. In FIG. 10C, this period is expressed as “continuous reading”. In the present embodiment, for example, the memory control unit 111 continuously reads only the focus detection pixel signal recorded in the recording unit 112 of the memory substrate 110, thereby realizing an example of the third reading unit. . Further, for example, the A signal and the B signal of the focus detection pixel realize an example of a signal that is a part of the pixel signal stored in the first storage unit and is used for detecting the focus. .

  The signal of the focus detection pixel transferred to the circuit board 120 is subjected to necessary signal preprocessing in the AF signal preprocessing unit 124, and is sent from the image sensor 12 to the signal processing unit 13 via the AF signal output unit 125. Is output. Signal preprocessing performed in the AF signal preprocessing unit 124 includes scratch correction, noise reduction processing, and the like. However, since the A signal and the B signal are signals that can be subjected to focus detection processing, the AF signal output unit 125 may output the A signal and the B signal to the signal processing unit 13 as they are. As a result, the AF signal preprocessing unit 124 outputs the A signal and the B signal as the focus detection pixel signals.

A focus detection processing unit (not shown) included in the signal processing unit 13 includes different photoelectric conversion elements Ta1 obtained by dividing the intra-pixel pupil based on the A signal and B signal for focus detection related to a plurality of pixels in the horizontal direction. The phase difference of incident light with respect to Ta2 is detected.
Next, the focus detection processing unit calculates an AF evaluation value that is an evaluation value corresponding to distance information from the imaging device to the subject and used for AF based on the detected phase difference, and performs synchronization. Output to the control unit 15. In the present embodiment, for example, an example of a detection unit is realized by using a focus detection processing unit.

Up to this point, focus detection processing is executed by the signal processing unit 13, and in FIG. 10C, this period is represented as “AF evaluation value calculation”.
The synchronization control unit 15 drives the focusing lens included in the imaging optical system 11 based on the AF evaluation value, and controls the subject to focus. In FIG. 10C, the shift to the control by the synchronization control unit 15 is represented as “AF control”.

The focus detection processing unit of the signal processing unit 13 continuously performs focus detection processing based on the signals of the focus detection pixels that have been “continuously read”. Here, the data processing amount per unit time required for the focus detection processing of the focus detection pixels is referred to as an AF rate as necessary.
In the AF processing operation of FIG. 10C, the focus detection processing unit of the signal processing unit 13 uses the A signal and the B signal of the focus detection pixels continuously read from the recording unit 112 at the AF rate to perform AF. Focus detection processing is performed at the rate, and an AF evaluation value is calculated. Then, the focus detection processing unit outputs the AF evaluation value to the synchronization control unit 15. For this reason, in FIG. 10C, the period of “continuous reading” and the period of “AF evaluation value calculation” are equal. Accordingly, at this time, the recording unit 112 is used to convert the rate from a low imaging rate to a high AF rate.

  As described above, in the present embodiment, it is possible to simultaneously acquire a focus detection signal and a captured image signal in one imaging operation. In the imaging operation, the processing rate for the image processing of the captured image is made slower than the imaging rate for reading the signal of the pixel 200 of the imaging unit 101 and transferring it to the recording unit 112. Therefore, the image processing period can be determined based on the processing capability of the signal processing unit 13, the power consumption, and the heat generation amount. Furthermore, only the signal of the focus detection pixel in the focus detection area 903 is read out continuously. Therefore, in the imaging operation, the operations of the focus detection processing unit of the signal processing unit 13 and the AF signal preprocessing unit 124 are compared with the case where the focus detection processing is performed simultaneously while reading all the pixels in the focus detection region 903. The period can be shortened. Therefore, power consumption of the imaging device can be reduced. In addition, since the transfer period of the focus detection signal from the image sensor 12 to the signal processing unit 13 is shortened, the time required for the focus detection process can be significantly shortened. In addition, since only the A + B signal is output from the image sensor 12 to the signal processing unit 13, the processing rate can be further reduced and the data transfer capacity can be reduced. In addition, by recording the pixel signal of the imaging unit 101 in the recording unit 112, there is no restriction on the processing rate for image processing, and the imaging operation can be performed at high speed, so that rolling distortion can be reduced. As described above, in this embodiment as well, in the same way as in the first embodiment, when an imaging apparatus including a pixel having an in-pixel pupil division function performs focus detection using a phase difference detection method and creation of a subject image, a frame It is possible to suppress rate reduction and reduce power consumption.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  11: imaging optical system, 12: imaging device, 13: signal processing unit, 15: synchronization control unit, 17: image display unit

Claims (20)

Imaging means comprising a plurality of pixels each having a plurality of photoelectric conversion units;
First reading means for reading signals of the pixels in a first area corresponding to an image picked up by the image pickup means;
First storage means for storing a signal of the pixel read by the first reading means in a first storage medium;
Second readout means for reading out the signal of the pixel stored by the first storage means and used to create an image captured by the imaging means;
A third readout unit that continuously reads out a signal that is a part of the signal of the pixel stored in the first storage unit and that is used to detect a focus;
The second readout speed, which is the readout speed of the signal by the second readout means, is slower than the first readout speed, which is the readout speed of the signal of the pixel by the first readout means. An imaging device. The plurality of photoelectric conversion units include a first photoelectric conversion unit and a second photoelectric conversion unit,
The signal of the pixel is obtained by adding a first signal that is a signal output from the first photoelectric conversion unit and a signal output from the first photoelectric conversion unit and the second photoelectric conversion unit. The imaging apparatus according to claim 1, further comprising at least one of a signal and a second signal that is a signal output from the second photoelectric conversion unit.   The imaging apparatus according to claim 2, wherein the first readout unit reads out the first signal and the second signal at the first readout speed. The second signal is a signal obtained by adding signals output from the first photoelectric conversion unit and the second photoelectric conversion unit,
The first reading means is an area narrower than the first area, and the pixel in the second area, which is an area including the pixel that outputs a signal used to detect the focus, The first signal and the second signal are read at the first readout speed, and the second signal is output from the pixel in a region different from the second region in the first region. 4. The imaging apparatus according to claim 2, wherein reading is performed at a first reading speed. The second signal is a signal obtained by adding signals output from the first photoelectric conversion unit and the second photoelectric conversion unit,
The second reading means reads the second signal at the second reading speed as a signal used to create an image picked up by the image pickup means. The imaging device according to any one of the above. The second signal is a signal obtained by adding signals output from the first photoelectric conversion unit and the second photoelectric conversion unit,
The generator according to any one of claims 2 to 5, further comprising generating means for generating a pixel signal output from the second photoelectric conversion unit based on the first signal and the second signal. The imaging apparatus according to item 1. The second signal is a signal output from the second photoelectric conversion unit,
The second reading means reads the first signal and the second signal as the signals used for creating the image picked up by the image pickup means at the second reading speed. The imaging device according to claim 2 or 3. The second signal is a signal output from the second photoelectric conversion unit,
8. The apparatus according to claim 2, further comprising an adding unit that adds the first signal and the second signal read by the second reading unit. 9. Imaging device.   The said 3rd reading means reads the said 1st signal and the said 2nd signal continuously as a signal used in order to detect the said focus, The any one of Claims 2-8 characterized by the above-mentioned. The imaging device described in 1.   3. The image processing apparatus according to claim 2, further comprising: an image processing unit that performs image processing on a signal obtained by adding the first signal and the second signal to generate a signal of an image captured by the imaging unit. The imaging apparatus according to any one of 9. Second storage means for storing the signal generated by the image processing means in a second storage medium at a first writing speed slower than the first reading speed;
11. The apparatus according to claim 10, further comprising a fourth reading unit that reads the signal stored by the second storage unit at a third reading speed that is slower than the first reading speed. Imaging device.   The imaging apparatus according to claim 11, wherein the third reading speed is slower than the second reading speed and the first writing speed.   The imaging apparatus according to claim 11, wherein the second reading speed and the first writing speed are the same.   11. The second reading speed is a speed corresponding to a speed at which an image based on the signal stored in the first storage means is output to an output destination. The imaging apparatus of Claim 1.   15. The first writing speed is the same as a second writing speed, which is a writing speed of the pixel signal from the first storage means to the first storage medium. The imaging device according to any one of the above.   It further has a detection means for detecting a phase difference between the signal output from the first photoelectric conversion unit and the signal output from the second photoelectric conversion unit as a signal used for detecting the focal point. The imaging device according to any one of claims 1 to 15. The imaging means includes a first substrate including the plurality of pixels;
A second substrate comprising the first storage medium; and a third substrate comprising output means for outputting the signal read by the second readout means and the third readout means to the outside of the imaging means. And having
The first substrate, the second substrate, and the third substrate are stacked in this order, and are electrically connected to each other. The imaging device according to item.   The imaging device according to claim 17, wherein the plurality of pixels are arranged on a surface of the first substrate on a side on which subject light is incident. A method for controlling an image pickup apparatus having an image pickup unit including a plurality of pixels each having a plurality of photoelectric conversion units,
A first reading step of reading a signal of the pixel in a first area corresponding to an image picked up by the image pickup means;
A first storage step of storing the signal of the pixel read out in the first readout step in a first storage medium;
A second readout step of reading out the signal of the pixel stored in the first storage step and used to create an image captured by the imaging unit;
A third readout step of continuously reading out a signal that is a part of the signal of the pixel stored in the first storage step and that is used for detecting a focus;
The second readout speed, which is the readout speed of the signal in the second readout process, is slower than the first readout speed, which is the readout speed of the pixel signal in the first readout process. A method for controlling the imaging apparatus. A program for causing a computer to control an imaging device having an imaging means including a plurality of pixels each having a plurality of photoelectric conversion units,
A first reading step of reading a signal of the pixel in a first area corresponding to an image picked up by the image pickup means;
A first storage step of storing the signal of the pixel read out in the first readout step in a first storage medium;
A second readout step of reading out the signal of the pixel stored in the first storage step and used to create an image captured by the imaging unit;
A third readout step of continuously reading out a signal that is a part of the signal of the pixel stored in the first storage step and is used for detecting a focus;
The second readout speed, which is the readout speed of the signal in the second readout process, is slower than the first readout speed, which is the readout speed of the pixel signal in the first readout process. Program.

Images