JP6883082B2 - Imaging device and its control method - Google Patents

Description

The present invention relates to an image pickup apparatus and a control method thereof.

In recent years, image pickup devices using image pickup devices such as CMOS sensors have become more multifunctional, and subjects obtained by the image pickup device not only for generating captured images such as still images / moving images but also for controlling the image pickup device such as focus adjustment. It is designed to be informed.

For example, Patent Document 1 discloses a technique capable of pupil division type focus detection using a signal obtained from an image sensor. In Patent Document 1, by providing one microlens and two photodiodes for each pixel of the image sensor, each photodiode receives light that has passed through different pupils of the photographing lens. Focus detection is possible by comparing the output signals from these two photodiodes, and it is also possible to generate an captured image by adding the output signals from the two photodiodes.

Japanese Unexamined Patent Publication No. 2001-124984

However, when each pixel has a plurality of photodiodes as in Patent Document 1, it takes a long time to read the signals of all the pixels.

An object of the present invention is to provide an image pickup apparatus capable of obtaining good image quality while suppressing an increase in reading time and a control method thereof.

According to one aspect of the present invention, an image sensor having a plurality of microlenses arranged in the matrix direction and a pixel region in which a plurality of unit pixels are arranged in the matrix direction corresponding to each of the plurality of microlenses. Each of the unit pixels is generated in an image sensor including a first photoelectric conversion unit and a second photoelectric conversion unit, and in the first photoelectric conversion unit and the second photoelectric conversion unit, respectively. A first mode, which is a mode for reading the signal corresponding to the charged charge, and a second mode, which is a mode for reading the signal corresponding to the charge generated by the first photoelectric conversion unit. The control means includes a control means capable of taking at least the above, and the control means performs the reading in the first mode with respect to the first row located in the first region of the pixel region. For the second row located in the second region of the pixel region different from the first region, the read in the second mode and the read in the first mode The control means resets the charges generated by the first photoelectric conversion unit and the second photoelectric conversion unit of each row included in the pixel region, respectively, in the first mode. The number of signals read from the image sensor in the first mode for the second row by the control means , which is controlled at a timing synchronized with the reading in the second mode, is the second mode. in the equal his rather the number of signals read from the imaging device, the first number of signals read from the imaging device for line, characterized in that less than the number of signals read from the imaging device for said second row An image sensor is provided.

According to the present invention, it is possible to provide an image pickup apparatus capable of obtaining good image quality and a control method thereof while suppressing an increase in reading time for acquiring a signal for focus detection.

It is an overall block diagram of the image pickup apparatus according to 1st Embodiment of this invention. It is a figure which shows typically the pixel arrangement of the image sensor according to 1st Embodiment of this invention. It is a figure which shows typically the relationship between the light flux emitted from the exit pupil of a photographing lens, and a pixel. It is a figure for demonstrating the relationship between the focus adjustment state and the correlation of an image signal. It is a figure which shows the whole structure of the image sensor according to 1st Embodiment of this invention. It is a figure which shows the circuit structure of the unit pixel of the image pickup device according to 1st Embodiment of this invention. It is a figure which shows the structure of the reading circuit of each unit pixel string of the image pickup device by 1st Embodiment of this invention. It is a figure which shows the distance measuring frame set with respect to the pixel arrangement of the image sensor according to 1st Embodiment of this invention. It is a figure which shows the timing chart of the reading operation of the unit pixel line of the image pickup device in the image pickup apparatus according to 1st Embodiment of this invention. It is a figure which shows the timing chart of the reading operation of the unit pixel line of the image pickup device in the image pickup apparatus by 1st Embodiment of this invention. It is a figure which shows the timing chart of the reading operation of the unit pixel line of the image pickup device in the image pickup apparatus according to 1st Embodiment of this invention. It is a figure which shows the timing chart of the slit rolling operation of the image pickup device in the image pickup apparatus according to 1st Embodiment of this invention. It is a figure which shows the timing chart of the slit rolling operation of the conventional image sensor. It is a figure which shows the timing chart of an inappropriate slit rolling operation. It is a figure for demonstrating the setting of the distance measuring frame in the image pickup apparatus according to 2nd Embodiment of this invention. It is a figure which shows the flowchart of the still image shooting operation in the image pickup apparatus according to 3rd Embodiment of this invention. It is a figure which shows the timing chart of the reading operation in the still image shooting operation in the image pickup apparatus according to 3rd Embodiment of this invention. It is a figure which shows the timing chart of the slit rolling operation in the image pickup apparatus according to 4th Embodiment of this invention. It is a figure which shows the timing chart of the reading operation of the unit pixel line of the image pickup device in the image pickup apparatus according to 4th Embodiment of this invention. It is a figure which shows the timing chart of the reading operation of the unit pixel line of the image pickup device in the image pickup apparatus according to 4th Embodiment of this invention.

When each pixel has a plurality of photodiodes as in Patent Document 1, it takes a long time to read the signals of all the pixels.

Therefore, in the pixel row used for the focus detection process, the signal is read independently for each photodiode of each pixel, and in the pixel area where the focus detection process is not performed, the photodiode signal is added within the pixel to generate an image. Read only the signal. By doing so, it is possible to suppress an increase in the reading time.

However, in that case, the pixel row used for focus detection and the pixel row not used for focus detection differ in the time required for reading. Therefore, in the slit rolling operation, which is a general operation during live view and movie shooting, a phenomenon occurs in which the accumulation time differs depending on the pixel row and the exposure amount differs (hereinafter, referred to as "exposure amount deviation"). It ends up.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[First Embodiment]
The image pickup apparatus according to the first embodiment will be described with reference to the drawings. FIG. 1 is a diagram showing an overall configuration of an image pickup apparatus 100 according to the present embodiment.

As shown in FIG. 1, the first lens group 101 arranged at the tip of the imaging optical system is held so as to be able to move forward and backward in the optical axis direction. The aperture 102 adjusts the amount of light at the time of photographing by adjusting the aperture diameter thereof. The second lens group 103 performs a magnification-changing action (zoom function) in conjunction with the advancing / retreating operation of the first lens group 101. The third lens group 104 adjusts the focus by moving forward and backward in the optical axis direction.

The optical low-pass filter 105 is an optical element for reducing false color and moire of a captured image. The image sensor (imaging unit) 106 generates an imaging signal (pixel signal) by photoelectric conversion (imaging) of the subject image imaged by the lens groups 101, 103, 104. Here, it is assumed that a Bayer array CMOS image sensor is used for the image sensor 106.

The analog image signal output from the image sensor 106 is converted into a digital signal (image data) by the AFE (Analog Front End) 107, input to the DFE (Digital Front End) 108, and subjected to predetermined arithmetic processing. The DSP (Digital Signal Processor) 109 performs correction processing, development processing, and the like on the image data output from the DFE 108. The DSP 109 also performs AF (autofocus) calculation for calculating the amount of defocus from the image data.

Image data is recorded on the recording medium 110. The display unit 111 is for displaying captured images, various menu screens, and the like, and a liquid crystal display (LCD) or the like is used.

The RAM 112 temporarily stores image data and the like, and is connected to the DSP 109. The timing generator (TG) 113 supplies a drive signal to the image sensor 106.

The CPU (controller, control unit) 114 controls the AFE107, DFE108, DSP109, TG113, and the aperture drive circuit 115. Further, the CPU 114 controls the focus drive circuit 116 based on the AF calculation result of the DSP 109. These controls performed by the CPU 114 are realized by the CPU 114 reading and executing a control program stored in the ROM 119 or a memory (not shown).

The diaphragm drive circuit 115 drives the diaphragm 102 by driving and controlling the diaphragm actuator 117. The focus drive circuit 116 drives and controls the focus actuator 118 to move the third lens group 104 forward and backward in the optical axis direction, thereby adjusting the focus. The ROM 119 stores various correction data and the like. The mechanical shutter 120 controls the amount of exposure to the image sensor 106 during still image shooting. The mechanical shutter 120 keeps the open state during the live view operation or the moving image shooting, and keeps exposing the image sensor 106. The shutter drive circuit 121 controls the mechanical shutter 120.

FIG. 2 is a diagram schematically showing the pixel arrangement of the image sensor 106. As shown in FIG. 2, the unit pixels 200 are arranged in a matrix (two-dimensional, matrix direction), that is, in a matrix, and R (Red) / G (Green) / B (for each unit pixel 200). Blue) color filters are arranged in a Bayer shape. Further, sub-pixels a and sub-pixels b are arranged in each unit pixel 200, and photodiodes (hereinafter referred to as PDs) 201a and 201b are arranged in the sub-pixels a and b, respectively. Each image pickup signal output from the sub-pixels a and b is used for focus detection, and the a / b composite signal, which is a signal obtained by adding the image pickup signals output from the sub-pixel a and the sub-pixel b, is used for image generation. Will be done.

FIG. 3 is a diagram schematically showing the relationship between the light flux emitted from the exit pupil of the photographing lens composed of the first to third lens groups 101, 103, 104 and the aperture 102 and the unit pixel 200. In FIG. 3, the same parts as those in FIG. 2 are shown with the same reference numerals.

As shown in FIG. 3, the color filter 301 and the microlens 302 are formed on each unit pixel 200. The light that has passed through the exit pupil 303 of the photographing lens is incident on the unit pixel 200 about the optical axis 304. The luminous flux that passes through the pupil region 305, which is a part of the exit pupil 303 of the photographing lens, passes through the microlens 302 and is received by the sub-pixel a. On the other hand, the luminous flux passing through the pupil region 306, which is another part of the exit pupil 303, is received by the sub-pixel b through the microlens 302. Therefore, the sub-pixel a and the sub-pixel b receive the light of the separate pupil regions 305 and 306 of the exit pupil 303 of the photographing lens, respectively. Therefore, the focus detection of the phase difference method becomes possible by comparing the output signals of the sub-pixel a and the sub-pixel b.

FIG. 4 is a diagram showing the correlation between the image signal waveform 401 obtained from the sub-pixel a and the image signal waveform 402 obtained from the sub-pixel b for different focal states. As shown in FIG. 4A, when the image signal waveforms are out of focus, the image signal waveforms 401 and 402 obtained from the sub-pixels a and b do not match, and the image signal waveforms 401 and 402 are largely deviated from each other. As the focus approaches, as shown in FIG. 4B, the deviation between the image signal waveforms 401 and 402 becomes smaller, and the image signal waveforms 401 and 402 overlap each other in the focus state. In this way, the amount of defocus (defocus amount) can be detected from the amount of defocus between the image signal waveforms 401 and 402 obtained from the sub-pixels a and b, and the focus can be adjusted.

FIG. 5 is a diagram showing the overall configuration of the image sensor 106. In the pixel region PA, the unit pixels 200 are arranged in a matrix (n rows × k columns), that is, in a matrix like p11 to pkn. Here, the configuration of the unit pixel 200 will be described with reference to FIG. FIG. 6 is a diagram showing a circuit configuration of a unit pixel of an image sensor.

In FIG. 6, the optical signals incident on the PDs (photoelectric conversion units) 601a and 601b of the sub-pixels a and b described above are photoelectrically converted by the PDs 601a and 601b, and charges corresponding to the exposure amount are accumulated in the PDs 601a and 601b. .. By setting the signals txa and txb applied to the gates of the transfer gates 602a and 602b to high levels, respectively, the charges accumulated in the PD601a and 601b are transferred to the FD (floating diffusion) unit 603 (charge transfer). The FD unit 603 is connected to the gate of the floating diffusion amplifier 604 (hereinafter referred to as an FD amplifier), and the amount of electric charge transferred from the PD601a and 601b is converted into a voltage amount by the FD amplifier 604.

The FD unit 603 is reset by setting the signal res applied to the gate of the FD reset switch 605 for resetting the FD unit 603 to the High level. When resetting the charges of PD601a and 601b, the signal res and the signals txa and txb are set to the high level at the same time. As a result, both the transfer gates 602a and 602b and the FD reset switch 605 are turned on, and the PD601a and 601b are reset via the FD unit 603. By setting the signal sel applied to the gate of the pixel selection switch 606 to the High level, the pixel signal converted into a voltage by the FD amplifier 604 is output to the output vout of the unit pixel 200.

As shown in FIG. 5, the vertical scanning circuit 501 supplies drive signals such as res, txt, txb, and sel that control each switch of the unit pixel 200 to each unit pixel 200. These drive signals res, txa, txb, and sel are common to each row. The output vout of each unit pixel 200 is connected to the column common read circuit 503 via a vertical output line 502 for each column.

Here, the configuration of the column common readout circuit 503 will be described with reference to FIG. 7.

The vertical output line 502 is provided for each row of the unit pixel 200, and the output vout of the unit pixel 200 for one row is connected. A current source 504 is connected to the vertical output line 502, and a source follower circuit is formed by the current source 504 and the FD amplifier 604 of the unit pixel 200 connected to the vertical output line 502.

In FIG. 7, the clamp capacitance 701 has a capacitance of C1, the feedback capacitance 702 has a capacitance of C2, and the operational amplifier 703 has a non-inverting input terminal connected to the reference power supply Vref. There is. The switch 704 is for short-circuiting both ends of the feedback capacitance 702, and the switch 704 is controlled by the signal cfs.

The transfer switches 705 to 708 are for transferring the signals read from the unit pixels 200 to the signal holding capacities 709 to 712, respectively. The pixel signal Sa output from the sub-pixel a is stored in the first S signal holding capacitance 709 by the reading operation described later. Further, in the second S signal holding capacity 711, an a / b combined signal Sab, which is a signal obtained by combining (adding) the signal output from the sub-pixel a and the signal output from the sub-pixel b, is stored. .. Further, the noise signal N of the unit pixel 200 is stored in the first N signal holding capacity 710 and the second N signal holding capacity 712, respectively. The signal holding capacitances 709 to 712 are connected to the outputs vsa, vna, vsb, and vnb of the column common read circuit 503, respectively.

As shown in FIG. 5, horizontal transfer switches 505 and 506 are connected to the outputs vsa and vna of the column common readout circuit 503, respectively. The horizontal transfer switches 505 and 506 are controlled by the output signal ha * (* is a column number) of the horizontal scanning circuit 511. When the signal ha * reaches the High level, the signals of the first S signal holding capacity 709 and the first N signal holding capacity 710 are transferred to the horizontal output lines 509 and 510, respectively.

Further, horizontal transfer switches 507 and 508 are connected to the outputs vsb and vnb of the column common read circuit 503, respectively. The horizontal transfer switches 507 and 508 are controlled by the output signal hb * (* is a column number) of the horizontal scanning circuit 511. When the signal hb * reaches the high level, the signals of the second S signal holding capacity 711 and the second N signal holding capacity 712 are transferred to the horizontal output lines 509 and 510, respectively. The horizontal output lines 509 and 510 are connected to the input of the differential amplifier 514, and the differential amplifier 514 takes the difference between the S signal and the N signal and at the same time applies a predetermined gain, and outputs the final output signal to the output terminal 515. Output to.

When the signal chest applied to the gate of the horizontal output line reset switches 512 and 513 is set to High, the horizontal output line reset switches 512 and 513 are turned on, and the horizontal output lines 509 and 510 are reset to the reset voltage Vchres.

Hereinafter, the reading operation of the image signal A and the reading operation of the image signal AB, which is a composite signal of the image signal A and the image signal B, will be described.

FIG. 8 is a diagram showing the relationship between the pixel region PA of the image sensor 106 and the ranging frame 801 set in the pixel region PA for performing focus detection. The ranging frame 801 is set for the DSP 109 by the CPU 114 and is controlled by the generation of the drive signal by the TG 113. The distance measuring frame 801 is set according to the setting data stored in the ROM 119 in advance, but the distance measuring frame 801 may be set according to the data input by the user operating an operation member (not shown).

As described above, in the pixel region PA composed of pixels of k columns × n rows, the distance measuring frame 801 is indicated by the dotted line. In the present embodiment, the image signal A and the image signal AB are read out from the line of the unit pixel included in the area shown by the shaded area, that is, from the line of the unit pixel in the area of Region_i, and image generation and image generation are performed. Used for focus detection calculations. Only the image signal AB is read from the line of the unit pixel included in the region Region_c, which is an region other than the region Region_i, and is not used for the focus detection calculation but used only for image generation.

As shown in FIG. 8, when a plurality of regions Region_i are set in the vertical direction of the pixel region, the number of rows of the unit pixel 200 in each region Region_i may be set differently from each other.

Next, the reading operation of the image pickup device 106 will be described with reference to FIGS. 9a to 9c. FIG. 9a is a timing chart of the read operation performed for each row of the above-mentioned region Region_c.

First, the operational amplifier 703 is put into a buffer state by setting the signal cfs to the High level and turning on the switch 704. Next, the signal sel is set to the High level and the pixel selection switch 606 of the unit pixel is turned on. After that, the signal res is set to the Low level, the FD reset switch 605 is turned off, and the reset of the FD unit 603 is released.

Subsequently, after returning the signal cfs to the Low level and turning off the switch 704, the signals tna and tnb are set to the High level, and the first N signal holding capacity 710 and the second N signal are set via the transfer switches 706 and 708. The noise signal N is stored in the holding capacitance 712.

Next, the signals tna and tnb are set to Low, and the transfer switches 706 and 708 are turned off. After that, the transfer switch 707 is turned on by setting the signal tsb to the high level, and the transfer gates 602a and 602b are turned on by setting the signals txa and txb to the high level. By this operation, the signal obtained by synthesizing the charge signal stored in the PD601a of the sub-pixel a and the charge signal stored in the PD601b of the sub-pixel b is sent to the vertical output line 502 via the FD amplifier 604 and the pixel selection switch 606. It is output. The signal of the vertical output line 502 is amplified by the operational amplifier 703 with a gain corresponding to the capacitance ratio of the capacitance C1 of the clamp capacitance 701 and the capacitance C2 of the feedback capacitance 702, and the second S signal holding capacitance is amplified via the transfer switch 707. It is stored in 711 (a / b composite signal Sab). After the transfer gates 602a and 602b and the transfer switch 707 are sequentially turned off, the signal res is set to the high level, the FD reset switch 605 is turned on, and the FD unit 603 is reset.

Next, when the output hb1 of the horizontal scanning circuit 511 reaches the High level, the horizontal transfer switches 507 and 508 are turned on. As a result, the signals of the second S signal holding capacity 711 and the second N signal holding capacity 712 are output to the output terminal 515 via the horizontal output lines 509 and 510 and the differential amplifier 514. The horizontal scanning circuit 511 outputs a / b composite signal (image signal AB) for one row by sequentially setting the selection signals hb1, hb2, ..., Hbk in each column to High. While the signals of each row are read by the signals hb1 to hbk, the horizontal output line reset switches 512 and 513 are turned on by setting the signal chess to the High level, and the horizontal output lines 509 and 510 are once reset voltage. Reset to Vchres level.

The above is the reading operation of each line of the unit pixel in the region Region_c. As a result, the image signal AB is read out.

Subsequently, the reading operation of each line of the region Region_i will be described with reference to FIGS. 9b and 9c. FIG. 9b is a timing chart of the operation until the image signal A is read out. The operation from setting the signal cfs to the High level, setting the signals tna and tnb to Low, and storing the N signal in the first N signal holding capacity 710 and the second N signal holding capacity 712 is shown in FIG. 9a. It is the same as the operation described.

When the storage of the noise signal N is completed, the transfer gate 602a is turned on by setting the signal tsa to the high level and turning on the transfer switch 705, and setting the signal txa to the high level. By such an operation, the signal stored in the PD601a of the sub-pixel a is output to the vertical output line 502 via the FD amplifier 604 and the pixel selection switch 606. The signal of the vertical output line 502 is amplified by the operational amplifier 703 with a gain corresponding to the capacitance ratio of the capacitance C1 of the clamp capacitance 701 and the capacitance C2 of the feedback capacitance 702, and the first S signal is held via the transfer switch 705. It is stored in the capacity 709 (pixel signal Sa).

Next, when the output ha1 of the horizontal scanning circuit 511 reaches the High level, the horizontal transfer switches 505 and 506 are turned on. As a result, the signals of the first S signal holding capacity 709 and the first N signal holding capacity 710 are output to the output terminal 515 via the horizontal output lines 509 and 510 and the differential amplifier 514. The horizontal scanning circuit 511 outputs a signal (image signal A) of the sub-pixel a for one row by sequentially setting the selection signals ha1, ha2, ..., Hak in each column to High.

The signal res remains at the Low level, the signal sel remains at the High level, and the reading of the image signal A ends. As a result, the image signal A on the FD unit 603 is not reset and is held.

When the reading of the image signal A is completed, the process proceeds to the reading operation of the image signal AB shown in FIG. 9c. The transfer switch 707 is turned on by setting the signal tsb to the high level, and the transfer gates 602a and 602b are turned on by setting the signals txa and txb to the high level. By such an operation, the signal accumulated in the PD 602b of the sub pixel b is added to the signal of the sub pixel a held in the FD unit 603, and the added signal is passed through the FD amplifier 604 and the pixel selection switch 606. Is output to the vertical output line 502. The subsequent part is the same as the operation of the region Region_c described with reference to FIG. 9a.

As a result, the reading operation of each line in the region Region_i is completed. As a result, the image signal A and the image signal AB are sequentially read out.

By the way, as described above, since the image signal A and the image signal AB are read out in the region Region_i, it takes longer than the reading of the image signal AB in the region Region_c, and the exposure amount is deviated. There is a fear. Therefore, in the present invention, the length of time required for each of the read operations of FIGS. 9b and 9c, that is, the length of the read cycle (unit read cycle) is determined by the length of time required for the read operation of FIG. 9a, that is, , Make it the same as the length of the read cycle (unit read cycle). Here, the length of the read time for one line of the region Region_c is defined as the length of the read time (unit read cycle, unit cycle) of one unit as a reference, and is represented by 1H. Here, "H" does not mean "how". The length of the read cycle in the read operation shown in FIG. 9a, the length of the read cycle in the read operation shown in FIG. 9b, and the length of the read cycle in the read operation shown in FIG. 9c are all 1H, that is, one unit read cycle. Is set to the length of. When reading each line of the region Region_i, the image signal A and the image signal AB are read. Therefore, the read cycle in each line of the region Region_i is the length of 2 units read cycle, that is, 2H. .. When reading each line of the region Region_c, only the image signal AB is read, so that the read cycle of each line of the region Region_c is the length of one unit read cycle, that is, 1H. That is, the read time of each row in the region Region_i is twice the read time of each row in the region Region_c. As a result, it is possible to obtain an image in which the exposure amount does not deviate even in the slit rolling operation.

Hereinafter, the slit rolling operation when the length of the read cycle in the region Region_i and the length of the read cycle in the region Region_c are the same will be described with reference to FIG.

In the slit rolling operation, the reset scan is started in advance, and then the read scan is performed, so that an image is acquired in each row with a constant accumulation time. FIG. 10 shows an example of operating with a storage time Tab of 4H. As described above, 1H is the read time (unit read cycle, unit cycle) for one line of the region Region_c.

As shown in FIG. 10, this operation is realized when the line number of the reset scan advances four lines ahead of the line number of the read scan. In the operation of each line, the frame with dots indicates the reset of the charge accumulated in the PD, and the white frame indicates the transfer (reading) of the charge accumulated in the PD. Here, the m-th row and the m + 1-th row are located in the region Region_i. The m-5th to m-1st rows and the m + 2nd to m + 5th rows are located in the region Region_c. The length of the read-out cycle 1001 of the image signal A in the m-th row and the m + 1-th row is 1H, respectively. Further, the length of the read-out cycle 1002 of the image signal AB in the mth line and the m + 1th line is 1H, respectively. The length of the read-out cycle 1003 of the image signal AB in the m-5th line to the m-1th line and the m + 2nd line to the m + 5th line is 1H, respectively. By taking 1H minutes for reading the image signal A and the image signal AB in the region Region_i, it can be seen that a constant accumulation time Tab is maintained in both the region Region_i and the region Region_c. For this purpose, it is important that the timing at which the txa and txb pulses are output (particularly the falling timing at which the transfer gate is turned off) is the same in each H.

When reading the unit pixels 200 located in the mth row and the m + 1th row, the reading of the image signal A is performed within the reading cycle 1001, and the reading of the image signal AB is performed within the reading cycle 1002. .. Therefore, within the read cycle 1001, the charge stored in the PD 601a is transferred to the FD unit 603, and within the read cycle 1002, the charge stored in the PD 601b is transferred to the FD unit 603.
Further, when reading is performed on the unit pixels 200 located in the m-5th line to the m-1th line and the m + 2nd line to the m + 5th line, the image signal AB is read out within each reading cycle 1003. Will be done. Therefore, within the read cycle 1003, the electric charge stored in the PD601a and the electric charge stored in the PD601b are transferred to the FD unit 603.

In the read-out cycle 1003, the first mode, which is a mode in which the signals for the charges generated in the PD601a and PD601b are combined and the combined-processed signals are read out, is applied.
Further, in the read cycle 1001, a second mode, which is a mode in which the signal corresponding to the electric charge generated in the PD601a is read without performing the synthesis process, is applied.
Further, in the read cycle 1002, a mode different from the second mode applied in the read cycle 1001 is applied. In the present embodiment, the first mode in which the signals for the charges generated in the PD601a and PD601b are combined and the combined-processed signals are read out is applied in the read-out cycle 1002. ..

Here, the accumulation time Ta of the image signal A is 1H shorter than the accumulation time Tab of the image signal AB because it is the time Ta from the reset AB to the transfer A. Depending on the subject and the like, it is possible that this difference may affect the accuracy deterioration of the focus detection calculation in DFE108, DSP109 and the like. In such a case, a gain may be applied and corrected before the focus detection calculation. For example, in DFE108, the levels of the image signal AB and the image signal A can be matched by multiplying the image signal A by Tab / Ta.

In the case of a circuit in which the line number of the reset scan and the line number of the read scan proceed at the same time as in the conventional case, as shown in FIG. 11, as shown in FIG. The accumulation time will be different, and the exposure amount will be different. After all, with DFE108 and DSP109, the signal level of each line can be adjusted by correcting the accumulation time difference, but the S / N is not good depending on the line. It is not desirable that the image signal becomes uniform. Further, as shown in FIG. 12, even in the same operation as in FIG. 10, it takes 1 H minutes to read or reset the image signal A and the image signal AB in the region Region_i. Otherwise, the accumulation time will differ depending on the row, and the exposure amount will deviate.

A signal is read from the image sensor 106 by the above operation, and the DSP 109 generates an image from the image signal AB of each pixel and performs an AF calculation using the image signal A and the image signal AB of each pixel in the region Region_i. In the present embodiment, the AF calculation is performed for each of the ranging frames 801 at 5 × 5 = 25 locations shown in FIG.

As described above, according to the configuration of the present embodiment, it is possible to obtain an image of good image quality with little deviation in the exposure amount while suppressing an increase in the readout time due to signal acquisition for focus detection.

[Second Embodiment]
The image pickup apparatus according to the second embodiment will be described with reference to FIG. The same components as those of the imaging apparatus according to the first embodiment shown in FIGS. 1 to 12 are designated by the same reference numerals, and the description thereof will be omitted or simplified.
In the first embodiment, as shown in FIG. 8, in order to perform the AF calculation of the 5 × 5 ranging frame 801 from the information of one frame, the region Region_i is set to five places (hereinafter, such measurement). The distance mode is referred to as a multi-frame distance measurement mode). On the other hand, in the present embodiment, an example in which the signal reading time of the image pickup device 106 is shortened by limiting the AF calculation performed on the information of one frame will be described. The configuration of the image pickup apparatus in the present embodiment is the same as the configuration shown with reference to FIGS. 1 to 7 and 9 to 10 in the first embodiment. The description is omitted. The difference between the first embodiment and the present embodiment is the setting of the ranging frame 1301 used for reading out the image signal for focus detection (AF calculation) in each frame. Therefore, here, the setting of the ranging frame 1301 will be mainly described.

FIG. 13 shows a distance measuring frame 1301 set for acquiring an image signal A and performing an AF calculation for each frame. In the frame f, the AF calculation is performed only on the ranging frame 1301 corresponding to the uppermost five ranging frames 801 of the 5 × 5 ranging frames 801 shown in FIG. Therefore, the area corresponding to the uppermost stage of the five stages is designated as the region Region_i, and the other region is designated as the region Region_c, and the pixel signal is read out.

Next, in the frame f + 1, the AF calculation is performed only on the five AF frames 1301 in the second row from the top. Therefore, the area corresponding to the second stage from the top is set as the area Region_i, and the other area is set as the area Region_c, and the pixel signal is read out. The same applies to the subsequent frame f + 2, in which the five ranging frames 1301 in the third row from the top are the targets of AF calculation, the region corresponding to the third row from the top is the region Region_i, and the other regions are the region Region_c. To read out the pixel signal. By reading the pixel signal from the pixel area PA while changing the position of the area Region_i for each frame in this way, AF calculation is performed at 5 × 5 = 25 points in 5 frames. The area in which the AF calculation should be performed is limited according to the subject. For example, when only the five frames in the middle row are sufficient, the operation of frame f + 2 in FIG. 13 may be repeated in each frame. In this way, the setting mode in which the number of AF frames 1301 to be AF calculated in one frame is limited to a small number is referred to as a limited frame distance measurement mode here, and a large number of measurements are performed in one frame as shown in FIG. The setting mode in which the distance frame is the target of AF calculation is referred to as a multi-frame distance measurement mode.

The designation or change of the position of the region Region_i is realized by instructing the TG 113 from the CPU 114 and changing the drive signal of the image sensor 106 generated from the TG 113.

In the limited frame ranging mode as in the present embodiment, the number of rows of the unit pixel 200 located in the region Region_i is small, so that the image sensor is more than in the case of the multiple frame ranging mode as shown in the first embodiment. The signal reading time of 106 can be shortened. Therefore, according to the present embodiment, it is possible to suppress the average power consumption or increase the frame rate.

The multi-frame ranging mode and the limited-frame ranging mode may be switched according to the shooting mode and setting of the imaging device (camera), the focus state, and the like. For example, a method such as a limited frame ranging mode in the high frame rate shooting mode and a multi-frame ranging mode in the low frame rate shooting mode can be considered.

It is also conceivable to use the image pickup device properly depending on the focus adjustment mode (AF mode) setting. For example, in the mode in which the focus position is determined by adding the information of a plurality of ranging frames, the multi-frame ranging mode and the mode in which the focus is focused only on the subject in any one ranging frame are limited. A method such as using the frame distance measurement mode is also conceivable.

Alternatively, first, AF calculation is performed on many frames in the multi-frame distance measurement mode, one point or a small number of distance measurement frames are selected based on the result, and thereafter, the operation is performed in the limited frame distance measurement mode. A method is also conceivable.

In the present embodiment, it is described that the number of lines per region of the region Region_i is the same in each setting mode of the multi-frame ranging mode and the limited frame ranging mode, but of course, the number of lines is set in each mode. You can change it. For example, in the multi-frame ranging mode, the region Region_i having a small number of lines may be set to a plurality of areas, and in the limited frame ranging mode, the region Region_i having a large number of lines may be set to only one area. When changing the position of the area Region_i or the number of lines per area of the area Region_i, it is set immediately in the next frame by changing the setting from the next reset scan following the read scan with the setting before the change. The changed image and distance measurement information can be obtained.

[Third Embodiment]
The image pickup apparatus according to the third embodiment will be described with reference to FIGS. 14 to 15. The same components as those of the image pickup apparatus according to the first or second embodiment shown in FIGS. 1 to 13 are designated by the same reference numerals, and the description thereof will be omitted or simplified.
In the first and second embodiments, in the reading scan of the pixel signal in the region Region_i, it takes 1H to read the image signal A and the image signal AB, respectively, and the reading of the image signal A and the reading of the image signal AB are performed. When it was performed, it took a total of 2 hours. However, in the case of not the slit rolling operation, it is not always necessary to take the time of 2H. For example, in the still image shooting mode in which the exposure is controlled by the mechanical shutter 120, the slit rolling operation is not performed. Therefore, in the region Region_i, the image signal A and the image signal AB can be read out in a time shorter than 2H minutes. It can be carried out. Hereinafter, an example of changing the reading operation in the still image shooting mode, which is a mode for shooting a still image, and the moving image shooting mode, which is a mode for shooting a moving image, will be described. Since the operation in the moving image shooting mode is the same as the operation of the first and second embodiments described above, the description thereof is omitted here.

FIG. 14 is a flowchart of a shooting mode of the imaging device in the still image shooting mode. When the CPU 114 gives an instruction to shoot a still image, first, in step S1401, all the pixels of the image sensor 106 are reset. This operation is performed by driving the signals res, txa, and txb from the vertical scanning circuit 501 for all the pixels.

When the vertical scanning circuit 501 sets the signals txa and txb to Low and turns off the transfer gate, charge accumulation of all pixels is started (step S1402). After the start of charge accumulation, in step S1403, the mechanical shutter 120 is opened via the shutter drive circuit 121, and exposure to the image sensor 106 is started. The CPU 114 waits until a predetermined exposure time elapses in the following step S1404, and then closes the mechanical shutter 120 in step S1405 to end the exposure.

After the end of the exposure, the TG 113 is used to start reading the signal from the image sensor 106 (step S1406). When the reading of the final line is completed in step S1407, the still image acquisition operation ends.

As described above, in still image shooting, the optical signal charge of the image sensor 106 is controlled by the exposure control of the mechanical shutter, so that even if the accumulation time of the image sensor 106 differs for each pixel (for each line), it has almost an effect on the image. No slit rolling operation is required.

FIG. 15 shows a reading operation during still image shooting. Since it is not a slit rolling operation, reset scanning is not necessary (reset is performed in S1401). In the read operation for the region Region_i, the time required for reading the image signal A and reading the image signal AB does not have to be adjusted to 1H each, and the reading is performed and scanned in the minimum time required for each.

With the above configuration, it is possible to acquire an image without exposure deviation while performing the focus detection operation during the slit rolling operation in the moving image shooting mode, and the minimum readout time while performing the focus detection operation even in the still image shooting mode. It is possible to acquire an image with. The method of the present embodiment may be applied to the second embodiment.

[Fourth Embodiment]
The image pickup apparatus according to the fourth embodiment will be described with reference to FIGS. 16 to 17b. The same components as those of the image pickup apparatus according to the first to third embodiments shown in FIGS. 1 to 15 are designated by the same reference numerals, and the description thereof will be omitted or simplified.
In the first to third embodiments, the image signal A and the image signal AB are read out in the region Region_i. However, a configuration in which the image signal A and the image signal B are read out is also conceivable. In this embodiment, the configuration in this case is presented. The configuration of the imaging device in this embodiment is the same as that of the first embodiment for the configurations shown in FIGS. 1 to 8, and the description thereof will be omitted. The difference between the present embodiment and the first embodiment is the configuration in which the image signal A and the image signal B are read out in the region Region_i. Therefore, only the configuration will be described here.

FIG. 16 shows a timing chart of the slit rolling operation according to the present embodiment. The operations other than the m-th row and the m + 1-th row corresponding to the region Region_i are the same as those described with reference to FIG. During the read scan for the region Region_i, the PD601a (image signal A) is first read, and then the PD601b (image signal B) is read. The same applies to the reset scan.

The m-th row and the m + 1-th row are located in the region Region_i. The m-5th to m-1st rows and the m + 2nd to m + 5th rows are located in the region Region_c. The length of the read cycle 1601 of the image signal A in the mth row and the m + 1th row is 1H, respectively. Further, the length of the read cycle 1602 of the image signal B in the mth line and the m + 1th line is 1H, respectively. The length of the read-out cycle 1603 of the image signal AB in the m-5th line to the m-1th line and the m + 2nd line to the m + 5th line is 1H, respectively.

When reading the unit pixels 200 located in the mth row and the m + 1th row, the reading of the image signal A is performed within the reading cycle 1601, and the reading of the image signal B is performed within the reading cycle 1602. .. Therefore, within the read cycle 1601, the charge stored in the PD 601a is transferred to the FD unit 603, and within the read cycle 1602, the charge stored in the PD 601b is transferred to the FD unit 603.
Further, when reading is performed on the unit pixels 200 located in the m-5th line to the m-1th line and the m + 2nd line to the m + 5th line, the image signal AB is read out within each reading cycle 1603. Will be done. Therefore, within the read-out cycle 1603, the electric charge stored in the PD601a and the electric charge stored in the PD601b are transferred to the FD unit 603.

In the read-out cycle 1603, the first mode, which is a mode in which the signals for the charges generated in the PD601a and PD601b are combined and the combined-processed signals are read out, is applied.
Further, in the read cycle 1601, a second mode, which is a mode in which the signal corresponding to the electric charge generated in the PD601a is read without performing the synthesis process, is applied.
Further, in the read cycle 1602, a mode different from the second mode applied in the read cycle 1601 is applied. In the present embodiment, a third mode, which is a mode in which the signal corresponding to the electric charge generated in the PD601b is read out without performing the synthesis processing, is applied in the read-out cycle 1602.

The timing chart of 1H at the time of reading the image signal A is shown in FIG. 17a. FIG. 17a differs from FIG. 9b only in that the signal tnb is not set to High when the noise signal N is read out, and the noise signal N is not stored in the second N signal holding capacitance 712.

Next, the timing chart of 1H at the time of reading the image signal B is shown in FIG. 17b. FIG. 17b differs from FIG. 9c in the following points. That is, first, the signal res is once set to the high level to reset the FD unit 603, the signal cfs is set to the high level to put the operational amplifier 703 in the buffer state, and then the signal tnb is set to the high level to set the second N signal holding capacity 712. It is a point where the noise signal N is stored in.

An image signal AB in the region Region_i is required for image generation. For this, the image signal AB may be generated by adding the image signal A and the image signal B in the DSP109. Further, the method of the present embodiment may be applied to the second embodiment and the third embodiment.

In the present embodiment, unlike the case of the first embodiment shown in FIG. 10, the accumulation time of the image signal A and the accumulation time of the image signal B are the same, and it is not necessary to perform gain correction for adjusting the levels. There are merits.

According to the embodiment of the present invention described above, an image pickup apparatus and its control capable of obtaining good image quality with little exposure amount deviation while suppressing an increase in reading time for acquiring a signal for focus detection. A method can be provided.

All of the above embodiments merely show examples of embodiment in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[Modification Embodiment]
Not limited to the above embodiment, various modifications are possible.
For example, in the above embodiment, the case where two sub-pixels a and b are provided in one unit pixel 200 has been described as an example, but the number of sub-pixels provided in one unit pixel 200 is two. It is not limited and may be set as appropriate.

Claims (11)

An image sensor having a plurality of microlenses arranged in a matrix direction and a pixel region in which a plurality of unit pixels are arranged in a matrix direction corresponding to each of the plurality of microlenses. Is an image sensor including a first photoelectric conversion unit and a second photoelectric conversion unit, and
The first mode, which is a mode in which the signal corresponding to the electric charge generated by the first photoelectric conversion unit and the second photoelectric conversion unit, respectively, is read out, and the first photoelectric conversion unit generates the signal. A control means capable of at least taking a second mode, which is a mode for reading a signal corresponding to an electric charge, is provided.
The control means reads the first row located in the first region of the pixel region in the first mode, and the first region of the pixel region. For the second row located in the second region different from the above, the read in the second mode and the read in the first mode are performed.
The control means resets the charges generated by the first photoelectric conversion unit and the second photoelectric conversion unit of each row included in the pixel region by reading the charges in the first mode and the first. Controlled at the timing synchronized with the reading in the mode of 2,
By said control means, said number of signals read from the imaging device in the first mode for the second row, the second mode rather his equal and the number of signals read out from the image sensor,
An image pickup apparatus characterized in that the number of signals read from the image sensor for the first row is less than the number of signals read from the image sensor for the second row. In the imaging device according to claim 1,
The reset is performed at a timing corresponding to the start of reading in the reading.
An imaging device characterized by this. In the imaging device according to claim 2,
The reset is performed at the same timing as the charge transfer in the read.
An imaging device characterized by this. In the imaging apparatus according to any one of claims 1 to 3,
The control means includes a signal read in the first mode from the first unit pixel located in the second region of the plurality of unit pixels, and the second unit pixel to the second unit pixel. An imaging device characterized in that focus detection is performed based on a signal read out in a mode. In the imaging apparatus according to any one of claims 1 to 4,
The control means is an imaging device that generates an image of a subject based on a signal read from each of the plurality of unit pixels in the first mode. In the imaging apparatus according to any one of claims 1 to 5,
The control means may have a first setting mode that limits the setting of the second region and a second setting mode that does not limit the setting of the second region.
The limitation is at least one of the number of the second region, the position of the second region, and the number of rows of the unit pixels contained in one of the second regions. Imaging device. In the imaging device according to claim 6,
The second region corresponds to a ranging frame for detecting the in-focus state.
The control means switches between the first setting mode and the second setting mode according to the shooting mode or the focus adjustment mode of the imaging device, and the shooting mode is a high frame rate shooting mode and a low frame rate shooting mode. The image pickup apparatus including the above, wherein the focus adjustment mode includes a focus adjustment mode using a plurality of the distance measurement frames and a focus adjustment mode using one of the distance measurement frames. In the imaging device according to claim 7,
When the distance measuring frame is selected according to the focusing state detected by the distance measuring frame set in the second setting mode in the second setting mode, the control means measures the selected measurement. An imaging device characterized by switching to the first setting mode for setting a distance frame. In the imaging device according to claim 6,
The control means is an imaging device, characterized in that, when changing the setting of the first region in the first setting mode, the setting is changed from a reset scan of each row of the pixel region. In the imaging device according to any one of claims 1 to 9.
The first region is a plurality of first regions set in the vertical direction of the pixel region.
The number of rows of the unit pixel included in the first region of one of the plurality of first regions and the other first region of the plurality of first regions. An imaging device characterized in that the number of rows of the unit pixel is different. An image sensor having a plurality of microlenses arranged in a matrix direction and a pixel region in which a plurality of unit pixels are arranged in a matrix direction corresponding to each of the plurality of microlenses. Is a control method for an image pickup device including a first photoelectric conversion unit and a second photoelectric conversion unit.
The step of reading in the first mode, which is a mode of reading the signal corresponding to the electric charge generated by the first photoelectric conversion unit and the second photoelectric conversion unit, respectively, and the first It includes a step of reading out in a second mode, which is a mode of reading out a signal corresponding to the electric charge generated by the photoelectric conversion unit.
The read in the first mode is performed on the first row located in the first region of the pixel region, which is different from the first region of the pixel region. For the second row located in the region 2, the read in the second mode and the read in the first mode are performed.
The reset of the electric charge generated by the first photoelectric conversion unit and the second photoelectric conversion unit of each row included in the pixel region is the reading in the first mode and the reading in the second mode. It is controlled at the timing synchronized with the read.
The number of signals read from the imaging device in the first mode for the second row, the second mode rather his equal and the number of signals read out from the image sensor,
A control method for an image pickup device , wherein the number of signals read from the image pickup device for the first row is smaller than the number of signals read out from the image pickup device for the second row.

Images