JP2017203829A - Control device, imaging device, control method, program and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve a high-speed frame rate and high resolving power of phase difference detection.SOLUTION: A control device, which controls an image pick-up element including a first photoelectric conversion unit and second photoelectric conversion unit with respect to one microlens, has control means (105) that controls a first read mode that reads an image generation signal from the first photoelectric conversion unit and second photoelectric conversion unit to be provided in a partial area of a pixel area of the image pick-up element in synchronization with a synchronous signal, and a second read mode that reads a focus detection signal from the first photoelectric conversion unit or the second photoelectric conversion unit to be provided in a focus detection area smaller than the partial area of the partial area in synchronization with the synchronous signal. The control means is characterized to switch the first read mode and the second read mode in accordance with the synchronous signal.SELECTED DRAWING: Figure 9

Description

  The present invention relates to a control device, and more particularly to a control device that controls an image sensor capable of performing focus detection on an imaging surface.

  In Patent Document 1, a plurality of photoelectric conversion units A pixel and B pixel are provided in a pixel corresponding to one microlens in the image sensor, and phase difference detection is performed based on the A pixel output and the B pixel output. A method for obtaining an image based on an A + B pixel output obtained by combining a pixel and a B pixel is disclosed.

  Japanese Patent Application Laid-Open No. 2004-228561 includes a frame that performs phase difference detection based on the outputs of the phase difference detection dedicated pixels arranged at predetermined intervals in the image sensor, and a frame that acquires an image based on other image capture pixel outputs. A method of dividing is disclosed.

JP 2001-083407 A JP 2014-056088 A

  However, in the prior art disclosed in Patent Document 2 described above, the phase difference detection dedicated pixel cannot be used as an imaging signal, and thus it is necessary to perform correction processing such as interpolation processing using peripheral imaging pixels as defective pixels. is there. On the other hand, if the outputs of the A pixel and the B pixel can be obtained from all the pixels on the imaging surface as in the conventional technique disclosed in Patent Document 1, all the pixels can be used as the phase difference detection element. For this reason, it is thought that it is a more effective method as a phase difference detection means. However, in order to read out the output of the A pixel and the B pixel from all the pixels, for example, when performing non-decimation readout such as crop readout, the readout of the phase difference detection signal and the image generation signal is performed. The number of readout pixels increases. Therefore, it is difficult to achieve both a high frame rate and high resolution for phase difference detection.

  Accordingly, an object of the present invention is to realize a high frame rate and high resolution for phase difference detection.

  A control device according to one aspect of the present invention is a control device that controls an imaging device including a first photoelectric conversion unit and a second photoelectric conversion unit with respect to one microlens, and is synchronized with a synchronization signal. A first readout mode for reading out an image generation signal from the first photoelectric conversion unit and the second photoelectric conversion unit provided in a partial region of the pixel region of the image sensor, and the synchronization in synchronization with the synchronization signal. Control means for controlling a second readout mode for reading out a focus detection signal from the first photoelectric conversion unit or the second photoelectric conversion unit provided in a focus detection region smaller than the partial region of the partial region The control means switches between the first readout mode and the second readout mode in accordance with the synchronization signal.

  According to the present invention, a high frame rate and high resolution for phase difference detection can be realized.

It is a figure which shows the circuit block of the imaging device in this invention. It is a figure which shows the mechanical structure of the imaging device in this invention. It is a figure which shows the functional block inside DSP in this invention. It is a figure which shows the circuit of the image pick-up element in this invention. It is a figure which shows the circuit of the unit pixel of the image pick-up element in this invention. It is a top view of a unit pixel of an image sensor in the present invention. It is a figure which shows an example of the column circuit of the image pick-up element in this invention. It is a figure which shows the timing chart of the image pick-up element in this invention. It is a figure which shows the imaging sequence of the imaging device in this invention. It is a figure which shows the timing chart of the image pick-up element in this invention. It is a figure which shows the flowchart of the video recording in this invention. It is a figure which shows the vertical scanning of the image pick-up element in this invention. It is a figure which shows the imaging sequence of the imaging device in this invention. It is a figure which shows the read-out area | region of the focus detection frame and phase difference detection frame in this invention. It is a figure which shows an example of the image pick-up element in this invention.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the drawings. In addition, the same reference number is attached | subjected about the same member in each figure, and the overlapping description is abbreviate | omitted.

  FIG. 1 is a block diagram of an imaging apparatus according to the first embodiment of the present invention.

  The imaging apparatus according to the present embodiment includes an imaging device 101, Analog Front End (AFE) 102, Timing Generator (TG) 103, Digital Signal Processor (DSP) 104, and CPU 105.

  The image sensor 101 includes an amplifier circuit (not shown) that switches the gain according to the ISO sensitivity.

  The AFE 102 incorporates an A / D converter that converts an analog signal from the image sensor 101 into a digital signal, and has a function of clamping an offset level.

  In this embodiment, an example of a combination of a CMOS image sensor corresponding to analog output and AFE is shown, but a CMOS image sensor compatible with digital output may be used.

  The TG 103 generates and supplies a timing signal for driving the image sensor 101 in various readout modes.

  The DSP 104 performs memory access processing of the ROM 106 and RAM 107, various display processing on the display medium 108, and image data writing processing on the recording medium 109.

  Further, the DSP 104 includes an AF calculation block that performs detection using autofocus (AF) information by detecting defective pixels in the image data obtained from the image sensor 101 and correcting the same, and using pixel output. Will be described later.

  The CPU 105 controls the entire imaging apparatus, and specifically controls the imaging element 101, the AFE 102, the TG 103, and the DSP 104. As described above, the CPU 105 is configured as a control unit (control device) that controls each configuration of the imaging device such as the imaging device 101, the AFE 102, the TG 103, and the DSP 104. Further, camera functions such as AF using an output of a dedicated element for phase difference AF that performs independent phase difference AF (not shown), AF using autofocus information calculated using an output of a pixel incorporated in the image sensor 101, and the like. Take control.

  The CPU 105 is connected to a power switch 110, a first-stage shutter switch SW1 (111), a second-stage shutter switch SW2 (112), and a mode dial 113, and according to the settings of these switches and dials. Execute the process.

The ROM 106 holds a control program for the imaging apparatus, that is, a program executed by the CPU 105 and various correction data.

  The RAM 107 is used as a work area for temporarily storing image data processed by the DSP 104.

  The display medium 108 (display device) displays menus such as camera shooting mode information and shooting information such as ISO sensitivity. Also, playback and display of still images and moving images is performed.

  The recording medium 109 is a detachable medium for storing captured image data. For example, a memory card is used, and the recording medium 109 is connected to the DSP 104 via a connector (not shown).

  The power switch 110 is operated by the user when starting the imaging apparatus.

  When the first-stage shutter switch SW1 (111) is turned on, pre-shooting processing (shooting preparation operation) such as photometry processing and focus detection processing is executed.

  When the second-stage shutter switch SW2 (112) is turned on, a series of mirrors and shutters (not shown) are driven, and image data captured by the image sensor 101 is written to the recording medium 109 via the AFE 102 and the DSP 104. The imaging operation is started.

  The mode dial 113 is used for setting various operation modes of the imaging apparatus.

  Next, the mechanical configuration of the imaging apparatus will be described with reference to FIG. FIG. 2 is a side sectional view showing the configuration of the image pickup apparatus. The state when the optical viewfinder for taking a still image is used, and the mirror up and the shutter open state when moving image shooting or live view is used will be described.

  FIG. 2A shows a state when the optical viewfinder is used, and FIG. 2B shows a mirror-up state and a shutter-open state during moving image shooting and live view.

  A photographing lens 202 is attached to the front surface of the camera body 201.

  The photographing lens 202 can be exchanged, and the camera body 201 and the photographing lens 202 are electrically connected via a mount contact group 203.

  Further, an aperture 204 and a focus adjustment lens group 205 are arranged in the photographing lens 202, and both are controlled by the lens control device 206.

  The main mirror 207 has a half mirror configuration. When the optical viewfinder for still images shown in FIG. 2A is used, the optical viewfinder is obliquely arranged on the photographing optical path, and reflects light from the photographing lens and guides it to the viewfinder.

  On the other hand, the transmitted light is incident on the AF unit 209 via the sub mirror 208. The AF unit 209 is a phase difference detection type AF sensor, and performs an AF operation by controlling the focus adjustment lens group of the photographing lens 202 based on the detection result of the AF sensor.

  The focal plane shutter 210 is normally closed, and performs an opening operation so as to perform exposure at the time specified by the user at the time of shooting.

  The light reflected by the main mirror 207 is projected onto the focus plate 211, further guided by the pentaprism 212 to the eyepiece lens group 213 after changing the optical path, and the user observing the focus plate 211 from the eyepiece lens group 213. The subject image can be confirmed.

  In conjunction with the user pressing a release button (not shown), a shooting preparation operation such as AE or AF is performed by SW1 (111) which is turned on by half-pressing the release button.

  Further, the SW2 (112) that is turned on when the release button is fully pressed operates the main mirror 207 and the sub mirror 208 to be retracted from the optical path, and then the focal plane shutter is opened for a time corresponding to the second setting, so that the image sensor 101 is opened. An image pickup operation is performed after exposure.

  Next, the state at the time of moving image shooting and live view of the imaging apparatus will be described with reference to FIG.

  When the shooting mode is switched by the mode dial 113 and, for example, a live view state is set, the main mirror 207 and the sub mirror 208 operate so as to retract from the optical path.

  Further, the focal plane shutter 210 is kept open, and the image sensor 101 is constantly exposed.

  A so-called live view mode is realized by through-displaying an image signal obtained from the image sensor 101 on a display medium (not shown). In this state, a moving image is recorded by recording a moving image on a recording medium (not shown).

  In this case, since the main mirror 207 and the sub mirror 208 are retracted, confirmation of the subject image by the optical finder and phase difference AF using the AF unit 209 cannot be used.

  Next, FIG. 3 shows a block diagram of the DSP 104.

  The DSP 104 has a memory control block 303, a recording medium control block 304, and a display medium control block 305 in addition to the development block 301 and the image compression block 302 related to image processing.

  In addition, an image correction block 308 is provided for digitally correcting gain and offset level errors caused by individual variations of the image sensor 101 when generating an image.

  Further, there is an AF block 306 for calculating autofocus information from the pixel output, and a communication control block 307 for generally performing bidirectional communication with the CPU 105 in addition to transmitting this information to the CPU 105.

  Further, before outputting data to the AF block 306, correction of gain and offset level errors caused by individual variations of the image sensor 101, and digital correction of optical conditions at the time of shooting such as the focal length and aperture value of the photographing lens are performed. There is an AF correction block 309.

  Next, the pixel structure of the image sensor 101 will be described with reference to FIG. FIG. 4 shows an example of the configuration of the image sensor 101 in this embodiment.

  In the pixel region, unit pixels including a plurality of photoelectric conversion units are arranged in a matrix at equal intervals in the horizontal and vertical directions.

  First, the configuration of the unit pixel will be described. FIG. 5 shows the configuration of the unit pixel in the nth row of the image sensor 101.

  The photoelectric conversion unit 501 and the photoelectric conversion unit 502 are configured by photodiodes. The photoelectric conversion unit is divided into two parts inside a region corresponding to one microlens 601 described later. As described above, the imaging element 101 has a plurality of unit pixels each including the photoelectric conversion unit 501 (first photoelectric conversion unit) and the photoelectric conversion unit 502 (second photoelectric conversion unit) for one microlens 601.

  Hereinafter, for convenience, the photoelectric conversion unit 501 is referred to as an A pixel, and the photoelectric conversion unit 502 is referred to as a B pixel.

  The electric charges generated in the A pixel and the B pixel are transferred to the storage capacitor 505 via the transfer transistors 503 and 504. The storage capacitor 505 holds charges generated in the A pixel and the B pixel. Here, an image generation signal (image generation signal) is generated by the first operation of transferring the charges of the plurality of photoelectric conversion units (A pixel and B pixel) to the storage capacitor 505. Further, a focus detection signal (focus detection signal) is generated by the second operation of transferring the charge of one of the photoelectric conversion units (A pixel or B pixel) to the storage capacitor 505 among the plurality of photoelectric conversion units. . As described above, the image sensor 101 includes a plurality of pixels, and each pixel (unit pixel) includes the microlens 601, the photoelectric conversion unit 501, the photoelectric conversion unit 502, and the storage capacitor 505.

  The transfer transistors 503 and 504 are ON / OFF controlled by control signals PTXA_n and PTXB_n output from the vertical scanning circuit 506. The control lines 507 and 508 are connected to the gates of the transfer transistors 503 and 504.

  The storage capacitor 505 can reset the charge by turning on the reset transistor 509 with the control signal PRES_n output from the vertical scanning circuit 506. Control line 510 is connected to the gate of reset transistor 509.

  Further, by turning on the transfer transistors 503 and 504 and turning on the reset transistor 509, the charges of the A pixel and the B pixel can also be reset.

  The storage capacitor 505 is connected to the gate of the amplification transistor 511 and constitutes a pixel amplifier.

  The row selection transistor 512 is ON / OFF controlled by a control signal PSEL_n output from the vertical scanning circuit 506. Control line 513 is connected to the gate of row selection transistor 512.

  When the row selection transistor is turned on, the output of the amplification transistor 511 is connected to the vertical output line 514. The vertical output line 514 is connected to a constant current source (not shown).

  FIG. 6 is an overhead view of the unit pixel from the surface of the image sensor, and the microlens 601 is formed so as to be in contact with the pixel pitch indicated by the broken line.

  Two photoelectric conversion units 501 and 502 divided in the horizontal direction are arranged below the micro lens 601, and the left photoelectric conversion unit is an A pixel (501) and the right side is a B pixel (502).

  The A pixel and the B pixel photoelectrically convert the subject image from the divided pupil region, and the phase difference can be detected by reading the A pixel output and the B pixel output, thereby enabling the focus detection operation.

  Returning to FIG. In FIG. 4, 4 × 2 pixels are arranged in a matrix form of unit pixels. This schematically shows the microlens 601 and the A and B pixels of each pixel. In an actual image sensor, tens of millions of such pixels are arranged.

  A plurality of pixels in the row direction are connected to the vertical output line 514. Each vertical output line 514 is connected to the column circuit 401.

  Details of the column circuit 401 are shown in FIG. The output of the vertical output line 514 is connected to the column amplifier 701. The column amplifier 701 amplifies the signal with a predetermined gain determined by the capacitance ratio between the input capacitor 702 and the feedback capacitor 703 and outputs the amplified signal to the follower 704 at the subsequent stage.

  The follower 704 outputs the noise component holding capacitor 707 and the signal component holding capacitor 708 via the transfer switches 705 and 706. Each transfer switch is controlled by control signals PTN and PTS.

  Each signal held in the noise component holding capacitor 707 and the signal component holding capacitor 708 is sent to horizontal output lines 402 and 403 via transfer switches 709 and 710 controlled by a control signal PH output from a horizontal scanning circuit (not shown). Is output.

  The output amplifier 404 multiplies the difference between the signals output to the horizontal output lines 402 and 403 by a predetermined gain and outputs the result to the outside of the image sensor 101.

  By sequentially repeating this operation for each row of the image sensor, an output of a predetermined pixel of the image sensor can be obtained.

  Here, when both the A pixel and the B pixel are simultaneously transferred, a charge signal generated in the photoelectric conversion unit disposed under the same microlens suitable for image generation can be output.

  This will be described according to the timing chart shown in FIG.

  In FIG. 8, the horizontal synchronization signal HD becomes “L” at time t1, the control signal PSEL_n becomes “H” at time t2, and the selection transistor 512 of the pixel in the n-th row is turned on.

  As a result, the output of the amplification transistor 511 is connected to the vertical output line 514.

  At time t2, the control signal PRES_n becomes “H”, and the reset transistor 509 of the pixel in the n-th row is turned on.

  At time t3 after the elapse of a predetermined time from time t2, the control signal PRES_n becomes “L”, the reset transistor 509 of the pixel in the n-th row is turned off, and the pixel reset operation is released.

  At time t3, the control signal PTN simultaneously becomes “H”, and the signal level of the vertical output line 514 at the time of reset release is amplified by the column amplifier 701 and written to the noise component holding capacitor 707.

  When the control signal PTN is set to “L” at time t 4, the signal charge corresponding to the noise component is held in the noise component holding capacitor 707.

  Next, at time t5, the control signals PTXA_n and PTXB_n are set to “H”, and the transfer transistors 503 and 504 of the A pixel and B pixel of the n-th row pixel are turned ON.

  Further, when the transfer transistors 503 and 504 are turned off at time t6 after a predetermined time, the signal charges of the A pixel and the B pixel are both transferred to the pixel amplifier 511.

  When the control signal PTS is set to “H” at time t 7, the signal level of the vertical output line corresponding to the signal charge is amplified by the column amplifier 701 and written to the signal component holding capacitor 708.

  When the control signal PTS is set to “L” at time t8, the signal level corresponding to the signal charge is held in the signal component holding capacitor 708.

  Up to this point, the output in the reset state of each pixel in the n-th row is held in the noise component holding capacitor 707 and the output corresponding to the signal charge is held in the signal component holding capacitor 708.

  Thereafter, the output of each column is sequentially read out to the horizontal output lines 402 and 403 by a horizontal scanning circuit (not shown), and the difference is multiplied by a predetermined gain by the output amplifier 404, thereby outputting the A pixel + B in the nth row. Reading of the pixel signal is completed.

  Next, the operation when performing the phase difference detection operation will be described. In the present embodiment, after scanning a predetermined area of the image sensor in the vertical direction in one frame period, the area for performing focus detection of the image sensor in the next frame is scanned in the vertical direction.

  In other words, the pixel signal of the target row is read as the pixel signal of each pixel at the first scan, and the pixel signal of the target row is read out separately from the pixel signal of the target row at the second scan. The phase difference is detected using the time output.

  As described above, the present embodiment has the first readout mode in which the image generation signal is read out from the A pixel + B pixel provided in the partial region in the entire pixel region of the image sensor in synchronization with the synchronization signal. In addition, a second readout mode is provided in which a focus detection signal is read from an A pixel (or B pixel) provided in a focus detection area smaller than the partial area in synchronization with the synchronization signal. In this embodiment, the number of read rows in the second read mode is ½ or less of the number of read rows in the first read mode. The CPU 105 can control the first readout mode and the second readout mode, and alternately switches between the first readout mode and the second readout mode for each frame in accordance with the synchronization signal. Can do. Here, the synchronization signal is a vertical synchronization signal, and is a timing signal for starting data creation for each frame during continuous shooting.

  Such a reading method is suitable in the moving image mode, and in the present embodiment, description will be made assuming application in an enlarged display mode of a predetermined area.

  First, this vertical scanning will be described with reference to FIG. FIG. 9 schematically illustrates readout in frame units in the enlarged display mode of the imaging apparatus.

  VD in FIG. 9 indicates a frame synchronization signal. The image sensor 101 of this imaging apparatus switches readout in units of the frame synchronization signal VD. For example, the period of the frame synchronization signal is 60 frames / sec. Here, in the present embodiment, the display medium 108 (display device) updates the display at a predetermined cycle (30 frames / sec), which is twice the cycle of the frame synchronization signal.

  At this time, during a period indicated by the frame 1, as shown by a solid line 901, a predetermined area (partial area) of the image sensor is scanned in the vertical direction. A broken line 902 indicates reset scanning that is performed in advance before pixel reading.

  That is, the period between the solid line 901 and the broken line 902 means the image data accumulation time.

  Next, in the period indicated by the frame 2, as shown by a solid line 903, a region (focus detection region) where focus detection of the image sensor is performed is scanned in the vertical direction. A broken line 904 indicates reset scanning performed in advance before pixel reading.

  Similar to frame 1, the period between the solid line 903 and the broken line 904 means the accumulation time of phase difference detection data.

  In frame 1, the A pixel + B pixel signal is read according to the above-described driving method of the image sensor to generate a moving image. Here, for example, when performing crop display, the CPU 105 performs control so that the image generation signal is read out in a non-decimated manner from a partial region that is a part of the image sensor.

  On the other hand, in frame 2, the A pixel and the B pixel are individually read out in accordance with the following driving method of the image sensor. This will be described according to the timing chart shown in FIG.

  At time t1, the horizontal synchronization signal HD becomes “L”, at time t2, the control signal PSEL_n becomes “H”, and the selection transistor 512 of the pixel in the n-th row is turned on.

  As a result, the output of the amplification transistor 511 is connected to the vertical output line 514.

  At time t2, the control signal PRES_n becomes “H”, and the reset transistor 509 of the pixel in the n-th row is turned on.

  At time t3 after the elapse of a predetermined time from time t2, the control signal PRES_n becomes “L”, the reset transistor 509 of the pixel in the n-th row is turned off, and the pixel reset operation is released.

  At time t3, the control signal PTN simultaneously becomes “H”, and the signal level of the vertical output line 514 at the time of reset release is amplified by the column amplifier 701 and written to the noise component holding capacitor 707.

  When the control signal PTN is set to “L” at time t 4, the noise component holding capacitor 707 holds it.

  Next, at time t5, the control signal PTXA_n is set to “H”, and the transfer transistor 503 of the A pixel of the pixel in the nth row is turned on.

  Further, when the transfer transistor 503 is turned off at a time t6 after a predetermined time, the signal charge of the A pixel is transferred to the pixel amplifier 511.

  When the control signal PTS is set to “H” at time t 7, the signal level of the vertical output line corresponding to the signal charge is amplified by the column amplifier 701 and written to the signal component holding capacitor 708.

  When the control signal PTS is set to “L” at time t8, the signal level corresponding to the signal charge is held in the signal component holding capacitor 708.

  Up to this point, the output in the reset state of each pixel in the n-th row is held in the noise component holding capacitor 707, and the output corresponding to the signal charge of the A pixel is held in the signal component holding capacitor 708.

  Thereafter, the output of each column is sequentially read to the horizontal output lines 402 and 403 by a horizontal scanning circuit (not shown), and the difference is multiplied by a predetermined gain and output by the output amplifier 404, whereby the A pixel signal of the nth row is output. Reading is completed.

  Subsequently, the horizontal synchronization signal HD becomes “L” at time t9, but the control signal PSEL_n maintains the “H” state at time t2, so the selected row remains unchanged and the nth row is selected. It is a state that has been.

  At time t10, the control signal PRES_n becomes “H”, and the reset transistor 509 of the pixel in the n-th row is turned on.

  At time t11 after the elapse of a predetermined time from time t10, the control signal PRES_n becomes “L”, the reset transistor 509 of the pixel in the n-th row is turned off, and the pixel reset operation is released.

  At time t11, the control signal PTN simultaneously becomes “H”, and the signal level of the vertical output line 514 at the time of reset release is amplified by the column amplifier 701 and written to the noise component holding capacitor 707.

  When the control signal PTN is set to “L” at time t12, the noise component holding capacitor 707 holds it.

  Next, at time t13, the control signal PTXB_n is set to “H”, and the transfer transistor 504 of the B pixel of the pixel in the nth row is turned on.

  Further, when the transfer transistor 504 is turned off at time t14 after a predetermined time, the signal charge of the B pixel is transferred to the pixel amplifier 511.

  When the control signal PTS is set to “H” at time t15, the signal level of the vertical output line corresponding to the signal charge is amplified by the column amplifier 701 and written to the signal component holding capacitor 708.

  When the control signal PTS is set to “L” at time t16, the signal level corresponding to the signal charge is held in the signal component holding capacitor 708.

  Thereafter, the output of each column is sequentially read to the horizontal output lines 402 and 403 by a horizontal scanning circuit (not shown), and the difference is multiplied by a predetermined gain and output by the output amplifier 404, whereby the B pixel signal of the nth row is output. Reading is completed.

  Here, the readout of the A pixel signal and the B pixel signal of the pixel in the nth row is completed.

  That is, by repeating such a reading operation, A + B pixels in a predetermined area on the imaging surface are obtained in odd frames, and A pixels and B pixels in a predetermined area on the imaging surface are obtained in even frames.

  The A + B pixel signal obtained in the odd frame is A / D converted by the AFE 102 to be converted into a digital signal, and input to the DSP 104.

  Digital correction is performed in the image correction block 308 of the DSP 104, and through the development block 301 and the image compression block 302, through display is performed on the display medium 108 at 30 frames / sec using the display medium control block 305. At the same time, the data is recorded on the recording medium 109 using the recording medium control block 304.

  On the other hand, the A pixel signal and the B pixel signal obtained in the even frame are also A / D converted by the AFE 102 and converted into a digital signal, and input to the DSP 104 as in the odd frame.

  Through the AF correction block 309 of the DSP 104, correction for each pixel and correction according to optical conditions at the time of shooting are performed.

  After that, the AF block 306 uses the correction result to perform a known correlation operation from the A pixel signal and the B pixel signal, and calculates the phase difference detection result to enable the focus detection operation.

  Upon receiving the calculated phase difference detection result, the CPU 105 performs AF control by controlling the position of the focus adjustment lens group 205 of the photographing lens 202.

  Next, the moving image shooting process of the imaging apparatus shown in FIG. 2 will be described along the flowchart of FIG.

  The CPU 105 is set to the moving image mode selected by the mode dial 113.

  The CPU 105 opens the main mirror 207, the sub mirror 208, and the focal plane shutter 210. With this operation, the subject image enters the image sensor 101 as shown in FIG.

Next, the CPU 105 determines whether or not the shutter switch SW2 is turned on.
When the shutter switch SW2 (112) is ON, the CPU 105 starts a recording operation for writing moving image data to the recording medium 109.

  When the shutter switch SW2 (112) is OFF, the CPU 105 stops the recording operation when a recording operation for writing moving image data to the recording medium 109 is currently being executed.

  That is, the CPU 105 continues to record moving image data while the shutter switch SW2 (112) is ON, and stops recording processing of moving image data while the shutter switch SW2 (112) is OFF.

  The CPU 105 performs exposure adjustment in order to perform monitor display of moving image data on the display medium 108.

  In this exposure adjustment, the exposure amount is determined from the image data taken immediately before, and the lens diaphragm 204 and the AFE 102 gain settings are changed so as to obtain an appropriate exposure amount.

  Next, the CPU 105 starts shooting processing. During moving image shooting, the image sensor 101 repeatedly executes charge reset, charge accumulation, and charge read in accordance with a control signal from the TG 103.

  The readout position of the phase difference detection row is changed according to the focus detection frame (AF frame) set by the user.

  The CPU 105 receives the phase difference detection result, and performs AF control by changing the position of the focus adjustment lens group 205 in the photographing lens 202 as needed.

  Next, if the power switch 110 is OFF, the CPU 105 performs a moving image end process.

  In the movie end process, if the recording operation is currently in progress, the recording operation is stopped, the reading of the image sensor 101 is stopped, the development processing of the DSP 104 is stopped, the focal plane shutter 210 is closed, and the main mirror 207 and the sub mirror 208 are lowered. To do.

  On the other hand, if the power switch 110 remains ON, the CPU 105 checks the mode dial 112.

  If the mode dial 112 remains in the moving image mode, the process returns to the ON / OFF determination of the shutter switch SW2.

  According to the above configuration, in enlarged display or crop display that requires high resolution in the moving image mode, it is possible to perform display at a high frame rate without reducing the resolution of phase difference detection. That is, as described above, by switching the acquisition frame of the image generation signal and the acquisition frame of the phase difference detection signal at a predetermined cycle, a high frame rate and high resolution of phase difference detection can be realized.

  Next, a second embodiment of the present invention will be described. In the present embodiment, switching between full screen display and enlarged display in the moving image mode will be described.

  First, reading of the image sensor in the full screen display mode will be described.

  When displaying the entire screen, the vertical direction is thinned and read every predetermined number of rows, the entire surface of the image sensor is scanned in the vertical direction, and then the scan is returned to the upper row in the vertical direction again. Scan again only those rows. In other words, a first vertical scan that reads out all the pixel regions of the image sensor in the vertical direction with a predetermined thinning rate, and a second vertical scan that reads out with a thinning rate different from the first vertical scan, I do.

  During the first scan, the A + B pixel signal of each pixel in the target row is read, and during the second scan, the A pixel and B pixel of each pixel in the target row are read, and the A + B pixel signal read out during the first scan. A moving image is formed, and phase difference detection is performed using the A and B pixels read out during the second scanning.

  First, the state of this vertical scanning will be described with reference to FIG. FIG. 12A shows the entire imaging surface of the imaging device.

  The hatched portion in FIG. 12 indicates an optical black (OB) portion.

  FIG. 12B shows a schematic diagram of a readout row when performing thinning readout in the vertical direction. In the figure, lines surrounded by a thick frame are read target lines, and the other lines are lines that are thinned out by a thinning operation.

  That is, after the V1 row is read, the next V2 row after the third row is read. Subsequently, the V3 to V7 rows are read at the same thinning rate. The process up to this point is referred to as first vertical scanning for convenience.

  After reading to the V7 line by the first vertical scanning, the vertical scanning circuit returns to the V8 line again. Then, after reading the V8 row, the V9 row is read next.

  The V4 row that has already been read in the first scan is skipped, and the V10 and V11 rows are read.

  Similarly, the V5 row already read in the first scan is skipped, and the V12 and V13 rows are read. The vertical scanning so far is referred to as a second vertical scanning.

  As described above, in the present embodiment, the first readout mode (first vertical scanning) in which the image generation signal is read out from the A pixel + B pixel provided in the partial region in the entire pixel region of the image sensor in synchronization with the synchronization signal. ). Further, a second readout mode (second vertical scanning) in which a focus detection signal is read out from an A pixel (or B pixel) provided in a focus detection area smaller than the partial area in synchronization with the synchronization signal. Have Specifically, by the operation of the vertical scanning circuit 506, the first vertical scanning for reading out a plurality of pixels at a predetermined thinning rate in the vertical direction and the first reading at a thinning rate different from that of the first vertical scanning are performed. Two vertical scans are possible. In addition, the imaging apparatus includes a first operation for generating an image generation signal, a first drive control unit that reads out pixels by a first vertical scan, and a second operation for generating a focus detection signal. And a second drive control means for reading out pixels by the first vertical scanning or the second vertical scanning.

  As described above, the V1 to V7 rows read in the first vertical scan read out the A pixel + B pixel, form a moving image using this output, and the V8 to V13 rows read out in the second vertical scan. Reads out the A pixel and the B pixel and performs phase difference detection using this output.

  As described above, in this embodiment, the entire pixel area of the image sensor is thinned and read out, thereby controlling the image generation signal to be read from the first partial area (V1 to V7 lines) of all the pixel areas. . Further, control is performed so that the focus detection signal is read from the second partial area (line V8 to line V13) different from the first partial area.

  In the present embodiment, full-screen display is read by the first vertical scan and second vertical scan, and enlarged display is performed by dividing the frame described in the first embodiment.

  Next, a third embodiment of the present invention will be described. In this embodiment, a description will be given of the restriction on the readout row in the enlarged phase difference detection frame in the moving image mode.

  It is conceivable to stop reading in the phase difference detection frame of the enlarged display described above. This is because it is assumed that power consumption increases because reading from the image sensor is always performed in all frames.

  For example, when there is no instruction to execute phase difference detection from the user, that is, when SW1 is not pressed, reading of the phase difference detection frame is stopped without executing phase difference detection. In other words, the CPU 105 performs control so that the second readout mode is stopped during a period in which a signal instructing the shooting preparation operation is not received (period in which SW1 is not pressed). Alternatively, after the focus detection operation is completed and after focusing, reading of the phase difference detection frame may be stopped and power saving control of the image sensor may be performed until the focus detection operation starts again.

  FIG. 13 shows a case where reading of the phase difference detection signal is stopped until a user instruction is given.

  Although the time t1 in FIG. 13 is the start time of the phase difference detection frame, the trigger signal SW1 is “H” because SW1 is not pressed by the user.

  At this time, reading of the phase difference detection frame from time t1 to time t2 is stopped.

  At this time, by controlling the currents of various amplifiers of the image sensor 101, the power saving efficiency can be increased.

  For example, the output amplifier 404 shown in FIG. 4, the column amplifier 701 and the follower 704 shown in FIG.

  Further, it is conceivable to perform current control of an I / F (for example, LVDS) between the AFE 102 and the DSP 104.

  Further, control may be performed so as to change the phase difference detection readout region (focus detection region) according to the focus detection frame set in the image sensor. Specifically, a power saving effect can be obtained by reducing the number of readout rows of the phase difference detection frame in accordance with the focus detection frame set in the imaging apparatus.

  FIG. 14 shows the relationship among the crop display area, the focus detection frame, and the phase difference detection readout area.

  For example, FIG. 14A shows a case where the center of the imaging surface is displayed in a cropped manner, and the crop display area is represented by an area 1401. The focus detection frame is a region 1402, which is a region indicated by dots.

  Further, an area surrounded by a thick frame indicates an area read by the phase difference detection frame. That is, in the example of FIG. 14A, the entire range of the focus detection frame is read with the phase difference detection frame.

  Next, in FIG. 14B, the focus detection frame is a region 1403, and a region 1404 surrounded by a thick frame is a region read by the phase difference detection frame.

  This is different from the example shown in FIG. 14A in that the area to be read out by the phase difference detection frame is extended to the horizontal direction area of the crop display area 1401. Can be tracked.

  Further, FIG. 14C shows an example in which there are a plurality of focus detection frames. Areas 1405 and 1406 surrounded by a thick frame are areas to be read out in the phase difference detection frame.

  In common with FIGS. 14A to 14C, in the phase difference detection frame, the area other than the area surrounded by the thick frame is a non-read target.

  As described above, the power saving effect can be obtained by limiting the readout region of the phase difference detection frame in accordance with the focus detection frame set in the imaging apparatus.

  Further, the phase difference detection readout area (focus detection area) may be changed according to the luminance of the subject. Specifically, it is possible to save power by changing the number of read rows of the phase difference detection frame in accordance with the luminance level of the subject.

  For example, in the case of a bright subject, the number of readout rows in the phase difference detection frame is thinned out to limit the number of readout rows, and in the case of a dark subject, readout is performed without thinning out. In other words, the CPU 105 controls to read out the focus detection area when the luminance of the subject is larger than the predetermined value, and reads out the focus detection region when the luminance of the subject is lower than the predetermined value. To control.

  As described above, it is possible to obtain a power saving effect by adaptively limiting the number of read lines in the phase difference detection frame in accordance with a user's shooting instruction, a focus detection frame preset in the imaging apparatus, and a luminance level of the subject. It is possible.

  Furthermore, power saving efficiency can be further improved by performing current control of various amplifiers of the image sensor 101 and I / F current control between the AFE 102 and the DSP 104 in the non-reading period.

  Next, a fourth embodiment of the present invention will be described. In this embodiment, a power saving method in the case of using a CMOS type image sensor that supports digital output will be described.

  The above-described image pickup apparatus is premised on the combination of a CMOS image pickup element corresponding to analog output and an AFE, but a CMOS image pickup element compatible with digital output can also be used.

  In this case, since the digital output is naturally obtained from the image sensor, the AFE 102 is not necessary.

  Accordingly, a configuration in which the output of the image sensor 101 is directly input to the DSP 104 is assumed as the imaging system.

  FIG. 15 shows an example of the configuration of a digital output type image sensor.

  The pixels 1501 are arranged in a matrix, and the pixels in the vertical direction are connected by a vertical output line 1502.

  The vertical output line 1502 is connected to the input of the column amplifier 1503.

  The column amplifier 1503 multiplies the input from the vertical output line by a predetermined gain and outputs it to the comparator 1505. A control switch 1504 controls the power supply of each column amplifier.

  The comparator 1505 is prepared for each column, and a vertical output line is connected to one input, and a lamp source 1506 is connected to the other input. A control switch 1507 controls the power supply of each comparator.

  The comparator 1505 compares the output of the vertical output line with the ramp signal output from the lamp source.

  The latch unit 1508 holds the count value of the counter 1509 according to the comparison result of the comparator.

  The column memory 1510 holds the result latched by the latch unit 1508 in units of rows. The data held in the column memory 1510 is sequentially output outside the image sensor by a horizontal transfer circuit (not shown).

  When the digital output type image sensor shown in FIG. 15 is applied to the imaging sequence that divides the moving image generation frame and the phase difference detection frame as described above, per unit pixel in the moving image generation frame and the phase difference detection frame. It is conceivable to change the number of constituent bits.

  For example, since a moving image generation frame is used for moving image recording or monitor display by a display medium, it is composed of 14-bit data per unit pixel in order to ensure gradation of data.

  On the other hand, since the signal read in the phase difference detection frame is used for the phase difference detection calculation, it is considered that there is no problem even if the number of bits is reduced with respect to the moving image generation frame. Therefore, for example, it is composed of 12-bit data.

  Therefore, the ratio of the readout period in the phase difference detection frame is reduced, and the ratio of the non-readout period is relatively increased, so that a power saving effect can be obtained.

  Further, by turning off the power control switch 1504 of the column amplifier 1503 and turning off the power control switch 1507 of the comparator 1506 during the non-reading period, the power consumption of the image sensor can be suppressed and a further power saving effect can be obtained.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

  Also, when a software program that realizes the functions of the above-described embodiments is supplied from a recording medium directly to a system or apparatus having a computer that can execute the program using wired / wireless communication, and the program is executed Are also included in the present invention.

  Accordingly, the program code itself supplied and installed in the computer in order to implement the functional processing of the present invention by the computer also realizes the present invention. That is, the present invention includes a computer program itself in which a procedure for realizing the functional processing of the present invention is described.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS. As a recording medium for supplying the program, for example, a magnetic recording medium such as a hard disk or a magnetic tape, an optical / magneto-optical storage medium, or a nonvolatile semiconductor memory may be used.

  As a program supply method, a computer program that forms the present invention is stored in a server on a computer network, and a connected client computer downloads and programs the computer program.

  The present invention can be suitably used for an imaging apparatus such as a compact digital camera, a single-lens reflex camera, and a video camera.

105 CPU

Claims (16)

A control device that controls an imaging device including a first photoelectric conversion unit and a second photoelectric conversion unit for one microlens,
A first readout mode for reading out an image generation signal from the first photoelectric conversion unit and the second photoelectric conversion unit provided in a partial region of the pixel region of the image sensor in synchronization with a synchronization signal, and the synchronization A second readout mode for reading out a focus detection signal from the first photoelectric conversion unit or the second photoelectric conversion unit provided in a focus detection region smaller than the partial region in synchronization with a signal; Control means for controlling
The control device, wherein the control means switches between the first readout mode and the second readout mode in accordance with the synchronization signal.   The control device according to claim 1, wherein the synchronization signal is a timing signal for starting creation of data of each frame during continuous shooting.   The control device according to claim 1, wherein the control unit switches between the first readout mode and the second readout mode alternately for each frame.   4. The control device according to claim 1, wherein the control unit performs control so as to read out the image generation signal from the partial region in a non-decimated manner. 5.   5. The control apparatus according to claim 1, wherein the control unit changes the focus detection area in accordance with a focus detection frame set in the image sensor. 6.   The control means controls to read out the image generation signal from a first partial area of the entire pixel area by thinning out and reading out the entire pixel area of the imaging element, 4. The control device according to claim 1, wherein control is performed so that the focus detection signal is read from different second partial areas. 5.   7. The control device according to claim 1, wherein the control unit stops the second readout mode during a period in which a signal for instructing an imaging preparation operation is not received.   The control device according to claim 1, wherein the control unit changes the focus detection area according to a luminance of a subject.   The control device according to claim 8, wherein the control unit reads out the focus detection area when the luminance of the subject is larger than a predetermined value. The control means causes the display device to update the display at a predetermined cycle,
The control apparatus according to claim 1, wherein the predetermined period is twice the period of the synchronization signal.   11. The control device according to claim 1, wherein the number of read rows in the second read mode is equal to or less than ½ of the number of read rows in the first read mode. An image sensor comprising a first photoelectric conversion unit and a second photoelectric conversion unit for one microlens;
An imaging device comprising: the control device according to claim 1. The image sensor has a plurality of pixels,
Accumulation that holds charges generated by the microlens, the first photoelectric conversion unit and the second photoelectric conversion unit, and the first photoelectric conversion unit and the second photoelectric conversion unit in each pixel Capacity, and
Generating the image generation signal in a first operation of transferring charges of the first photoelectric conversion unit and the second photoelectric conversion unit to the storage capacitor;
Generating the focus detection signal in a second operation of transferring the charge of the first photoelectric conversion unit or the charge of the second photoelectric conversion unit to the storage capacitor;
A vertical scanning circuit capable of performing a first vertical scanning for reading out the plurality of pixels by thinning out at a predetermined thinning rate in the vertical direction and a second vertical scanning for reading out the plurality of pixels by thinning out at a thinning rate different from the first vertical scanning. When,
First drive control means for reading out pixels by the first operation and the first vertical scanning;
Second drive control means for reading out pixels by the second operation and the first vertical scanning or the second vertical scanning;
The imaging apparatus according to claim 12, comprising: A method for controlling an imaging apparatus having an imaging element including a first photoelectric conversion unit and a second photoelectric conversion unit for one microlens,
A first readout mode for reading out an image generation signal from the first photoelectric conversion unit and the second photoelectric conversion unit provided in a partial region of the pixel region of the image sensor in synchronization with a synchronization signal, and the synchronization A second readout mode for reading out a focus detection signal from the first photoelectric conversion unit or the second photoelectric conversion unit provided in a focus detection region smaller than the partial region in synchronization with a signal; A control step for controlling,
A control method comprising: a switching step of switching between the first readout mode and the second readout mode in accordance with the synchronization signal.   A computer-executable program in which each step of the control method according to claim 14 is described.   A computer-readable storage medium storing a program for causing a computer to execute each step of the control method according to claim 14.