JP6525498B2 - Image pickup apparatus, control method thereof and control program - Google Patents

Description

  The present invention relates to an imaging device, and more particularly to an imaging device capable of image plane phase difference autofocus (hereinafter referred to as image plane phase difference AF).

  Conventionally, Japanese Patent Application Laid-Open No. 2008-112147 discloses a technique for suppressing image quality deterioration of a moving image and live view caused by focus detection pixels while realizing image plane phase difference AF.

JP, 2010-181751, A

  However, in the prior art disclosed in Patent Document 1 described above, since the focus detection pixels and the image pixels are read by the same output system (stream), the acquisition rate of the image plane phase difference AF image is that of the image pixels. It was regulated by the read rate. For this reason, the number of times of sampling of the focus detection pixel per unit time is limited, which hinders the improvement of the AF accuracy.

  Therefore, an object of the present invention is to provide an imaging device capable of improving the AF accuracy by increasing the number of times of sampling of focus detection pixels per unit time in the image plane phase difference AF.

According to the present invention, an imaging unit having a pixel array in which a plurality of pixels each including a photoelectric conversion unit, and a plurality of microlenses shared by a plurality of adjacent pixels are two-dimensionally arranged, and a microlens reads the first pixel signal corresponding plurality of pixels in the subject image formed by the photographing optical system having a focus lens to share, different pupil regions of the photographing optical system from each of a plurality of pixels sharing a microlens a reading means for reading the second pixel signal corresponding to the subject image from the partial region and a second output system for outputting a first output system and the second pixel signal for outputting a first pixel signal with an output unit, a pixel signal outputted from the output means, and signal processing means for generating an evaluation value for performing focusing of the focus lens, the reading means includes first pixel signal Preliminary the second pixel signal, the image pickup apparatus characterized by reading at different read rates, respectively, are provided.

  According to the present invention, it is possible to increase the number of times of signal sampling of focus detection pixels for performing image plane phase difference AF without being restricted by the readout rate of image generation pixels, and the AF accuracy is improved. It becomes possible to provide an imaging device.

The figure which shows the structure of the imaging device which concerns on the 1st Example of this invention. The figure which shows the pixel structure of the image pick-up element concerning the 1st and 2nd Example of this invention. The figure which shows typically the relationship between the pixel of the image pick-up element which concerns on the 1st and 2nd Example of this invention, and an optical part. FIG. 7 is a timing chart of the pixel signal readout operation of the imaging device according to the first embodiment of the present invention. The figure which shows the flowchart of imaging | photography operation | movement of the imaging device which concerns on the 1st Example of this invention. The figure which shows the flowchart of imaging | photography operation | movement of the imaging device which concerns on the 3rd Example of this invention.

  Hereinafter, preferred embodiments of the present invention will be described in detail based on the accompanying drawings.

  The configuration and operation of the imaging apparatus according to the first embodiment of the present invention will be described below with reference to FIGS. 1 to 5.

FIG. 1 is a diagram showing the configuration of an imaging apparatus 1000 according to a first embodiment of the present invention.
In FIG. 1, reference numeral 1001 denotes an optical unit for forming a subject image on the light receiving unit of the imaging device. In the optical unit 1001, reference numeral 1001-1 denotes a zoom lens, reference numeral 1001-2 denotes a focus lens, reference numeral 1001-3 denotes an IS (image stabilizer) lens, and reference numeral 1001-4 denotes an aperture.

  An imaging element 1002 converts an optical signal of an image formed by the optical unit 1001 into an electrical signal by photoelectric conversion, and has a two-dimensional pixel array configured by a CMOS sensor or the like. The imaging element 1002 performs OB clamping and AD conversion on an imaging signal (pixel signal), generates a digital image signal and a focus detection signal, and outputs the digital image signal and a focus detection signal to a DSP 1007 described later. Further, as described later, the imaging element 1002 has two output systems of a first stream that outputs an image pixel signal and a second stream that outputs a focus detection pixel signal.

  A CPU 1004 controls the entire system. A ROM 1005 stores information such as firmware for driving the imaging apparatus 1000, and a RAM 1006 temporarily stores control information of the imaging apparatus 1000 and an output of the imaging device 1002. The CPU 1004 controls the operation of each unit of the imaging apparatus 1000 by executing a program stored in the ROM 1005 using the RAM 1006.

  Reference numeral 1007 denotes a signal processing unit (hereinafter referred to as a DSP (digital signal processor)). The DSP 1007 performs various signal processing on the signal from the first stream from the imaging device 1002, and generates a still image or a moving image signal (for example, a YUV signal or the like) in a predetermined format. Further, based on the signal from the first stream of the image sensor 1002, the DSP 1007 performs an exposure control operation at the time of live view driving and moving image recording and detection of a main subject. Further, the DSP 1007 detects a subject distance from the imaging device at the time of live view driving or moving image recording based on the first stream and the signal from the second stream.

  An external interface 1008 includes various encoders and a D / A converter, and exchanges various control signals and data with an external element (in this example, the display 1012, the memory medium 1009, and the operation panel 1011).

  Reference numeral 1009 denotes a memory medium in which images captured on various memory cards or the like can be stored appropriately. Reference numeral 1010 denotes a memory medium controller capable of exchanging the memory medium 1009. In addition to various memory cards, a disk medium using magnetism or light may be used as the memory medium 1009.

  Reference numeral 1011 denotes an operation panel provided with input keys for the user to issue various instructions when the imaging apparatus 1000 performs a shooting operation. The CPU 1004 monitors an input signal from the operation panel 1011 and executes various operation control based on the input content.

  A display 1012 is incorporated in the imaging apparatus 1000 and displays an image captured by the imaging element 1002. The image pickup apparatus 1000 can of course be configured to transmit image data to an external display apparatus in addition to the built-in display apparatus, and to display the image data.

  FIG. 2 is a diagram showing the pixel configuration of the imaging device according to the embodiment of the present invention, and FIG. 2 (a) shows the configuration of the pixel portion of the imaging device according to the present embodiment, and FIG. 7 shows a configuration of a pixel unit according to a second embodiment described later.

  In FIG. 2A, reference numeral 2001-1 denotes a pixel (first pixel) for obtaining an image signal, and the signal of this pixel is read out from the first stream.

  Reference numeral 2001-2 denotes a focus detection pixel (second pixel) formed by shifting the aperture of the pixel in the first horizontal direction, and an image of the pixel 2001-3 in the horizontal direction described later is formed. It is a reference pixel for detecting the amount of deviation. The signal of pixel 2001-2 is read from the second stream.

  Reference numeral 2001-3 denotes a focus detection pixel (third pixel) formed by shifting the opening of the pixel in the opposite direction (second direction) to the pixel 2001-2. It is a pixel for detecting the image shift amount in the horizontal direction. The signal of pixel 2001-3 is read from the second stream.

  In this embodiment, the first pixel 2001-1, the second pixel 2001-2, and the third pixel 2001-3 are arranged row by row to form a pixel portion. However, the focus detection pixel of this embodiment can not be used for an image because it receives only the light passing through the partial area of the pupil. Therefore, in the present embodiment, the pixel arrangement for focus detection is discretely arranged at a certain interval in the row direction, for example, in a 4-row cycle.

  2003 is a vertical signal line for reading out the pixel signal, and 2002 is a selection transistor for outputting the pixel signal to the vertical signal line 2003. Reference numeral 2004 denotes a vertical scanning circuit which selectively supplies voltages to the selection lines ROW-1 to ROW-n and scans pixel signals.

  ROW-1 to ROW-n are selection lines for operating the selection transistor 2002. Here, the subsequent numbers represent line numbers in ascending order from the top of the screen, and the total number of lines is n.

  Reference numeral 2005 denotes an AD converter (hereinafter referred to as an ADC) which receives the signal of the vertical signal line 2003 and converts it into a digital signal. Reference numeral 2006 denotes a horizontal scanning circuit for outputting a pixel signal converted into a digital signal to the output selector 2007. An output selector 2007 switches the output destination of the signal from the horizontal scanning circuit 2006. An output destination is selected between the first stream and the second stream depending on whether the pixel to be read out is an image pixel or a focus detection pixel, and an imaging signal is output.

  FIG. 3 is a view schematically showing the relationship between the arrangement of pixels of the image sensor 1002 and the optical unit 1001 according to this embodiment. In the figure, parts corresponding to those in FIG. 2 are denoted by the same reference numerals as in FIG. FIGS. 3A, 3 B, 3 C and 3 D show the array structure of the image pixels 2001-1 according to the present embodiment, the focus detection pixels 2001-2 and 2001-3, and the optical unit 1001. It is a figure which shows typically the relationship of. FIGS. 3E and 3F schematically show the relationship between the arrangement of pixels and the optical unit 1001 according to the second embodiment to be described later. The pixel according to the second embodiment is a pixel capable of generating both an image signal and a focus detection signal.

  The pixel array of the pixel section in the present embodiment has a unit of 4 pixels of 2 × 2, and a pixel having a spectral sensitivity of G (green) is arranged on two diagonal pixels thereof, and R (red) is arranged on the other 2 pixels. A Bayer array is employed in which one pixel having spectral sensitivity of B and B (blue) is arranged. Then, the pixel array of the pixel section is configured by dispersively arranging focus detection pixels having a structure to be described later according to a predetermined rule between the image pixels of the Bayer array.

  FIGS. 3A and 3B schematically show the relationship between the arrangement structure of image pixels and the optical unit 1001. FIG. 3 (a) is a plan view of a 2 × 2 image pixel, and FIG. 3 (b) shows a cross section A-A of FIG. 3 (a).

  As shown in FIG. 3A, in the Bayer arrangement, G pixels are arranged in two diagonal pixels, and R and B pixels are arranged in the other two pixels, and the structure of 2 rows × 2 columns is repeated. Make up the department. In FIG. 3B, ML is an on-chip micro lens disposed at the front of each pixel, CFR is a color filter of R, and CFG is a color filter of G. Further, PD schematically shows the photoelectric conversion portion of the pixel of the CMOS sensor of FIG. 2A, CL denotes a wiring layer forming a signal line for transmitting various signals in the CMOS sensor, and TL denotes an optical portion 10 schematically shows a photographing optical system 1001.

  Here, the on-chip microlens ML of the image pixel and the photoelectric conversion unit PD are configured to take in the light flux that has passed through the photographing optical system TL as effectively as possible. In other words, the exit pupil EP of the photographing optical system TL and the photoelectric conversion unit PD are in a conjugate relationship by the microlens ML, and the effective area of the photoelectric conversion unit is designed to be large.

  Further, although only the incident light of the R pixel is shown in FIG. 3B, the G pixel and the B pixel also have the same structure. Therefore, the exit pupil EP corresponding to each pixel of RGB for image has a large diameter, and the luminous flux from the subject is efficiently taken in to improve the S / N of the image signal.

  3C and 3D show the relationship between the arrangement of focus detection pixels and the optical unit 1001 for performing pupil division (division of the pupil region) in the horizontal direction (lateral direction in the drawing) of the photographing lens. It is a figure shown typically. FIG. 3D shows a cross section B-b in FIG. 3C, in which the microlens ML and the photoelectric conversion unit PD have the same structure as the image pixel shown in FIG. 3B. .

  In the present embodiment, as shown in FIG. 3C, of the 2 rows × 2 columns of pixels which are units of Bayer arrangement (for example, 2 × 2 of ROW-3 and ROW-4 of FIG. 2A). Pixels) and G pixels are left as image pixels, and R and B pixels are formed as focus detection pixels. These focus detection pixels are shown as a pixel SA and a pixel SB in FIG. 3C. Here, the pixel SA corresponds to the second pixel 2001-2, and the pixel SB corresponds to the third pixel 2001-3.

  In the present embodiment, since the pixel signal of the focus detection pixel is not used for image display or image recording, transparent films CFWA and CFWB are disposed instead of the color separation color filter in order to improve the light reception sensitivity. There is.

  In addition, in order to perform pupil division by the imaging element, the opening of the wiring layer CL is shifted in one direction horizontal to the center line of the microlens ML to restrict the direction of light incident on the pixel. There is. Specifically, since the opening part OPHA of the pixel SA is shifted to the right in the figure, the light flux which has passed through the exit pupil EPH1A on the left side of the photographing lens TL1 is received. Similarly, since the opening OPHB of the pixel SB is shifted to the left, the light beam passing through the exit pupil EPH1B on the right side of the photographing lens TL1 is received. The pixels SA are regularly arranged in the horizontal direction (row direction of the pixel array), and an object image acquired by these pixels is an A image. In addition, assuming that the pixels SB are also regularly arranged in the horizontal direction, and the subject image acquired by these pixels is the B image, the relative position of the A and B images is detected to obtain the defocus amount of the subject image ( The focus amount can be detected.

  Further, when it is desired to detect the focus shift amount in the vertical direction (in the same column direction), the pixel SA and its opening OPHA may be shifted upward and the SB and its opening OPHB may be shifted downward.

  The arrangement configuration of the focus detection pixels is not limited to that shown in FIG. 2A, and all pixels in one row may be configured by the pixels SA and the pixels SB.

  FIG. 4 is a timing chart of the pixel signal readout operation of the imaging device according to the present embodiment. Here, reading of pixel signals in a state in which both the first stream and the second stream are operating will be described for the operation from time t1 to t21 when the time (t) is divided at equal intervals. . Here, the time of equal intervals is set to 1/480 sec, and the second stream reads out the focus detection pixels at a high speed of 240 fps, and the image pixels are divided by 1/4 from the first stream based on this. Read out at 30 fps.

  In FIG. 4, at t1, reading of the image pixel signal is started. The output of the imaging element 1002 is read from the upper portion (ROW-1) of the screen to the lower portion (ROW-n) of the screen, and pixel signals are sequentially read in row units.

  At t2, readout of the image pixel signal is completed to the lower part of the screen. Furthermore, at t2, reset of the image pixels is started from the top of the screen.

  At t3, the reset of the image pixels is completed to the lower part of the screen. From t4 to t8, readout and reset of the image pixel and the focus detection pixel are not performed.

  Next, at t9, readout of the image pixel signal is started from the top of the screen. Furthermore, at t9, reset of focus detection pixels is started from the top of the screen.

  Next, at t10, readout of the image pixel signal is completed to the lower part of the screen, and reset of the image pixels is started from the upper part of the screen. Furthermore, at t10, reset of the focus detection pixels is completed to the lower part of the screen, and readout of the focus detection pixel signals is started from the upper part of the screen.

  Next, at t11, reset of the image pixels is completed to the lower part of the screen. Furthermore, at t11, readout of the focus detection pixel signal is completed to the lower part of the screen, and reset of the focus detection pixels is started from the upper part of the screen.

  Next, at t12, readout of the focus detection pixel signal is started from the top of the screen, and reset of the focus detection pixels is completed to the bottom of the screen.

  At t13, readout of the focus detection pixel signal is completed to the lower part of the screen, and reset of the focus detection pixels is started from the upper part of the screen.

  The same operation as t12 is performed at t14, the same operation as t13 is performed at t15, and the same operation as t14 is performed at t16.

  Next, at t17, reset of the focus detection pixels is completed to the lower part of the screen, and readout of image pixel signals is started from the upper part of the screen. Thereafter, readout of the image pixel signal is performed from the first stream.

  Next, an operation similar to t2 is performed at t18, an operation similar to t3 is performed at t19, an operation similar to t4 is performed at t20, and an operation similar to t5 is performed at t21.

  As described above, in this embodiment, the second stream is operated only during the necessary period from t9 to t17 to read out the focus detection pixels in a short time, so that high precision AF can be realized.

  In the case of the pixel array in FIG. 3C (for example, ROW-3 and ROW-4 in FIG. 2A), when pixel signals of all the columns in one row are output, the second stream is output. In addition to the signal of the focus detection pixel, the image pixel signal is also included. As described above, although the output from the output selector 2009 may be used, if only the signals of the second pixel and the third pixel are selectively output by the horizontal scanning circuit 2008, the number of output pixels of the second stream is reduced. The frame rate can be improved. In particular, when high precision AF such as auto focus before still image shooting is required, sampling is performed by improving the frame rate by outputting only the signal of the second pixel and the third pixel as described above. The number increases, and high precision AF becomes possible.

  Further, in the pixel portion of the image pickup element of this embodiment, since the first signal stream and the second signal stream share the vertical signal line 2003 to the output selector 2007, the readout of the image pixel and the focus detection pixel are performed. The readout periods do not overlap.

  Further, in FIG. 4, the reset operation of the focus detection pixel is performed at t9, but the timing thereof is the same timing as the readout of the signal of the image pixel here for simplification of the description. However, in practice, there is no problem if the reset timing is shifted in order to change the exposure time according to the appropriate exposure condition.

  In addition, using signals of focus detection pixels for a plurality of frames (a plurality of exposures), signals outside 1σ are excluded from median processing, addition averaging processing, and signal value variations (dispersion: σ) for a plurality of frames. It is possible to improve the accuracy of AF by adopting a method such as performing averaging processing.

  In addition, when acquiring signals of focus detection pixels for a plurality of frames, the IS lens is operated to acquire signals shifted in phase with respect to the pixel pitch, thereby improving spatial resolution and improving AF accuracy. It becomes possible.

  Furthermore, as described above, by setting the frame rate of the first stream to be an integral fraction of the second stream, it becomes possible to design an easy-to-handle timing on the electric circuit.

  The photographing operation of the imaging apparatus 1000 of this embodiment configured as described above will be described with reference to FIG. FIG. 5 is a diagram showing a flowchart of the photographing operation of the imaging apparatus 1000 according to the present embodiment. The photographing operation is realized by the CPU 1004 executing a program stored in the ROM 1005 to control the operation of each unit of the apparatus 1000.

  First, when the imaging apparatus 1000 is powered on in step S5001, the imaging apparatus is activated.

  In step S5002, when live view (or moving image recording) is started according to the operation mode of the imaging apparatus, each live view control parameter is initialized under control by the CPU 1004 to read out an image pixel signal, and processing is performed. Transfer to S5003.

  In step S5003, under the control of the CPU 1004, a so-called live view operation is performed in which signals are continuously read from the image sensor and display and recording are sequentially performed. Further, photometry is performed by a photometry detection unit configured by the DSP 1007 and the CPU 1004, and exposure conditions such as an accumulation time of a photoelectric conversion signal, a gain, and an aperture are determined.

  The CPU 1004 operates the diaphragm 1004-2 based on the determined diaphragm value to drive the diaphragm.

  Here, switching of the read pattern from the first stream to the second stream from step S5004 to step S5013 will be described.

  In step S5004, the CPU 1004 selects a readout pixel and selects an AF method in accordance with the state of a not-shown lens or an AF or manual focus (MF) switch installed in the camera. When the MF is selected, the signal of the focus detection pixel is not read out, and a notification of the end of live view and moving image recording is awaited. If AF is selected, the process proceeds to step S5005. Here, the AF / MF mode switching may be semi-automatically changed by moving a menu or a focusing ring, and is not limited to a physical switch.

  In step S5005, the CPU 1004 determines whether or not the automatic focusing function (AF) operation is performed, and while the AF operation is not performed, readout of the signal of the focus detection pixel is not performed, and the live view and the moving image are performed. Wait for notification of end of recording. If the AF operation is being performed, the CPU 1004 advances the process to step S5006.

  In step S5006, the CPU 1004 selects whether to operate the second stream according to the state of the aperture set as the current exposure control. This is because, when the stop is narrowed by a predetermined amount or more, the phase difference of the incident light to the focus detection pixels SA and SB becomes small, and the focus detection is not suitable. In the present embodiment, the threshold value of the amount of reduction is set to F8.0, but is not limited to this.

  If the aperture is narrowed by F8.0 or more, the CPU 1004 shifts the processing to step S5008. In step S5008, the AF evaluation value is acquired using the image data from the first stream. The acquisition of the AF evaluation value at this time may be, for example, a known contrast AF technique.

  If the aperture is controlled to the open side from F8.0, the CPU 1004 shifts the processing to step S5012.

  In step S5012, under control of the CPU 1004, acquisition of an AF evaluation value using pixel data of the second stream is performed. Further, in order to pick up data of focus detection pixels and detect the focus state of the imaging lens, the data is also transferred to a phase difference detection unit (not shown) in the DSP 1007. Then, in this circuit block, correlation calculation of the pupil-divided SA pixel group and the SB pixel group is performed to calculate a phase difference AF evaluation value.

  In step 5013, the CPU 1004 drives the focus lens 1001-2 according to the evaluation value of the previous step (S5012 or S5008) to adjust the focus of the optical unit 1001.

  In step S5014, when the end of live view and moving image recording is notified, the CPU 1004 performs an end process of live view and moving image recording, and then the camera shifts to a standby state. On the other hand, when there is no notification of the end, the process proceeds to step S5004 to continue the live view and moving image recording process.

  As described above, according to the present embodiment, the number of times of sampling of the focus detection pixel can be increased without being restricted by the readout rate of the image signal pixel, and imaging with improved AF accuracy can be performed. It becomes possible to provide an apparatus.

  Furthermore, by changing the AF method according to the state of the camera at the time of reading during live view and moving image recording, it becomes possible to read out the focus detection pixel signal only when it is effective, thereby reducing power consumption. Is possible.

  Hereinafter, the configuration and operation of an imaging apparatus according to the present embodiment will be described with reference to the attached drawings, according to a second embodiment of the present invention. The configuration of the imaging apparatus according to the embodiment of the present invention is the same as that of the first embodiment, and thus the description thereof is omitted here. The difference from the first embodiment is the structure of the pixel portion of the image sensor 1002. Hereinafter, the imaging device of the present embodiment will be described according to the difference from the first embodiment with reference to FIGS. 2 (b) and 3 (e) and 3 (f).

  First, the structure of a pixel according to the present embodiment will be described with reference to FIGS. 3 (e) and 3 (f). FIGS. 3E and 3F schematically show the relationship between the structure of the pixel array and the optical unit 1001 according to this embodiment.

  FIG. 3E is a plan view of 2 rows by 4 columns of pixels, which is a basic unit of the pixel array of the imaging device according to the present embodiment. For each color of the Bayer array, two adjacent pixels (one row and two columns) are formed, and these two pixels share one microlens ML2. The positional relationship between the photoelectric conversion unit of the 1 × 2 pixel and the microlens ML2 is divided into pupils in the horizontal direction (lateral direction) of the photographing lens as shown in FIG. 3F. It is an arrangement configuration that can Therefore, each of the two pixels functions as a focus detection pixel for acquiring images of different phases. Further, by adding the signals (the same color) of these two pixels, it is possible to generate a pixel signal corresponding to the pixel signal of the image pixel 2001-1 of the first embodiment. Therefore, in the pixel array structure of the present embodiment, it is necessary to dispose color filters for all pixels.

  FIG. 3F is a view showing a cross section C-c (two adjacent G pixels) of FIG. 3E. As shown in the figure, light having passed through the microlens ML2 is divided into pupils (division of a pupil region) and is incident on the photoelectric conversion unit PD2-A and the photoelectric conversion unit PD2-B. In order to perform pupil division by the imaging element, the photoelectric conversion unit PD2-A is shifted in one direction in the horizontal direction with respect to the center line of the microlens ML2. The photoelectric conversion unit PD2-B is shifted in the opposite direction to the photoelectric conversion unit PD2-A with respect to the center line of the microlens ML2.

  Accordingly, the pixel Ga receives the light flux that has passed through the exit pupil EPH2A on the right side of the imaging lens TL. Similarly, the pixel Gb receives the light flux that has passed through the exit pupil EPH2B on the left side of the imaging lens TL.

  The pixels Ga are regularly arranged in the horizontal direction, and an object image obtained by these pixels is an A image. In addition, assuming that the pixels Gb are also regularly arranged in the horizontal direction, and the subject image acquired by these pixels is a B image, the relative position of the A and B images is detected to obtain the defocus amount of the subject image ( The focus amount can be detected.

  If it is desired to detect the amount of defocus in the vertical direction (longitudinal direction), the arrangement of the pixels Ga and Gb may be rotated by 90 degrees.

  The R pixel and the B pixel also have the same structure as that of the G pixel described above.

  FIG. 2B shows the configuration of the pixel array of the imaging unit of the imaging device according to the present embodiment. In FIG. 2 (b), elements 2002 to 2004 and 2006, 2007 are the same as FIG. 2 (a), so the description thereof is omitted here.

  In FIG. 2B, all the arranged pixels 2001-4 are pixels formed by shifting the photodiode PD2-A shown in FIG. 3F in the horizontal direction with respect to the microlens ML2, It is a reference pixel for detecting the image shift amount of the horizontal direction of the pixel 2001-5 described later. For example, the pixel Ga in FIG. 3E corresponds to the pixel 2001-4.

  Reference numeral 2001-5 is a pixel formed by shifting the photodiode PD2-B in the opposite direction to the pixel 2001-4, and is a pixel for detecting an image shift amount in the horizontal direction with the pixel 2001-4. The pixel Gb in FIG. 3E corresponds to the pixel 2001-5, for example.

  In this embodiment, the mixed signal of the pixel 2001-4 and the pixel 2001-5 is output from the first stream, and the output mixed signal is processed by the DSP 1007 as an image signal. In the present embodiment, the signals of the pixels 2001-4 and 2001-5 are mixed by the ADC 2005, but the details are known and thus the description thereof is omitted.

  The signal of the pixel 2001-4 and the signal of the pixel 2001-5 are individually output from the second stream as a focus detection pixel signal.

  Thus, the pixel signal for image output from the first stream is treated in the same manner as in the first embodiment by the DSP 1007 to generate a photographed image, and the pixel for focus detection output from the second stream The signal is used to detect the amount of image shift in the horizontal direction.

  The timing chart of the imaging apparatus configured by the imaging element having the pixel array structure according to the present embodiment described above is the same as that of the first embodiment, and thus the description thereof is omitted here. Further, the flowchart of the photographing operation of the imaging apparatus according to the present embodiment is also the same as that of the first embodiment, and thus the description thereof will be omitted.

  As described above, even in the pixel array structure as in this embodiment, the number of times of sampling of focus detection pixels can be increased without being restricted by the readout rate of image generation pixels, and AF accuracy can be improved. It is possible to provide an imaging device that

  Furthermore, by changing the AF method according to the state of the camera at the time of reading during live view and moving image recording, it becomes possible to read out the focus detection pixel signal only when it is effective, thereby reducing power consumption. Is possible.

  The configuration and operation of an imaging apparatus according to the third embodiment of the present invention will be described below with reference to FIG.

  The configuration of an imaging apparatus according to this embodiment is the same as that of the first embodiment, the pixel configuration of the imaging unit is the same as that of the second embodiment, and the configuration of the optical unit is the same as that of the second embodiment. The same is true, and the timing chart of the imaging apparatus is the same as that of the first embodiment, and therefore the description thereof is omitted here.

  The photographing operation of the imaging apparatus according to the present embodiment configured as described above will be described with reference to FIG.

  FIG. 6 is a diagram showing a flowchart of the photographing operation of the imaging device according to the present embodiment. In the figure, the same parts as those in FIG. 5 are denoted by the same reference numerals, and the description thereof will be omitted here unless particularly required.

  First, in step S6001, the power of the imaging apparatus is turned on to drive the imaging apparatus. Thereafter, the operations from step S5002 to step S5006 are performed, but since these are the same as in the first embodiment, the description here is omitted. In this embodiment, switching of the read pattern according to the result of the determination in step S5006 will be described using steps S6004 to S6013.

  If it is determined in step S5006 that the aperture is narrowed by F8.0 or more, the process proceeds to step S5008. Step S5008 is the same operation as in the first embodiment, and acquires an AF evaluation value from the first stream.

  If it is determined in step S5006 that the aperture is controlled to be more open than F8.0, the process proceeds to step S6007. As in the first embodiment, the threshold of the amount of narrowing is not limited to F8.0.

  In step S6007, the CPU 1004 determines the presence or absence of enlarged display during AF and selects an operation. The determination of the presence / absence of enlargement processing may be performed using an appropriate method, for example, determining whether it is better to use the pixel data of the second stream for generation of the captured image according to the magnification set for the electronic zoom. Good.

  If it is determined that enlarged display is present, the process proceeds to step 6009, where the output signal of the second stream is added and used as an image signal to perform an operation for realizing high-definition live view or moving image recording. On the other hand, if it is determined in step S6007 that enlargement display is not performed, the process advances to step S5012, and the operation giving priority to power consumption using only the output of the second stream for detecting the distance to the subject is performed. Note that steps S5012 and S5013 are the same focusing operations as in the first embodiment, and thus the description thereof is omitted here.

  In step S6009, the AF evaluation value is calculated using the focus detection pixel signal (non-addition signal) output from the second stream, and the signal of the focus detection pixel located below the common microlens ML2 is calculated. The addition is performed in the DSP 1007 to generate an image signal. At this time, it is desirable to add the frames of the second stream as much as the exposure conditions of the second stream and the exposure conditions of the first stream become equal. For example, if the exposure time of the first stream is 1/30 sec equal to the frame rate and the exposure time of the second stream is 1/240 sec equal to the frame rate, the second exposure time is equal to the second exposure time. Stream for 8 frames.

  In step S6010, a focusing operation is performed based on the AF evaluation value acquired in step S6009.

  In step S6011, a high-definition image signal is synthesized from the image signal generated by adding the outputs of the second stream in step S6009 and the image signal of the first stream.

  In step S5014, as in the first embodiment, when the end of live view and moving image recording is notified, live view and moving image recording end processing is performed, and then the camera is put in a standby state. On the other hand, if there is no notification of the end, the process proceeds to step S5004, and the live view and moving image recording operation is continued.

  As described above, according to the present embodiment, in addition to the same effects as in the first and second embodiments, the live view is enlarged by using the focus detection pixel signal as an image signal. Sometimes it is possible to display high definition images.

  The object of the present invention can also be achieved by supplying a storage medium storing program code of software for realizing the functions of the above-described embodiments to a system or apparatus. That is, it goes without saying that the object of the present invention can be achieved by the computer (or CPU or MPU) of the system or apparatus reading and executing the program code stored in the storage medium.

  In this case, the program code itself read out from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a non-volatile memory card, a ROM or the like can be used.

  The functions of the above-described embodiments can also be implemented by performing part or all of the actual processing such as an OS (a basic system or an operating system) running on a computer based on an instruction of a program code read by the computer. To be realized. Needless to say, this case is also included in the present invention.

  Furthermore, after the program code read out from the storage medium is written to the memory provided in the function expansion board inserted in the computer or in the function expansion unit connected to the computer, the processing based on the instruction of the program code is also the present invention. Included in the invention. That is, it goes without saying that the present invention is also included in the case where the CPU or the like provided in the function expansion board or the function expansion unit performs part or all of the actual processing based on the instructions of the program code to realize the functions of the embodiments described above. Yes.

  The above-described embodiments are merely examples of implementation for carrying out the present invention, and the technical scope of the present invention should not be interpreted in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

Claims (17)

An imaging unit having a pixel array in which a plurality of pixels each provided with a photoelectric conversion unit, and a plurality of microlenses shared by a plurality of adjacent pixels are two-dimensionally arranged;
The first pixel signal corresponding to the subject image formed by the photographing optical system having the focus lens is read out from the plurality of pixels sharing the microlens, and the photographing optical system is read from each of the plurality of pixels sharing the microlens Reading means for reading a second pixel signal corresponding to a subject image from different partial areas of the pupil area of
Output means having a first output system for outputting the first pixel signal and a second output system for outputting the second pixel signal;
Signal processing means for generating an evaluation value for performing focus adjustment of the focus lens using a pixel signal output from the output means;
The image pickup apparatus, wherein the reading unit reads the first pixel signal and the second pixel signal at different reading rates.   The imaging apparatus according to claim 1, wherein the output unit determines whether to output the second pixel signal according to a state of the imaging optical system.   The imaging optical system has an aperture, the state of the imaging optical system is the aperture value of the aperture, and the output unit is configured to output the second pixel signal when the aperture value is larger than a predetermined value. The image pickup apparatus according to claim 2, wherein the image pickup apparatus does not output   The signal processing means generates the evaluation value using the second pixel signal when the output means outputs the second pixel signal in accordance with the value of the aperture value, and the second processing The imaging device according to claim 3, wherein the evaluation value is generated using the first pixel signal when the pixel signal is not output.   When the signal processing means generates the evaluation value using the second pixel signal, a pixel signal obtained by performing an averaging process on the second pixel signal generated by a plurality of exposures The image pickup apparatus according to claim 4, wherein the evaluation value is generated using.   When the signal processing means generates the evaluation value using the second pixel signal, the signal processing means uses a pixel signal obtained by performing median processing on the second pixel signal generated by a plurality of exposures. The imaging device according to claim 4, wherein the evaluation value is generated.   When the signal processing unit generates the evaluation value using the second pixel signal, averaging is performed except for pixel signals that deviate from 1σ among the second pixel signals generated by a plurality of exposures. The imaging device according to claim 4, wherein the evaluation value is generated using a pixel signal obtained by performing processing.   The imaging optical system further includes an IS lens, and the signal processing unit performs the plurality of exposures by shifting the position of the IS lens when the evaluation value is generated using the second pixel signal. The imaging device according to any one of claims 5 to 7, characterized in that:   The image pickup apparatus according to any one of claims 5 to 8, further comprising focusing means for driving the focusing lens according to the evaluation value generated by the signal processing means.   The output means adds pixel signals read from a plurality of pixels sharing the micro lens and outputs the added signal as the first pixel signal, and a pixel of a predetermined pixel of the plurality of pixels sharing the micro lens The imaging device according to any one of claims 1 to 9, wherein a signal is output as the second pixel signal. And electronic zoom means for performing electronic zoom of a photographed image generated by the first pixel signal ,
Based on the ratio of expansion of the electronic zooming unit, the image pickup apparatus according to the captured image to claim 10, wherein determining whether to generate from the first pixel signal and the second pixel signal . And electronic zoom means for performing electronic zoom of a photographed image generated by the first pixel signal ,
On the basis of the magnification of expansion of the electronic zoom means, and determining the photographed image whether to generate from the first pixel signal and the second pixel signal,
When it is determined that the photographed image is to be generated from the first pixel signal and the second pixel signal, the signal processing unit is configured to read the second pixel read from a plurality of pixels corresponding to the microlens. 11. The image pickup apparatus according to claim 10, wherein signals are added, and the photographed image is generated from the added second pixel signal and the first pixel signal. An imaging unit having a pixel array in which a plurality of pixels each provided with a photoelectric conversion unit and a plurality of microlenses shared by a plurality of adjacent pixels are two-dimensionally arranged; a first output system and a second A control method of an image pickup apparatus comprising: output means having an output system of
The first pixel signal corresponding to the subject image formed by the photographing optical system having the focus lens is read out from the plurality of pixels sharing the microlens, and the photographing optical system is read from each of the plurality of pixels sharing the microlens Reading out a second pixel signal corresponding to a subject image from different partial areas of the pupil area of
Outputting the first pixel signal from the first output system and outputting the second pixel signal from the second output system;
Generating an evaluation value for performing focus adjustment of the focus lens using the pixel signal output in the output step;
The control method of an image pickup apparatus, wherein the reading means reads the first pixel signal and the second pixel signal at different reading rates. Program for controlling an imaging device including an imaging unit having a pixel array in which a plurality of pixels each including a photoelectric conversion unit and a plurality of microlenses shared by a plurality of adjacent pixels are two-dimensionally arranged And
Computer,
The first pixel signal corresponding to the subject image formed by the photographing optical system having the focus lens is read out from the plurality of pixels sharing the microlens, and the photographing optical system is read from each of the plurality of pixels sharing the microlens Readout means for reading out a second pixel signal corresponding to an object image from a different partial area of the pupil area of
An output unit having a first output system for outputting the first pixel signal and a second output system for outputting the second pixel signal;
It functions as a signal processing unit that generates an evaluation value for performing focus adjustment of the focus lens using a pixel signal output from the output unit.
The program that causes the reading unit to read the first pixel signal and the second pixel signal at different reading rates.   The computer-readable storage medium which recorded the program of Claim 14.   A program that causes a computer to function as each unit of the imaging device according to any one of claims 1 to 12.   A storage medium storing a program that causes a computer to function as each unit of the imaging device according to any one of claims 1 to 12.

Images