JP2009128579A - Focus detector and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To stably and accurately perform focus detection with respect to the movement of an image on a plane even in an imaging device which performs storage control in a line exposure system. <P>SOLUTION: The focus detector includes the imaging device, having a plurality of charge storage type pixels to receive luminous flux via an optical system; a storage control means for simultaneously performing charge storage control to a first pixel row, arranged in a first direction on the imaging device from among the pixels and successively perform charge storage control to a second pixel row, arranged in a second direction crossing with the first direction; and a focus detection means for perform focus detection of the optical system, based on output from the first pixel row and the second pixel row. The focus detection means performs focus detection by using output from the first pixel row (S130), and performs focus detection, by using the output from the second pixel row (S150), if a reliable result of focus detection cannot be obtained (S140). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a focus detection apparatus and an imaging apparatus.

  An imaging device that performs focus detection using an imaging device in which some imaging pixels are replaced with focus detection pixels in an imaging device in which imaging pixels are two-dimensionally arranged is known (see, for example, Patent Document 1). .

Prior art documents related to the invention of this application include the following.
JP 2000-292686 A

  However, in the above-described conventional apparatus, when an image sensor (CMOS image sensor) that performs pixel exposure control by a line exposure method (rolling shutter method) is used, exposure is performed between focus detection pixels belonging to a focus detection pixel column. Because the timing is different, when focus detection is performed on a moving subject, or when the focus detection device itself moves or vibrates, the focus detection accuracy deteriorates or the wrong focus detection result is output. There is.

(1) The invention of claim 1 is an image sensor having a plurality of charge storage type pixels that receive a light flux through an optical system, and is arranged along a first direction on the image sensor among the pixels. A charge control unit that simultaneously controls charge accumulation for the first pixel columns, and sequentially controls accumulation for the second pixel columns arranged along a second direction intersecting the first direction; A focus detection unit that performs focus detection of the optical system based on the output of the pixel column and the second pixel column, and the focus detection unit performs focus detection using the output of the first pixel column and When a characteristic focus detection result is not obtained, focus detection is performed using the output of the second pixel column.
(2) According to a second aspect of the present invention, in the focus detection apparatus according to the first aspect of the present invention, the focus detection unit further includes a motion detection unit that detects a motion of an image in the screen of the optical system, and the focus detection unit includes When the movement of the image is detected by the above, focus detection using the output of the second pixel row is prohibited.
(3) According to a third aspect of the present invention, in the focus detection apparatus according to the second aspect, the motion detection unit detects a shake applied to the focus detection apparatus.
(4) According to a fourth aspect of the present invention, in the focus detection apparatus according to the second aspect, the motion detecting unit detects an image motion by comparing outputs of the image sensor at different times.
(5) According to a fifth aspect of the present invention, in the focus detection apparatus according to any one of the first to fourth aspects, the imaging element has a plurality of imaging pixels arranged in a two-dimensional array for imaging an image by an optical system. The first pixel column and the second pixel column are provided in an array of the plurality of imaging pixels.
(6) According to a sixth aspect of the present invention, in the focus detection apparatus according to any one of the first to fifth aspects, the first pixel column and the second pixel column are arranged to intersect each other.
(7) According to a seventh aspect of the present invention, in the focus detection apparatus according to any one of the first to sixth aspects, the plurality of pixels receive a pair of light beams passing through different regions on the pupil of the optical system. Then, a signal for detecting the focus adjustment state of the optical system is output.
(8) According to an eighth aspect of the present invention, in the focus detection apparatus according to the seventh aspect, the pixel includes a microlens and a pair of photoelectric conversion units arranged with respect to the microlens.
(9) The invention according to claim 9 is the focus detection apparatus according to claim 7, wherein the pixel includes a microlens and a first photoelectric conversion unit disposed with respect to the microlens. A second focus detection pixel including a pixel, a microlens, and a second photoelectric conversion unit disposed with respect to the microlens.
(10) The invention of claim 10 is an imaging apparatus including the focus detection apparatus according to any one of claims 1 to 9.

  According to the present invention, even in an image sensor that performs accumulation control using a line exposure method (rolling shutter method) such as a CMOS image sensor, focus detection can be performed stably and accurately with respect to the movement of an image on the screen. Can do.

  A lens interchangeable digital still camera will be described as an example as an imaging device and an imaging apparatus according to an embodiment. FIG. 1 is a cross-sectional view of a camera showing the configuration of the camera of one embodiment. A digital still camera 201 according to an embodiment includes an interchangeable lens 202 and a camera body 203, and the interchangeable lens 202 is attached to the camera body 203 via a mount unit 204. An interchangeable lens 202 having various photographing optical systems can be attached to the camera body 203 via a mount unit 204.

  The interchangeable lens 202 includes a lens 209, a zooming lens 208, a focusing lens 210, an aperture 211, a lens drive control device 206, and the like. The lens drive control device 206 includes a microcomputer (not shown), a memory, a drive control circuit, and the like. The lens drive control device 206 includes drive control for adjusting the focus of the focusing lens 210 and the aperture diameter of the aperture 211, zooming lens 208, and focusing. In addition to detecting the state of the lens 210 and the aperture 211, the lens information is transmitted and the camera information is received through communication with a body drive control device 214 described later. The aperture 211 forms an aperture having a variable aperture diameter at the center of the optical axis in order to adjust the amount of light and the amount of blur.

  The camera body 203 includes an imaging element 212, a body drive control device 214, a liquid crystal display element drive circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, and the like. In the imaging element 212, imaging pixels are two-dimensionally arranged, and focus detection pixels are incorporated in portions corresponding to focus detection positions. Details of the image sensor 212 will be described later.

  The body drive control device 214 includes a microcomputer, a memory, a drive control circuit, and the like, and controls the drive of the image sensor 212, reads out the image signal and the focus detection signal, performs the focus detection calculation based on the focus detection signal, and the focus of the interchangeable lens 202. The adjustment is repeated, and image signal processing and recording, camera operation control, and the like are performed. The body drive control device 214 communicates with the lens drive control device 206 via the electrical contact 213 to receive lens information and send camera information (defocus amount, aperture value, etc.).

  The liquid crystal display element 216 functions as an electric view finder (EVF). The liquid crystal display element driving circuit 215 displays a through image by the imaging element 212 on the liquid crystal display element 216, and the photographer can observe the through image through the eyepiece lens 217. The memory card 219 is an image storage that stores an image captured by the image sensor 212.

  A subject image is formed on the light receiving surface of the image sensor 212 by the light beam that has passed through the interchangeable lens 202. This subject image is photoelectrically converted by the image sensor 212, and an image signal and a focus detection signal are sent to the body drive control device 214.

  The body drive control device 214 calculates the defocus amount based on the focus detection signal from the focus detection pixel of the image sensor 212 and sends the defocus amount to the lens drive control device 206. In addition, the body drive control device 214 processes the image signal from the image sensor 212 and stores it in the memory card 219, and sends the through image signal from the image sensor 212 to the liquid crystal display element drive circuit 215, and transmits the through image to the liquid crystal. The image is displayed on the display element 216. Further, the body drive control device 214 sends aperture control information to the lens drive control device 206 to control the aperture of the aperture 211.

  The lens drive controller 206 updates the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the positions of the zooming lens 208 and the focusing lens 210 and the aperture value of the aperture 211 are detected, and lens information is calculated according to these lens positions and aperture values, or a lookup prepared in advance. Lens information corresponding to the lens position and aperture value is selected from the table.

  The lens drive control device 206 calculates a lens drive amount based on the received defocus amount, and drives the focusing lens 210 to the in-focus position according to the lens drive amount. Further, the lens drive control device 206 drives the diaphragm 211 in accordance with the received diaphragm value.

  FIG. 2 is a diagram showing a focus detection position on the photographing screen, and shows an example of a region (focus detection area, focus detection position) on which an image is sampled on the screen when focus detection is performed using a focus detection pixel array to be described later. . In this embodiment, focus detection areas 101 to 105 are arranged at five locations in the center, left and right, and top and bottom in a rectangular shooting screen 100. A plurality of focus detection pixels are linearly arranged in the longitudinal direction of the focus detection areas 101 to 105 indicated by rectangles.

  3 is a front view showing a detailed configuration of the image sensor 212. FIG. 3A is an enlarged view of the vicinity of the focus detection areas 101, 104, and 105 on the image sensor 212 shown in FIG. 2, and FIG. The vicinity of the focus detection areas 102 and 103 on the image sensor 212 shown in FIG. First, in the vicinity of the focus detection areas 101, 104, and 105 in the horizontal direction (left and right direction) of the shooting screen 100, as shown in FIG. 3A, an imaging pixel 310 for imaging and a focus detection pixel for focus detection. 313 and 314 are arranged. The imaging pixels 310 are arranged in a two-dimensional square lattice in the horizontal and vertical directions, and the focus detection pixels 313 and 314 are arranged in the horizontal direction.

  As shown in FIG. 4, the imaging pixel 310 includes a microlens 10, a photoelectric conversion unit 11, and a color filter (not shown). There are three types of color filters, red (R), green (G), and blue (B), and each spectral sensitivity exhibits the characteristics shown in FIG. In this embodiment, an example of an imaging element 212 in which imaging pixels 310 having respective color filters are arranged in a Bayer array is shown.

  On the other hand, the focus detection pixel 313 includes the microlens 10 and the photoelectric conversion unit 16 as illustrated in FIG. The shape of the photoelectric conversion unit 16 is a left semicircle in contact with the vertical bisector of the microlens 10. Further, the focus detection pixel 314 includes the microlens 10 and the photoelectric conversion unit 17 as illustrated in FIG. The shape of the photoelectric conversion unit 17 is a right semicircle in contact with the vertical bisector of the microlens 10. The photoelectric conversion units 16 and 17 are arranged in the horizontal direction when the microlenses 10 are displayed in a superimposed manner, and have a symmetrical shape with respect to the vertical bisector of the microlens 10. In the focus detection areas 101, 104, and 105, the focus detection pixels 313 and the focus detection pixels 314 are alternately arranged in the horizontal direction (the arrangement direction of the photoelectric conversion units 16 and 17; the left and right direction of the photographing screen 100 shown in FIG. 2). .

  The focus detection pixels 313 and 314 are not provided with a color filter in order to increase the amount of light, and the spectral characteristics of the focus detection pixels 313 and 314 are the total of the spectral sensitivity of a photodiode that performs photoelectric conversion and the spectral characteristics of an infrared cut filter (not shown). The spectral characteristics are as shown in FIG. That is, the spectral characteristics are obtained by adding the spectral characteristics of the green pixel, the red pixel, and the blue pixel shown in FIG. 6, and the light wavelength region of the sensitivity includes the light wavelength regions of the sensitivity of the green pixel, the red pixel, and the blue pixel. ing. The focus detection pixels 313 and 314 are arranged in a row where the blue (B) filter and the green (G) filter of the imaging pixel 310 are to be arranged.

  Next, in the vicinity of the focus detection areas 102 and 103 in the vertical direction (up and down direction) of the shooting screen 100, as shown in FIG. 3B, the imaging pixel 310 for imaging and the focus detection pixel 315 for focus detection, 316 is arranged. The focus detection pixels 315 and 316 have a structure obtained by rotating the focus detection pixels 313 and 314 by 90 degrees, and are arranged in the vertical direction on the imaging surface. The focus detection pixels 315 and 316 are arranged in a column where the B and G filters of the imaging pixel 310 should be arranged.

  FIG. 8 is a cross-sectional view of the imaging pixel 310. In the imaging pixel 310, the microlens 10 is disposed in front of the photoelectric conversion unit 11 for imaging, and the photoelectric conversion unit 11 is projected forward by the microlens 10. The photoelectric conversion unit 11 is formed on the semiconductor circuit substrate 29. A color filter (not shown) is arranged between the microlens 10 and the photoelectric conversion unit 11.

  FIG. 9A is a cross-sectional view of the focus detection pixel 313. In the focus detection pixel 313, the microlens 10 is disposed in front of the photoelectric conversion unit 16, and the photoelectric conversion unit 16 is projected forward by the microlens 10. The photoelectric conversion unit 16 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon through a semiconductor image sensor manufacturing process. The photoelectric conversion unit 16 is disposed on one side of the optical axis of the microlens 10.

  FIG. 9B is a cross-sectional view of the focus detection pixel 314. In the focus detection pixel 314, the microlens 10 is disposed in front of the photoelectric conversion unit 17, and the photoelectric conversion unit 17 is projected forward by the microlens 10. The photoelectric conversion unit 17 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon through a semiconductor image sensor manufacturing process. The photoelectric conversion unit 17 is disposed on one side of the optical axis of the microlens 10 and on the opposite side of the photoelectric conversion unit 16.

  FIG. 10 shows a configuration of a pupil division type phase difference detection type focus detection optical system using a microlens. In the figure, reference numeral 90 denotes an exit pupil set at a distance d forward from the microlens arranged on the planned imaging plane of the interchangeable lens 202 (see FIG. 1). This distance d is a distance determined according to the curvature and refractive index of the microlens, the distance between the microlens and the photoelectric conversion unit, and is referred to as a distance measuring pupil distance in this specification. 91 is an optical axis of the interchangeable lens, 10a to 10d are microlenses, 13a, 13b, 14a, and 14b are photoelectric conversion units, 313a, 313b, 314a, and 314b are focus detection pixels, and 73, 74, 83, and 84 are focus detection light fluxes. It is.

  Reference numeral 93 denotes an area of the photoelectric conversion units 13a and 13b projected by the microlenses 10a and 10c, and is referred to as a distance measuring pupil in this specification. In FIG. 10, an elliptical region is shown for easy understanding, but the shape of the photoelectric conversion unit is actually an enlarged projection shape. Similarly, 94 is an area of the photoelectric conversion units 14a and 14b projected by the microlenses 10b and 10d, and is called a distance measuring pupil in this specification. In FIG. 10, an elliptical region is shown for easy understanding, but the shape of the photoelectric conversion unit is actually an enlarged projection shape.

  In FIG. 10, four adjacent focus detection pixels 313a, 313b, 314a, and 314b are schematically illustrated. However, in other focus detection pixels as well, the photoelectric conversion units are applied to the respective microlenses from the corresponding distance measurement pupils. Receives incoming light flux. The arrangement direction of the focus detection pixels is made to coincide with the arrangement direction of the pair of distance measuring pupils, that is, the arrangement direction of the pair of photoelectric conversion units.

  The microlenses 10a to 10d are disposed in the vicinity of the planned imaging plane of the interchangeable lens 202 (see FIG. 1), and the shapes of the photoelectric conversion units 13a, 13b, 14a, and 14b disposed behind the microlenses 10a to 10d. Is projected onto the exit pupil 90 separated from the microlenses 10a to 10c by the distance measurement pupil distance d, and the projection shape forms distance measurement pupils 93 and 94. That is, the projection direction of the photoelectric conversion unit in each focus detection pixel is determined so that the projection shape (ranging pupils 93 and 94) of the photoelectric conversion unit of each focus detection pixel matches on the exit pupil 90 at the projection distance d. Has been.

  The photoelectric conversion unit 13a passes through the distance measuring pupil 93 and outputs a signal corresponding to the intensity of the image formed on the microlens 10a by the light beam 73 directed to the microlens 10a. Similarly, the photoelectric conversion unit 13b outputs a signal corresponding to the intensity of the image formed on the microlens 10c by the light beam 83 passing through the distance measuring pupil 93 and directed to the microlens 10c. Further, the photoelectric conversion unit 14a outputs a signal corresponding to the intensity of the image formed on the microlens 10b by the light beam 74 passing through the distance measuring pupil 94 and directed to the microlens 10b. Similarly, the photoelectric conversion unit 14b outputs a signal corresponding to the intensity of the image formed on the microlens 10d by the light beam 84 passing through the distance measuring pupil 94 and directed to the microlens 10d.

  A large number of the two types of focus detection pixels described above are arranged in a straight line, and the output of the photoelectric conversion unit of each pixel is grouped into a distance measurement pupil 93 and an output group corresponding to the distance measurement pupil 94, thereby measuring the distance measurement pupil 93 and the measurement pupil 93. Information on the intensity distribution of the pair of images formed on the pixel array by the focus detection light beams that respectively pass through the distance pupil 94 is obtained. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process), which will be described later, to this information, an image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method. Further, by performing a conversion operation according to the center-of-gravity interval of the pair of distance measuring pupils on the image shift amount, the current imaging plane with respect to the planned imaging plane (the focal point corresponding to the position of the microlens array on the planned imaging plane) The deviation (defocus amount) of the imaging plane at the detection position is calculated.

  FIG. 11 is a conceptual diagram showing a circuit configuration of the image sensor 212. The image sensor 212 is configured as a CMOS image sensor, and accumulation control is controlled by a line exposure method (rolling shutter method). For easy understanding, the circuit configuration of the image sensor 212 is illustrated as a 4 × 4 pixel layout. Here, two focus detection pixels 313 and 314 are arranged in the second and third columns of the third row, and the other is the imaging pixel 310. That is, the focus detection pixels 313 and 314 are arranged in the horizontal direction corresponding to the focus detection areas 101, 104, and 105 arranged in the horizontal direction of the screen. In the case of the focus detection pixels 315 and 315 corresponding to the focus detection areas 102 and 103 arranged in the vertical direction of the screen, these focus detection pixels 315 and 315 are arranged on the same column. For example, it can be considered that the two focus detection pixels 315 and 315 are arranged in the second and third rows of the third column.

  As described above, since the accumulation control is performed for each row in the line exposure method, the simultaneity of exposure timing is guaranteed for the imaging pixels and focus detection pixels arranged in the same row, but in different rows. There is a deviation in the exposure timing between the imaging pixels and the focus detection pixels that are arranged in a straddling manner.

  The signal holding unit 502 is a buffer that temporarily holds an image signal of pixels for one row, and is based on a control signal ΦH that the vertical scanning circuit 503 generates an image signal of the same row that is output to the vertical signal line 501. And latch. Charge accumulation in the imaging pixel 310 and the focus detection pixels 313 and 314 is controlled for each row by a control signal (ΦR1 to ΦR4) generated by the accumulation control circuit 504. Output of image signals from the imaging pixel 310 and the focus detection pixels 313 and 314 is controlled independently for each row by a control signal (ΦS1 to ΦS4) generated by the vertical scanning circuit 503. The image signal of the pixel selected by the control signal is output to the vertical signal line 501. The image signals held in the signal holding unit 502 are sequentially transferred to the output circuit 506 by the control signals (ΦV1 to ΦV4) generated by the horizontal transfer circuit 505, amplified by the output circuit 506 with a preset amplification degree, and then transmitted to the outside. Is output.

  FIG. 12 is a detailed circuit diagram of the imaging pixel 310 and the focus detection pixels 313 and 314 shown in FIG. The photoelectric conversion unit is composed of a photodiode (PD). The charge accumulated in the PD is accumulated in a floating diffusion layer (floating diffusion: FD). The FD is connected to the gate of the amplification MOS transistor (AMP), and the AMP generates a signal corresponding to the amount of charge accumulated in the FD. The FD is connected to the power supply voltage Vdd via the reset MOS transistor 510. When the reset MOS transistor 510 is turned on by the control signal ΦRn (ΦR1 to ΦR4), the charges accumulated in the FD and PD are cleared and reset. Become. The output of the AMP is connected to the vertical output line 501 via the row selection MOS transistor 512. When the row selection MOS transistor 512 is turned on by the control signal ΦSn (ΦS1 to ΦS4), the output of the AMP is output to the vertical output line 501. Is output.

  FIG. 13 is a timing chart showing the operation of the image sensor 212 shown in FIG. The imaging pixel 310 in the first row is selected by the control signal ΦS1 generated by the vertical scanning circuit 503, and the image signal of the selected imaging pixel 310 is output to the vertical signal line 501. The image signal of the first row output to the vertical signal line 501 is temporarily held in the signal holding unit 502 by the control signal ΦH issued in synchronization with the control signal ΦS1. The image signals of the imaging pixels 310 in the first row held in the signal holding unit 502 are transferred to the output circuit 506 according to the control signals ΦV1 to ΦV4 sequentially issued from the horizontal scanning circuit 505, and the amplification set by the output circuit 506 Amplified at a degree and output to the outside.

  When the transfer of the image signals of the imaging pixels 310 in the first row to the signal holding unit 502 is completed, the imaging pixels 310 in the first row are reset by the control signal ΦR1 issued from the accumulation control circuit 504, and the control signal ΦR1 At the falling edge, the next charge accumulation of the imaging pixels 310 in the first row is started. At the time when the output of the image signal of the imaging pixel 310 of the first row from the output circuit 506 is completed, the imaging pixel 310 of the second row is selected by the control signal ΦS2 generated by the vertical scanning circuit 503, and the selected imaging pixel 310 is selected. Are output to the vertical signal line 501. Thereafter, similarly, holding of the image signal of the imaging pixel 310 in the second row, resetting of the imaging pixel 310, output of the image signal, and start of the next charge accumulation are performed.

  Subsequently, the image signals of the imaging pixels 310 and the focus detection pixels 313 and 314 in the third row are held, the pixels are reset, the image signals are output, and the next charge accumulation is started. Subsequently, the image signal of the imaging pixel 310 in the fourth row is held, the imaging pixel 310 is reset, the image signal of the imaging pixel 310 is output, and the next charge accumulation is started. Thereafter, the process returns to the first line again, and the above-described operation is repeated.

  The period Ts from the charge holding timing of the image signal of the imaging pixel 310 of the first row to the charge holding timing of the image signal of the imaging pixel 310 of the next first row is controlled to be constant. The charge accumulation time Ti (exposure time) of the imaging pixel 310 and the focus detection pixels 313 and 314 is determined from the pixel reset timing (falling edge of the control signal ΦRn) to the image signal holding timing (control signal ΦH when the control signal ΦSn is ON). Until the rise of Further, the charge accumulation time Ti (exposure time) of the imaging pixel 310 and the focus detection pixels 313 and 314 can be adjusted by changing the pulse widths of the control signals ΦR1 to ΦR4.

  FIG. 14 is an explanatory diagram of the exposure timing of the image sensor 212 according to the embodiment. In the image sensor 212, the charge accumulation timing of the pixel is controlled by the above-described line exposure method. In general, in an N-row image sensor, as shown in FIG. 14, the charge accumulation start and end of the pixels in the first row are at times t1s and t2e, and the charge accumulation start and end of the pixels in the N-th row are at time tNs, tNe. The charge accumulation start and end timings of the pixels in the middle row are intermediate timings between the charge accumulation timings of the first row pixels and the Nth row pixels, and the charge accumulation timing shifts for each row. Produce. That is, the charge accumulation starts and ends simultaneously between the imaging pixel and the focus detection pixel arranged in the g-th row, but the imaging pixel and the focus detection pixel arranged in the g-th row and the (g + 1) -th row. The start and end of charge accumulation differ between the arranged imaging pixels and focus detection pixels by (tNs−t1s) / (N−1), and the simultaneity of charge accumulation is lost.

  FIG. 15 is a flowchart showing the operation of the digital still camera (imaging device) shown in FIG. The body drive control device 214 starts this operation when the camera is turned on in step 100. In step 110, accumulation control of the imaging pixel and the focus detection pixel is performed by a line exposure method, and data is read from the imaging pixel and the focus detection pixel. In the following step 120, the image pickup pixel data is displayed on the electronic viewfinder. In step 130, based on a pair of image data read from the focus detection areas 101, 104, and 105 in which the focus detection pixels 313 and 314 are arranged in the horizontal direction on the screen, an image shift detection calculation process (correlation calculation process) described later is performed. ) To calculate an image shift amount for each focus detection area, and further convert the image shift amount into a defocus amount.

  In step 140, it is checked whether or not focus detection is impossible in all the focus detection areas 101, 104, and 105 in the horizontal direction of the screen (a reliable defocus amount has not been obtained), and at least one focus is detected. If focus detection is possible in the detection area, the process proceeds to step 160. If focus detection is not possible in all areas, the process proceeds to step 150.

  If focus detection is impossible in all the focus detection areas 101, 104, and 105 in the horizontal direction, in step 150, the focus detection pixels 315 and 316 are read out from the focus detection areas 102 and 103 arranged in the vertical direction. Based on the pair of image data, an image shift detection calculation process (correlation calculation process) described later is performed, an image shift amount is calculated for each focus detection area, and the image shift amount is converted into a defocus amount.

  In step 160, when the defocus amount is obtained in any one of the plurality of focus detection areas 101, 104, and 105 in the horizontal direction of the screen, the selection process (average of the plurality of defocus amounts or the closest distance is selected). The final defocus amount is calculated by selecting a defocus amount to be displayed). When the defocus amount can be obtained only in one focus detection area among the focus detection areas 101, 104, and 105 in the horizontal direction of the screen, the final focus for adjusting the focus of the interchangeable lens 202 is determined based on the defocus amount. Defocus amount.

  On the other hand, when the focus detection is impossible in all the focus detection areas 101, 104, and 105 in the horizontal direction of the screen and the defocus amount is obtained in both the focus detection areas 102 and 103 in the vertical direction of the screen, any of the above-described selection processing is performed. When the defocus amount is obtained only in one of the focus detection areas 102 and 103, the defocus amount is set as the final defocus amount for adjusting the focus of the interchangeable lens 202. When focus detection is impossible in all focus detection areas in the horizontal and vertical directions of the screen, focus detection is disabled.

  In step 170, it is checked whether or not the focus is close, that is, whether or not the calculated absolute value of the final defocus amount is within a predetermined value. If it is determined that the focus is not close, the process proceeds to step 180, the defocus amount is transmitted to the lens drive control device 206, the focusing lens 210 of the interchangeable lens 202 is driven to the focus position, and then the process returns to step 110. The above operation is repeated. Even when the focus cannot be detected, the process branches to step 180, a scan drive command is transmitted to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is driven to scan from infinity to the closest position. Returning to 110, the above-described operation is repeated.

  On the other hand, if it is determined that the focus is close, the process proceeds to step 190 to determine whether or not a shutter release has been performed by operating a release button (not shown). If it is determined that the shutter release has not been performed, step 110 is performed. Return to and repeat the above operation. If it is determined that the shutter release has been performed, the process proceeds to step 200, where an aperture adjustment command is transmitted to the lens drive control device 206, and the aperture value of the interchangeable lens 202 is controlled to a control F value (F set by the photographer or automatically). Value). When the aperture control is completed, the image sensor 212 is caused to perform an imaging operation, and image data is read from the imaging pixel 310 of the imaging element 212 and all the focus detection pixels 313, 314, 315, and 316.

  In step 210, pixel interpolation is performed on the pixel data at each pixel position in the focus detection pixel row based on the data of the imaging pixels around the focus detection pixel. In the next step 220, the image data composed of the imaged pixel data and the interpolated data is stored in the memory card 219, and the process returns to step 110 to repeat the above-described operation.

  By executing the above operation, focus detection is preferentially performed in the focus detection areas 101, 104, and 105 in which simultaneity of charge accumulation timing is guaranteed, and focus detection is impossible in the focus detection areas 101, 104, and 105. Only in the case, since the focus detection is controlled in the focus detection areas 102 and 103 where the simultaneity of the charge accumulation timing is not guaranteed, the charge accumulation timing is obtained even when the relative displacement between the subject and the camera occurs. It is possible to reduce the probability of a focus detection malfunction due to the non-simultaneity of the two.

Here, details of the image shift detection calculation process (correlation calculation process) in steps 130 and 150 of FIG. 15 will be described. The pair of images detected by the focus detection pixels has a possibility that the distance measurement pupil is displaced by the aperture of the lens and the balance of the light quantity is lost. Perform the operation. The correlation calculation shown in the following formula (1) is performed on a pair of data strings (A11 to A1M, A21 to A2M: M is the number of data) read from the focus detection pixel string, and the correlation amount C (k) is calculated. To do.
C (k) = Σ | A1n · A2n + 1 + k−A2n + k · A1n + 1 | (1)
In equation (1), the Σ operation is accumulated for n, and the range taken by n is limited to the range in which the data of A1n, A1n + 1, A2n + k, and A2n + 1 + k exist according to the image shift amount k. The The image shift amount k is an integer and is a relative shift amount with the data interval of the data string as a unit.

As shown in FIG. 16A, the calculation result of the expression (1) shows that the correlation amount C (k) is minimal at a shift amount with high correlation between a pair of data (k = kj = 2 in FIG. 16A). (The smaller the value, the higher the degree of correlation). Next, the shift amount x that gives the minimum value C (x) with respect to the continuous correlation amount is obtained by using the three-point interpolation method shown in the following equations (2) to (5).
x = kj + D / SLOP (2),
C (x) = C (kj) − | D | (3),
D = {C (kj-1) -C (kj + 1)} / 2 (4),
SLOP = MAX {C (kj + 1) -C (kj), C (kj-1) -C (kj)} (5)

  Whether or not the shift amount x calculated by the equation (2) is reliable is determined as follows. As shown in FIG. 16B, when the degree of correlation between the pair of data is low, the value of the minimal value C (x) of the interpolated correlation amount increases. Therefore, when C (x) is equal to or greater than a predetermined threshold value, it is determined that the calculated shift amount has low reliability, and the calculated shift amount x is canceled. Alternatively, in order to normalize C (x) with the contrast of data, when the value obtained by dividing C (x) by SLOP that is proportional to the contrast is equal to or greater than a predetermined value, the reliability of the calculated shift amount Is determined to be low, and the calculated shift amount x is canceled. Alternatively, when SLOP that is a value proportional to the contrast is equal to or less than a predetermined value, it is determined that the subject has low contrast and the reliability of the calculated shift amount is low, and the calculated shift amount x is canceled. As shown in FIG. 16C, when the correlation between the pair of data is low and there is no drop in the correlation amount C (k) between the shift ranges kmin to kmax, the minimum value C (x) is obtained. In such a case, it is determined that the focus cannot be detected.

  The correlation calculation formula is not limited to the above formula (1), and any correlation calculation formula can maintain the image shift detection accuracy even if the distance measurement pupil is displaced by the aperture of the lens and the light quantity balance is lost. Any expression is acceptable.

If it is determined that the calculated shift amount x is reliable, it is converted into the image shift amount shft by the equation (6).
shft = PY · x (6)
In the equation (6), PY is a detection pitch (a pitch of focus detection pixels). Next, the image shift amount calculated by the equation (6) is multiplied by a predetermined conversion coefficient k to be converted into a defocus amount def.
def = k · shft (7)
Note that the conversion coefficient k depends on the opening angle θ that anticipates the center of gravity of the pair of distance measuring pupils 93 and 94 shown in FIG. 10, and has a relationship of approximately k = 1 / (2 · Tan (θ / 2)). .

<< Other Embodiments of the Invention >>
FIG. 17 is a flowchart showing the operation of a modification of the digital still camera. In the operation of the embodiment described above (see FIG. 15), an example is shown in which focus detection is always performed in the focus detection areas 101, 104, and 105 in the horizontal direction in preference to the focus detection areas 102 and 103 in the vertical direction. However, the focus detection may be performed with priority given to the focus detection areas 101, 104, and 105 in the horizontal direction over the focus detection areas 102 and 103 in the vertical direction depending on the situation. In the operation shown in FIG. 17, steps that perform the same operation as the operation shown in FIG.

  In the operation of this modification, steps 121 and 122 are added between steps 120 and 130. In step 121, the movement of the image in the screen is detected. If there is a movement of the image, the process proceeds to step 130. If there is no movement of the image, the process proceeds to step 122.

  For example, a known motion detection sensor may be disposed in the camera body and the motion of the image may be detected according to the output of the motion detection sensor, or an image memory may be disposed in the camera body. Then, a known motion vector may be calculated between past image data and current image data, and the motion of the image may be detected according to the magnitude of the motion vector.

  In step 122, image shift detection calculation processing (correlation calculation processing) is performed based on a pair of image data read from all the focus detection areas 101 to 105, and an image shift amount is calculated for each focus detection area. Further, the image shift amount is converted into a defocus amount, and the process proceeds to step 160.

  According to the operation of this modification, focus detection is preferentially performed in the focus detection areas 101, 104, and 105 in which the simultaneity of charge accumulation timing is ensured only when there is an image movement, and the image movement is reduced. If not, focus detection is performed in all the focus detection areas 101 to 105, so that more focus detection areas can be used effectively.

  FIG. 18 is a flowchart showing the operation of another modification of the digital still camera. In the operation shown in FIG. 17, focus detection is performed preferentially with respect to the focus detection areas 102 and 103 in the vertical direction in the focus detection areas 101, 104, and 105 in the horizontal direction only when there is an image movement in the screen. However, the focus detection in the focus detection areas 102 and 103 in the vertical direction may be prohibited when there is an image movement. In the operation illustrated in FIG. 18, the operations in steps 140 and 150 in the operation illustrated in FIG. 17 are deleted, and the process proceeds from step 130 to step 160.

  According to the operation of this other modified example, when there is a movement of the image (when the relative displacement between the subject and the camera occurs), the focus detection malfunction caused by the non-simultaneous charge accumulation timing is detected. It can be surely prevented.

  The arrangement of the focus detection areas in the image sensor is not limited to the arrangement shown in FIG. 2, and the focus detection areas can be arranged in the horizontal direction and the vertical direction in the diagonal direction and other positions.

  Further, by arranging the focus detection areas in the horizontal direction and the focus detection areas in the vertical direction close to each other, it is possible to prevent the focus detection points from being substantially reduced even when the focus detection area in the horizontal direction is given priority. For example, by placing a cross-shaped focus detection area by crossing the horizontal focus detection area and the vertical focus detection area, focus detection in the vertical focus detection area is prohibited by the movement of the image on the screen. Even in this case, focus detection can be performed in the horizontal focus detection area that intersects the cross, and focus detection can be performed at the same position in the screen.

  When forming a cross-shaped focus detection area in the horizontal direction and the vertical direction, for example, in FIG. 3A, in the vicinity of the center of the pixel row of the focus detection pixels 313 and 314 in the horizontal direction, FIG. The vertical focus detection pixels 315 and 316 shown in FIG. The arrangement of the vertical focus detection pixels 315 and 316 is preferably arranged in a row of green and blue imaging pixels. It should be noted that although the vertical focus detection pixel rows cannot be continuously arranged at the intersections, the drop in focus detection accuracy due to this is negligible. It is desirable that the cross-shaped focus detection areas are arranged at many positions in the center and the periphery in the shooting screen.

  In the image sensor shown in FIG. 3, the focus detection pixels 313, 314, 315, and 316 have been described as having one photoelectric conversion unit in one pixel, but a pair of photoelectric conversion units is provided in one pixel. May be. For example, as shown in FIG. 19, an imaging element 212A is configured by arranging a plurality of focus detection pixels 311 having a pair of photoelectric conversion units in one pixel in a two-dimensional array of imaging pixels 310. Also good. In this image sensor 212A, the focus detection pixel 311 performs a function corresponding to the pair of the focus detection pixel 313 and the focus detection pixel 314 shown in FIG.

  In this image sensor 212A, as shown in FIG. 20, the focus detection pixel 311 is composed of a microlens 10 and a pair of photoelectric conversion units 12 and 13. The focus detection pixel 311 is not provided with a color filter in order to increase the amount of light, and its spectral characteristic is a spectral characteristic that combines the spectral sensitivity of a photodiode that performs photoelectric conversion and the spectral characteristic of an infrared cut filter (not shown). (See FIG. 7). That is, the spectral characteristics are obtained by adding the spectral characteristics of the green pixel, the red pixel, and the blue pixel shown in FIG. 6, and the light wavelength region of the sensitivity includes the light wavelength regions of the sensitivity of the green pixel, the red pixel, and the blue pixel. ing.

  In the image pickup device 212 shown in FIG. 3, an example in which focus detection pixels are arranged in a part of the arrangement of the image pickup pixels is shown. However, in FIG. 1, a half mirror is arranged in the shooting optical path of the camera body 203 to This is also applied to a camera having a configuration in which an image sensor dedicated for imaging is arranged in one optical path and an image sensor dedicated for focus detection in which only focus detection pixels are arranged in a two-dimensional manner is arranged in the other optical path. The invention can be applied.

  In the above-described embodiment, an example using the pupil division phase difference detection type focus detection pixel is shown, but the present invention can also be applied to a contrast detection method using an imaging pixel. That is, it is possible to configure a focus detection device that prioritizes contrast detection based on image data of imaging pixels arranged in the horizontal direction over contrast detection based on image data of imaging pixels arranged in the vertical direction.

  In the image pickup device shown in FIG. 3, the example in which the image pickup pixel includes a Bayer color filter is shown. However, the configuration and arrangement of the color filter are not limited to this, and complementary color filters (green: G, yellow: An arrangement of Ye, magenta: Mg, cyan: Cy) may be employed.

  In the image sensor shown in FIG. 3, an example in which a color filter is not provided in the focus detection pixel is shown. However, when one color filter (for example, a green filter) of the same color filters as the image pickup pixel is provided. However, the present invention can be applied.

  In FIGS. 5 and 20, an example in which the shape of the photoelectric conversion unit of the focus detection pixel is a semicircular shape is shown, but the shape of the photoelectric conversion unit is not limited to this, and may be another shape. For example, the shape of the photoelectric conversion unit of the focus detection pixel may be an ellipse, a rectangle, or a polygon.

  In the imaging device shown in FIG. 3, the example in which the imaging pixels and the focus detection pixels are arranged in a dense square lattice arrangement is shown, but a dense hexagonal lattice arrangement may be used.

  The imaging apparatus according to the present invention is not limited to a digital still camera or a film still camera including an interchangeable lens and a camera body, and can be applied to a lens-integrated digital still camera, a film still camera, and a video camera. Further, the present invention can be applied to a small camera module built in a mobile phone or the like, a surveillance camera, a visual recognition device for a robot, or the like. Alternatively, the present invention can be applied to a focus detection device other than a camera, a distance measuring device, or a stereo distance measuring device.

Cross-sectional view of the camera showing the configuration of the camera of one embodiment Diagram showing the focus detection position on the shooting screen The front view which shows the detailed structure of the image pick-up element of one embodiment The figure which shows the structure of an imaging pixel The figure which shows the structure of a focus detection pixel Diagram showing spectral characteristics of imaging pixels Diagram showing spectral characteristics of focus detection pixels Cross section of imaging pixel Cross section of focus detection pixel The figure which shows the structure of the focus detection optical system of the pupil division type phase difference detection method using a micro lens The conceptual diagram which shows the circuit structure of the image pick-up element of one embodiment Detailed circuit diagram of the imaging pixel and focus detection pixel shown in FIG. Timing chart showing the operation of the image sensor shown in FIG. Explanatory drawing of the exposure timing of the image sensor of one embodiment 1 is a flowchart showing the operation of the digital still camera (imaging device) shown in FIG. Diagram explaining the reliability of focus detection results The flowchart which shows operation | movement of the modification of the digital still camera (imaging device) shown in FIG. The flowchart which shows operation | movement of the other modification of the digital still camera (imaging device) shown in FIG. Front view of image sensor of modification The figure which shows the structure of the focus detection pixel used with the image pick-up element shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10; Micro lens, 11, 12, 13, 16, 17; Photoelectric conversion part, 201; Camera (imaging apparatus), 212, 212A; Imaging element, 214; Body drive control apparatus, 310; Imaging pixel, 313, 314, 315, 316; focus detection pixel, 503; vertical scanning circuit, 504; accumulation control circuit, 505; horizontal scanning circuit

Claims (10)

An imaging device having a plurality of charge storage type pixels that receive a light beam via an optical system;
Among the pixels, the charge accumulation control is simultaneously performed on the first pixel array arranged along the first direction on the image sensor, and the first pixel array arranged along the second direction intersecting the first direction is used. Accumulation control means for sequentially accumulating two pixel columns;
A focus detection unit that performs focus detection of the optical system based on outputs of the first pixel column and the second pixel column;
The focus detection unit performs focus detection using the output of the first pixel column, and performs focus detection using the output of the second pixel column when a reliable focus detection result is not obtained. A focus detection apparatus. The focus detection apparatus according to claim 1,
Motion detection means for detecting the motion of the image in the screen of the optical system,
The focus detection device, wherein the focus detection unit prohibits focus detection using the output of the second pixel row when the motion detection unit detects an image motion. The focus detection apparatus according to claim 2,
The motion detection means detects a shake applied to the focus detection device. The focus detection apparatus according to claim 2,
The focus detection apparatus characterized in that the motion detection means detects the motion of an image by comparing the outputs of the imaging elements at different times. In the focus detection apparatus according to any one of claims 1 to 4,
In the imaging device, a plurality of imaging pixels for imaging an image by an optical system are arranged in a two-dimensional shape,
The focus detection apparatus, wherein the first pixel column and the second pixel column are provided in an array of the plurality of imaging pixels. In the focus detection apparatus according to any one of claims 1 to 5,
The focus detection device, wherein the first pixel column and the second pixel column are arranged to intersect each other. In the focus detection apparatus according to any one of claims 1 to 6,
The plurality of pixels receive a pair of light beams passing through different regions on the pupil of the optical system and output a signal for detecting a focus adjustment state of the optical system. The focus detection apparatus according to claim 7,
The pixel includes a microlens and a pair of photoelectric conversion units arranged with respect to the microlens. The focus detection apparatus according to claim 7,
The pixel includes a first focus detection pixel including a microlens and a first photoelectric conversion unit disposed with respect to the microlens, and a second photoelectric conversion unit disposed with respect to the microlens and the microlens. A focus detection apparatus comprising: a second focus detection pixel configured.   An imaging apparatus comprising the focus detection apparatus according to claim 1.

Images