JP5332384B2 - Correlation calculation device, focus detection device, and imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To calculate an accurate amount of relative displacement for a pair of data lines corresponding to a pair of object images relatively distorted. <P>SOLUTION: The method performs: a first calculation (S220) in which a first data line in which a plurality of data are one-dimensionally arranged and a second data line in which a plurality of data are one-dimensionally arranged are relatively displaced while an amount of displacement is one-dimensionally changed, and an amount of correlation between the first and the second data lines is calculated, thereby calculating a first amount of displacement in which an extreme value of the amount for the correlation is obtained; a second calculation (S220) in which the first data line and the first data line are relatively displaced while an amount of displacement is one-dimensionally changed, and an amount of autocorrelation between the first data lines is calculated, thereby calculating a second amount of displacement in which an extreme value for the amount of autocorrelation is obtained; and a third calculation (S221) in which on the basis of the second amount of displacement, the first amount of displacement that gives the actual extreme value for the amount of correlation is selected. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a correlation calculation method, a correlation calculation device, a focus detection device, an imaging device, and a correlation calculation program.

There is known a focus detection device that calculates a relative displacement amount (image shift amount, shift amount) between a pair of data strings corresponding to a pair of relatively distorted object images (for example, Patent Document 1). reference).
In this apparatus, a pair of data strings {A1, A2, A3,...}, {B1, B2, B3,. The correlation amount C (k) is calculated using the correlation calculation formula including the displacement amount, and the displacement amount between the pair of data strings indicating the extreme value of the correlation amount C (k) is calculated.

Prior art documents related to the invention of this application include the following.
JP 2007-333720 A

  However, in the conventional focus detection apparatus described above, the correlation calculation for calculating the correlation amount C (k) has a characteristic capable of detecting a relative displacement amount even with respect to a pair of object images that are relatively distorted. Therefore, there is a case where a correlation amount (false correlation amount) showing a high degree of correlation is calculated even for a pair of originally different data.

According to a first aspect of the present invention, there is provided a correlation calculation device comprising: a first electric signal data string output means for outputting a first electric signal data string composed of a plurality of electric signal data; The first electric signal data string output means for outputting the electric signal data string, the first electric signal data string, and the second electric signal data string are relatively displaced while changing the displacement amount, and the first electric signal data string output means. A cross-correlation amount between the electrical signal data sequence and the second electrical signal data sequence is calculated by a first correlation calculation, and a plurality of extreme values and a plurality of extreme values corresponding to the plurality of extreme values respectively. a cross-correlation calculating means for calculating a first displacement amount of said first electrical signal data string and the first electrical signal data string, by relatively displacing while changing the amount of displacement, the first electrical signal Autocorrelation between data strings Self calculating a first calculated by the second correlation operation that is different from the correlation operation, a plurality of the second displacement amount corresponding to a plurality of extreme values and the plurality of extreme values of the autocorrelation First, a true extreme value of the cross-correlation amount is obtained from the plurality of first displacement amounts based on a correlation calculation means, and a plurality of extreme values of the autocorrelation amount and a separation amount of the plurality of second displacement amounts . Sorting means for sorting one displacement amount.
A focus detection apparatus according to a second aspect of the present invention receives a pair of light beams passing through a pair of regions of a pupil of an imaging optical system, and a first electric signal data string corresponding to a pair of images formed by the pair of light beams. An electric signal data string generating means for generating a second electric signal data string, the first electric signal data string and the second electric signal data string are relatively displaced while changing a displacement amount, and the first electric signal data string generating means generates the second electric signal data string. A cross-correlation amount between one electrical signal data sequence and the second electrical signal data sequence is calculated by a first correlation operation, and corresponds to a plurality of extreme values and a plurality of extreme values of the cross-correlation amount , respectively. a cross-correlation calculating means for calculating a plurality of first displacement, the said first electrical signal data string and said first electrical signal data string, and relatively displaced so while changing the amount of displacement, the first electrical The amount of autocorrelation between signal data strings is the first correlation. Calculated is calculated by the second correlation operation that is different from the, the autocorrelation calculating means for calculating a plurality of second displacement amount corresponding to a plurality of extreme values and the plurality of extreme values of the autocorrelation, A first displacement amount that gives a true extreme value of the cross-correlation amount is selected from the plurality of first displacement amounts based on the plurality of extreme values of the autocorrelation amount and the separation amounts of the plurality of second displacement amounts. And a focus detection unit that detects a focus adjustment state of the imaging optical system based on the first displacement selected by the selection unit.
According to a fourth aspect of the present invention, there is provided a correlation calculation apparatus comprising: a first electric signal data string output means for outputting a first electric signal data string comprising a plurality of electric signal data; and a second electric signal data string comprising a plurality of electric signal data. The first electric signal data string output means for outputting the electric signal data string, the first electric signal data string, and the second electric signal data string are relatively displaced while changing the displacement amount, and the first electric signal data string output means. A cross-correlation amount between the electrical signal data sequence and the second electrical signal data sequence is calculated by a first correlation calculation, and a plurality of extreme values and a plurality of extreme values corresponding to the plurality of extreme values respectively. a cross-correlation calculating means for calculating a first displacement amount of said first electrical signal data string and the first electrical signal data string, by relatively displacing while changing the amount of displacement, the first electrical signal Autocorrelation between data strings An autocorrelation calculating means for calculating by a second correlation calculation different from the first correlation calculation, and calculating a correlation amount between the autocorrelation amount and the cross-correlation amount, and obtaining an extreme value of the correlation amount The obtained second displacement amount is calculated, and the first displacement amount closest to the second displacement amount is selected from the plurality of first displacement amounts as a first displacement amount that gives a true extreme value of the cross-correlation amount. And sorting means for performing.
According to a fifth aspect of the present invention, a focus detection apparatus receives a pair of light beams passing through a pair of regions of a pupil of an imaging optical system, and a first electric signal data string corresponding to a pair of images formed by the pair of light beams. An electric signal data string generating means for generating a second electric signal data string, the first electric signal data string and the second electric signal data string are relatively displaced while changing a displacement amount, and the first electric signal data string generating means generates the second electric signal data string. A cross-correlation amount between one electrical signal data sequence and the second electrical signal data sequence is calculated by a first correlation operation, and corresponds to a plurality of extreme values and a plurality of extreme values of the cross-correlation amount , respectively. a cross-correlation calculating means for calculating a plurality of first displacement, the said first electrical signal data string and said first electrical signal data string, and relatively displaced so while changing the amount of displacement, the first electrical The amount of autocorrelation between signal data strings is the first correlation. An autocorrelation calculating means for calculating by a second correlation operation different from the calculation, and a second displacement for calculating the correlation amount between the autocorrelation amount and the cross-correlation amount to obtain an extreme value of the correlation amount Selecting means for calculating an amount and selecting the first displacement amount closest to the second displacement amount as a first displacement amount that gives a true extreme value of the cross-correlation amount from the plurality of first displacement amounts; Focus detection means for detecting a focus adjustment state of the imaging optical system based on the first displacement amount selected by the selection means.

  According to the present invention, even when a correlation calculation capable of detecting a relative displacement amount is used for a pair of first data strings and a second data string corresponding to a pair of object images that are relatively distorted. Therefore, it is possible to accurately calculate the displacement amount that gives the true extreme value of the cross-correlation amount by eliminating the displacement amount that gives the false extreme value of the cross-correlation amount.

  An embodiment of the present invention will be described by taking an interchangeable lens digital still camera as an example. FIG. 1 is a cross-sectional view of a camera showing the configuration of the camera of one embodiment. A digital still camera 201 according to an embodiment includes an interchangeable lens 202 and a camera body 203, and the interchangeable lens 202 is attached to the camera body 203 via a mount unit 204. An interchangeable lens 202 having various photographing optical systems can be attached to the camera body 203 via a mount unit 204.

  The interchangeable lens 202 includes a lens 209, a zooming lens 208, a focusing lens 210, an aperture 211, a lens drive control device 206, and the like. The lens drive control device 206 includes a microcomputer (not shown), a memory, a drive control circuit, and the like, and performs drive control for adjusting the focus of the focusing lens 210 and adjusting the aperture diameter of the aperture 211, zooming lens 208, and focusing. In addition to detecting the state of the lens 210 and the aperture 211, the lens information is transmitted and the camera information is received through communication with a body drive control device 214 described later. The aperture 211 forms an aperture having a variable aperture diameter at the center of the optical axis in order to adjust the amount of light and the amount of blur.

  The camera body 203 includes an imaging element 212, a body drive control device 214, a liquid crystal display element drive circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, and the like. In the imaging element 212, imaging pixels are two-dimensionally arranged, and focus detection pixels are incorporated in portions corresponding to focus detection positions. Details of the image sensor 212 will be described later.

The body drive control device 214 includes a microcomputer, a memory, a drive control circuit, and the like, and controls the drive of the image sensor 212, reads out the image signal and the focus detection signal, performs the focus detection calculation based on the focus detection signal, and the focus of the interchangeable lens 202. The adjustment is repeated, and image signal processing and recording, camera operation control, and the like are performed. The body drive control device 214 communicates with the lens drive control device 206 via the electrical contact 213 to receive lens information and send camera information (defocus amount, aperture value, etc.).

  The liquid crystal display element 216 functions as an electric view finder (EVF). The liquid crystal display element driving circuit 215 displays a through image by the imaging element 212 on the liquid crystal display element 216, and the photographer can observe the through image through the eyepiece lens 217. The memory card 219 is an image storage that stores an image captured by the image sensor 212.

  A subject image is formed on the light receiving surface of the image sensor 212 by the light beam that has passed through the interchangeable lens 202. This subject image is photoelectrically converted by the image sensor 212, and an image signal and a focus detection signal are sent to the body drive control device 214.

  The body drive control device 214 calculates the defocus amount based on the focus detection signal from the focus detection pixel of the image sensor 212 and sends the defocus amount to the lens drive control device 206. The body drive control device 214 processes the image signal from the image sensor 212 to generate an image, stores the image in the memory card 219, and sends the through image signal from the image sensor 212 to the liquid crystal display element drive circuit 215. A through image is displayed on the liquid crystal display element 216. Further, the body drive control device 214 sends aperture control information to the lens drive control device 206 to control the aperture of the aperture 211.

  The lens drive controller 206 updates the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the positions of the zooming lens 208 and the focusing lens 210 and the aperture value of the aperture 211 are detected, and lens information is calculated according to these lens positions and aperture values, or a lookup prepared in advance. Lens information corresponding to the lens position and aperture value is selected from the table.

  The lens drive control device 206 calculates a lens drive amount based on the received defocus amount, and drives the focusing lens 210 to the in-focus position according to the lens drive amount. Further, the lens drive control device 206 drives the diaphragm 211 in accordance with the received diaphragm value.

  FIG. 2 is a diagram illustrating a focus detection position on the imaging screen of the interchangeable lens 202, and a region (focal point) in which an image is sampled on the imaging screen when a focus detection pixel array on the image sensor 212 described later performs focus detection. An example of a detection area and a focus detection position is shown. In this example, focus detection areas 101 to 103 are arranged at the center and three locations above and below the rectangular shooting screen 100. Focus detection pixels are linearly arranged in the longitudinal direction of a focus detection area indicated by a rectangle.

  FIG. 3 is a front view showing a detailed configuration of the image sensor 212, and shows an enlarged vicinity of the focus detection area 101 on the image sensor 212. Imaging pixels 310 are densely arranged in a two-dimensional square lattice pattern on the imaging element 212, and focus detection pixels 313 and 314 for focus detection are on a vertical straight line at positions corresponding to the focus detection area 101. Are alternately arranged adjacent to each other. Although not shown, the configuration in the vicinity of the focus detection areas 102 and 103 is the same as the configuration shown in FIG.

  As shown in FIG. 4, the imaging pixel 310 includes the microlens 10, the photoelectric conversion unit 11, and a color filter (not shown). There are three types of color filters, red (R), green (G), and blue (B), and the respective spectral sensitivities have the characteristics shown in FIG. The imaging element 212 has a Bayer array of imaging pixels 310 having respective color filters.

  The focus detection pixel 313 includes the microlens 10 and the photoelectric conversion unit 13 as illustrated in FIG. 5A, and the photoelectric conversion unit 13 has a semicircular shape. Further, the focus detection pixel 314 includes the microlens 10 and the photoelectric conversion unit 14 as shown in FIG. 5B, and the photoelectric conversion unit 14 has a semicircular shape. The focus detection pixel 313 and the focus detection pixel 314 are used for focus detection in the vertical direction, and the photoelectric conversion units 13 and 14 are arranged at the upper part or the lower part of the pixel. The focus detection pixels 313 and the focus detection pixels 314 are alternately arranged in the vertical direction (alignment direction of the photoelectric conversion units 13 and 14) in the focus detection areas 101 to 103.

  The focus detection pixels 313 and 314 are not provided with a color filter to increase the amount of light, and the spectral characteristics thereof are the spectral sensitivity of a photodiode that performs photoelectric conversion and the spectral characteristics of an infrared cut filter (not shown). And the spectral characteristics (see FIG. 7). That is, the spectral characteristics are obtained by adding the spectral characteristics of the green pixel, the red pixel, and the blue pixel shown in FIG. 6, and the light wavelength region of the sensitivity includes the light wavelength regions of the sensitivity of the green pixel, the red pixel, and the blue pixel. ing.

  The focus detection pixels 313 and 314 for focus detection are arranged in a column in which B and G of the imaging pixel 310 are to be arranged. The focus detection pixels 313 and 314 for focus detection are arranged in a column in which B and G of the imaging pixel 310 are to be arranged when an interpolation error occurs in the pixel interpolation processing. This is because the interpolation error of the blue pixel is less noticeable than the interpolation error of the red pixel.

  The photoelectric conversion unit 11 of the imaging pixel 310 is designed in such a shape that the microlens 10 receives all the light beams that pass through the exit pupil diameter (for example, F1.0) of the brightest interchangeable lens. In addition, the photoelectric conversion units 13 and 14 of the focus detection pixels 313 and 314 are designed so as to receive all the light beams passing through a predetermined region (for example, F2.8) of the exit pupil of the interchangeable lens by the microlens 10. Is done.

  FIG. 8 is a cross-sectional view of the imaging pixel 310. In the imaging pixel 310, the microlens 10 is disposed in front of the imaging photoelectric conversion unit 11, and the shape of the photoelectric conversion unit 11 is projected forward by the microlens 10. The photoelectric conversion unit 11 is formed on the semiconductor circuit substrate 29. A color filter (not shown) is arranged between the microlens 10 and the photoelectric conversion unit 11.

  FIG. 9A is a cross-sectional view of the focus detection pixel 313. In the focus detection pixel 313 disposed in the focus detection area 101 at the center of the screen, the microlens 10 is disposed in front of the photoelectric conversion unit 13, and the shape of the photoelectric conversion unit 13 is projected forward by the microlens 10. The photoelectric conversion unit 13 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon by a semiconductor image sensor manufacturing process. The cross-sectional structure of the focus detection pixels 313 arranged in the focus detection areas 102 and 103 at the top and bottom of the screen is the same as the cross-sectional structure shown in FIG.

  FIG. 9B is a cross-sectional view of the focus detection pixel 314. In the focus detection pixel 314 disposed in the focus detection area 101 at the center of the screen, the microlens 10 is disposed in front of the photoelectric conversion unit 14, and the shape of the photoelectric conversion unit 14 is projected forward by the microlens 10. The photoelectric conversion unit 14 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon by the manufacturing process of the semiconductor image sensor. Note that the cross-sectional structure of the focus detection pixels 314 arranged in the focus detection areas 102 and 103 at the top and bottom of the screen is the same as the cross-sectional structure shown in FIG.

  FIG. 10 shows a configuration of a pupil division type phase difference detection type focus detection optical system using a microlens. The focus detection pixel portion is shown enlarged. In the figure, reference numeral 90 denotes an exit pupil set at a distance d forward from the microlens arranged on the planned imaging plane of the interchangeable lens 202 (see FIG. 1). This distance d is a distance determined according to the curvature and refractive index of the microlens, the distance between the microlens and the photoelectric conversion unit, and is referred to as a distance measuring pupil distance in this specification. 91 is an optical axis of the interchangeable lens, 10a to 10d are microlenses, 13a, 13b, 14a, and 14b are photoelectric conversion units, 313a, 313b, 314a, and 314b are focus detection pixels, and 73, 74, 83, and 84 are focus detections. Luminous flux.

  Reference numeral 93 denotes an area of the photoelectric conversion units 13a and 13b projected by the microlenses 10a and 10c, and is referred to as a distance measuring pupil in this specification. In FIG. 10, an elliptical region is shown for easy understanding, but the shape of the photoelectric conversion unit is actually an enlarged projection shape. Similarly, 94 is an area of the photoelectric conversion units 14a and 14b projected by the microlenses 10b and 10d, and is called a distance measuring pupil in this specification. In FIG. 10, an elliptical region is shown for easy understanding, but the shape of the photoelectric conversion unit is actually an enlarged projection shape.

  In FIG. 10, four focus detection pixels 313a, 313b, 314a, and 314b adjacent to the photographing optical axis are schematically illustrated. However, other focus detection pixels in the focus detection area 101 may also be displayed at the periphery of the screen. Also in the focus detection pixels in the focus detection areas 102 and 103, the photoelectric conversion units are configured to receive light beams that arrive at the microlenses from the corresponding distance measurement pupils 93 and 94, respectively. The arrangement direction of the focus detection pixels is made to coincide with the arrangement direction of the pair of distance measuring pupils, that is, the arrangement direction of the pair of photoelectric conversion units.

  The microlenses 10a to 10d are disposed in the vicinity of the planned imaging plane of the interchangeable lens 202 (see FIG. 1), and the shapes of the photoelectric conversion units 13a, 13b, 14a, and 14b disposed behind the microlenses 10a to 10d. Is projected onto the exit pupil 90 separated from the microlenses 10a to 10c by the distance measurement pupil distance d, and the projection shape forms distance measurement pupils 93 and 94. That is, the microlens and the photoelectric conversion unit in each focus detection pixel so that the projection shapes (distance measurement pupils 93 and 94) of the photoelectric conversion unit of each focus detection pixel coincide on the exit pupil 90 at the projection distance d. Is determined, and the projection direction of the photoelectric conversion unit in each focus detection pixel is thereby determined.

  The photoelectric conversion unit 13a passes through the distance measuring pupil 93 and outputs a signal corresponding to the intensity of the image formed on the microlens 10a by the light beam 73 directed to the microlens 10a. Similarly, the photoelectric conversion unit 13b outputs a signal corresponding to the intensity of the image formed on the microlens 10c by the light beam 83 passing through the distance measuring pupil 93 and directed to the microlens 10c. Further, the photoelectric conversion unit 14a outputs a signal corresponding to the intensity of the image formed on the microlens 10b by the light beam 74 passing through the distance measuring pupil 94 and directed to the microlens 10b. Similarly, the photoelectric conversion unit 14b outputs a signal corresponding to the intensity of the image formed on the microlens 10d by the light beam 84 passing through the distance measuring pupil 94 and directed to the microlens 10d.

  A large number of the above-mentioned two types of focus detection pixels are arranged in a straight line, and the output of the photoelectric conversion unit of each pixel is grouped into an output group corresponding to the distance measuring pupil 93 and the distance measuring pupil 94, whereby the distance measuring pupil 93 and Information on the intensity distribution of the pair of images formed on the pixel array by the focus detection light beams that respectively pass through the distance measuring pupil 94 is obtained. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process), which will be described later, to this information, an image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method. Further, by performing a conversion operation according to the center-of-gravity interval of the pair of distance measuring pupils on the image shift amount, the current image plane relative to the planned image plane (the focus corresponding to the position of the microlens array on the planned image plane) The deviation (defocus amount) of the imaging plane at the detection position is calculated.

  FIG. 11 is a flowchart illustrating an imaging operation of the digital still camera (imaging device) according to the embodiment. When the power of the camera is turned on in step 100, the body drive control device 214 starts the imaging operation after step 110. In step 110, the image pickup pixel data is read out and displayed on the electronic viewfinder. In subsequent step 120, a pair of image data corresponding to the pair of images is read from the focus detection pixel array. The focus detection area is assumed to be selected in advance by the photographer using one of the focus detection areas 101 to 103 using a focus detection area selection member (not shown).

  In step 130, an image shift detection calculation process (correlation calculation process) described later is performed based on the read pair of image data, and the image shift amount is calculated and converted into a defocus amount. In step 140, it is checked whether or not the focus is close, that is, whether or not the absolute value of the calculated defocus amount is within a predetermined value. If it is determined that the lens is not in focus, the process proceeds to step 150, where the defocus amount is transmitted to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is driven to the focus position. Then, it returns to step 110 and repeats the operation | movement mentioned above.

  Even when focus detection is impossible, the process branches to this step, a scan drive command is transmitted to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is driven to scan from infinity to the nearest. Then, it returns to step 110 and repeats the operation | movement mentioned above.

  If it is determined in step 140 that the focus is close to the in-focus state, the process proceeds to step 160, where it is determined whether or not a shutter release has been performed by operating a shutter button (not shown). If it is determined that the shutter release has not been performed, the process returns to step 110 to repeat the above-described operation. On the other hand, if it is determined that the shutter release has been performed, the process proceeds to step 170, where an aperture adjustment command is transmitted to the lens drive control device 206, and the aperture value of the interchangeable lens 202 is controlled to a control F value (F set by the photographer or automatically). Value). When the aperture control is completed, the image sensor 212 performs an image capturing operation, and image data is read from the image capturing pixel 310 and all the focus detection pixels 313 and 314 of the image sensor 212.

  In step 180, pixel interpolation is performed on pixel data at each pixel position in the focus detection pixel row based on data of imaging pixels around the focus detection pixel. In the subsequent step 190, image data composed of the image pickup pixel data and the interpolated data is stored in the memory card 219, and the process returns to step 110 to repeat the above-described operation.

  FIG. 12 is a flowchart showing details of the focus detection calculation process in step 130 of FIG. The body drive control device 214 starts this focus detection calculation process (correlation calculation process) from step 200.

In step 210, a pair of data strings (α1 to αM, β1 to βM, where M is the number of data) output from the focus detection pixel column is subjected to a high frequency cut filter process as shown in equation (1), One data string (A1 to AN) and a second data string (B1 to BN) are generated. As a result, noise components and high-frequency components that adversely affect the correlation processing can be removed from the data string. Note that the processing of step 210 can be omitted when shortening the calculation time or when it is already known that there is little high-frequency component since it has been greatly defocused.
An = αn + 2, αn + 1 + αn + 2,
Bn = βn + 2 · βn + 1 + βn + 2 (1)
In the formula (1), n = 1 to M−2.

  Next, at step 220, a correlation operation as described in detail below is performed on the data strings An and Bn. The data strings An and Bn should ideally be the same data string shifted relatively, but in the pair of data strings obtained by the above-described pupil division type focus detection pixels, the data strings An and Bn are for focus detection. There is a possibility that the identity may be lost (distortion occurs) due to vignetting of the light beam.

  FIG. 13 is a diagram for explaining vignetting (vignetting) of the focus detection light beam. In the figure, a pair of focus detection pixels at a position x0 (image height 0) and a position x1 (image height h) are each a distance measurement pupil region 93, A pair of focus detection light beams 53, 54 and 63, 64 passing through 94 are received. When there is an aperture stop 96 of the optical system on the front surface d1 (<d) 95 of the planned focal plane 92, a pair of focus detection light received by the pair of focus detection pixels at the position x0 (image height 0). The luminous fluxes 53 and 54 cause vignetting symmetrically with respect to the optical axis 91 due to the aperture 96, so that the balance of the amount of light received by the pair of focus detection pixels is not lost.

  On the other hand, the pair of focus detection light beams 63 and 64 received by the pair of focus detection pixels at the position x1 (image height h) are asymmetrically vignetted by the diaphragm opening 96, so The balance of the amount of light received by the detection pixels is lost.

  FIG. 14 is a diagram when the distance measuring pupil plane 90 is viewed from the planned focal plane 92 in the direction of the optical axis 91. It can be seen that the focus detection light beam 64 has a large amount of vignetting due to the aperture 96, whereas the focus detection light beam 63 has less vignetting due to the aperture 96.

  FIGS. 15A and 15B show a pair of images received by the focus detection pixel column in the vicinity of the position x0 (image height 0) and the position x1 (image height h) in the states of FIGS. The intensity distribution of a pair of images received by a nearby focus detection pixel array (the vertical axis indicates the amount of light, and the horizontal axis indicates the position on the imaging screen). When the vignetting balance of the focus detection light beam is balanced, as shown in FIG. 15A, the pair of image signals 400 and 401 are obtained by simply shifting the same image signal function in the horizontal direction. ing. On the other hand, when the vignetting balance of the focus detection light beam is lost, as shown in FIG. 15B, the pair of image signals 402 and 403 are obtained by relatively shifting the same signal. Must not.

In FIG. 15B, when the image signals 402 and 403 having a distorted relationship are expressed as F (x) and G (x) as a function of the position x, they can be approximated as shown in the equation (2). That is, when one function F (x) is multiplied by a low frequency gain component (1 + δ · x) · M with respect to the position x, the other function G (x) is obtained.
G (x) = (1 + δ · x) · M · F (x) (2)
In Equation (2), M is a fixed gain value, and δ (<< 1) is a primary coefficient related to the position x.

In the state where such vignetting has occurred, as a correlation calculation that can detect an accurate image shift amount with respect to a pair of image data strings An and Bn, the present applicant has disclosed (3) in Japanese Patent Laid-Open No. 2007-333720 and the like. A correlation calculation as shown in the equation is proposed.
C (k) = Σ | Ai · Bi + s + k−Bi + k · Ai + s | (3)
In the above equation (3), k is a relative shift value of the pair of image data sequences An and Bn, and C (k) is a correlation amount indicating the degree of correlation between the pair of image data sequences An and Bn at the shift value k. It is. Further, the integration process (Σ) is performed over a predetermined section of data with respect to the data index i. Further, the parameter s (= 1, 2, 3,...) Is a parameter for adjusting the frequency characteristic of the correlation calculation.

  When the shift value k is changed and the correlation amount C (k) is obtained by the equation (3), the correlation amount C (k) is a minimum value (ideal value) in the shift value having a high correlation degree between the pair of image data strings An and Bn. Therefore, by obtaining a shift value indicating the minimum value of the correlation amount C (k), the image shift amount of the pair of image data sequences An and Bn can be detected. In general, no problem occurs in the correlation calculation processing of the expression (3) in most cases, but the expression (3) can detect the image shift amount even if the pair of image data strings An and Bn are relatively distorted. For this reason, in a special case as described later, there is a possibility that a high degree of correlation may be calculated for a different function.

  FIGS. 16A and 16B are examples in the case where the above-described malfunction occurs, and show functions F (x) and G (x) having no distortion with the horizontal axis as the position x. The function G (x) is a function obtained by shifting the function F (x) by x1 in the negative direction of the position x. The function F (x) includes rectangular peak waveforms P1 and P2, and the function G (x) includes rectangular peak waveforms Q1 and Q2. The peak waveforms P1 and Q1 and the peak waveforms P2 and Q2 are the same waveform and have a corresponding relationship. However, the peak waveforms P2 and Q2 and the peak waveforms P1 and Q1 are different waveforms that are multiplied by a predetermined value. P2 (Q2) is at a position away from the peak waveform P1 (Q1) by (−x1 + x2) in the + direction of the position x.

  When the section W including the peak waveform P1 of the function F (x) is set as one data string and the function G (x) is relatively shifted as the other data string, the correlation calculation of the expression (3) is applied. When the correlation amount C (k) calculated with the horizontal axis as the shift value k is shown, a graph shown in FIG. 17B is obtained. In the graph shown in FIG. 17B, the shift values k1 and k2 corresponding to the drop (minimum values) V1 and V2 of the correlation amount C (k) are the shift values having high correlation between the two data.

  The shift value k1 corresponding to the drop V1 is obtained by matching the peak waveform P1 of the function F (x) shown in FIG. 16 (a) with the peak waveform Q1 of the function G (x) shown in FIG. 16 (b). This corresponds to the shift value when the function G (x) is shifted to the right by x1 with reference to the function F (x) in the section W.

  The shift value k2 corresponding to the drop V2 is determined by the coincidence between the peak waveform P1 of the function F (x) shown in FIG. 16A and the peak waveform Q2 of the function G (x) shown in FIG. This corresponds to the shift value obtained when the function G (x) is shifted to the left by x2 with reference to the function F (x) in the section W.

  Since the correlation amounts R1 and R2 of the drops V1 and V2 have almost the same value, it is not known which of the correlations shows the correct match between the functions F (x) and G (x). When the shift value k2 corresponding to the drop V2 is adopted as the shift amount between the two functions, a completely wrong focus detection result is calculated.

Therefore, in order to prevent a malfunction that occurs when the pair of images is a function as shown in FIGS. 16A and 16B, the following measures are taken. First, an autocorrelation calculation process as shown in Equation (4) is performed to determine the positional relationship in the axial direction of the shift value between the true correlation and the false correlation.
C (k) = Σ | Ai−Ai + k | (4)
In equation (4), k is a relative shift value between the image data sequence An and its own image data sequence An, and C (k) is a correlation indicating the degree of autocorrelation of the image data sequence An at the shift value k. Amount. Further, the integration process (Σ) is performed over a predetermined section of data (corresponding to section W) with respect to the data index i.

  In the autocorrelation calculation of equation (4), the absolute value of the data difference is integrated, so if the waveform is completely matched, the correlation amount is almost zero, and the waveform is incompletely matched. In this case, for example, when a gain is applied to one of the waveforms, the drop in the correlation amount is reduced.

  When the autocorrelation operation of the equation (4) is applied to the function F (x) with the section W including the peak waveform P1 of the function F (x) shown in FIG. 16A as a reference, the horizontal axis is calculated as the shift value k. When the correlation amount C (k) is shown, the graph shown in FIG.

  The above is the processing of step 220 in FIG. Next, in step 230, a drop indicating a true correlation is determined as follows from the two correlation graphs shown in FIGS. 17 (a) and 17 (b). In the graph shown in FIG. 17A, drops U1 and U2 of the correlation amount C (k) occur at a shift value 0, (−k1 + k2), but the shift value 0 corresponding to the drop U1 is a true autocorrelation ( (Correlation between peak waveforms P1), and the correlation value E1 is also a small value. On the other hand, the shift value (−k1 + k2) corresponding to the drop U2 is a drop due to false autocorrelation (correlation between the peak waveforms P1 and P2), and the value E2 of the correlation amount is larger than the value E1. Become.

  Therefore, from the graph of FIG. 17 (a), (a) there are two depressions (U1, U2), (b) the depression showing the true correlation is the depression on the left side of the two depressions, The drop is a false correlation, and (c) information that the distance between the drop U1 and the drop U2 is (−k1 + k2) is obtained. Based on these pieces of information, the reliability of two drops is judged from the substantially equivalent drops V1 and V2 shown in FIG. 17B, and the drop showing the true correlation is selected.

  In FIG. 17B, it is determined that there is a correspondence between the depressions V1 and V2 and the depressions U1 and U2 because the separation amounts of the depressions V1 and V2 at the two locations substantially coincide with the separation amounts of the depressions U1 and U2. Then, the drop V1 corresponding to the true drop U1 (the left side of the two drops) is recognized as a drop showing a true correlation, and the drop V2 corresponding to the false drop U2 (the right side of the two drops) ) Is identified as a drop indicating a false correlation. Therefore, in the case shown in FIG. 16, the shift value k1 corresponding to the drop V1 is recognized as a true shift value.

  In the above description, the two functions F (x) and G (x) shown in FIGS. 16 (a) and 16 (b) are relatively shifted functions. As shown in the equation (2), Even when the function obtained by multiplying the function F (x) by a low frequency gain component with respect to the position x is the other function G (x), the result of the correlation calculation of the equation (3) is shown in FIG. Since the graph is as shown, the true shift value can be determined in the same manner.

  In the above description, the case where the drop of the cross-correlation function is two has been described, but the true shift value can be determined by the same process also in the case of two or more.

  In step 230, when there is one drop in the cross-correlation function graph, it is not necessary to perform a comparison process with the autocorrelation function graph, and the shift value corresponding to the drop in one place is directly used as the true shift value. do it.

In step 240, the obtained true shift value is converted into a defocus amount as follows. In the true shift value of the graph of the correlation amount C (k) obtained by the correlation calculation formula (3), as shown in FIG. 18A, a shift amount having a high correlation between a pair of data (FIG. 18A). In k = kj = 2), the correlation amount C (k) is minimum (the smaller the correlation, the higher the correlation degree). The shift amount x that gives the minimum value C (x) with respect to the continuous correlation amount is obtained by using the three-point interpolation method according to the equations (5) to (8).
x = kj + D / SLOP (5),
C (x) = C (kj) − | D | (6),
D = {C (kj-1) -C (kj + 1)} / 2 (7),
SLOP = MAX {C (kj + 1) -C (kj), C (kj-1) -C (kj)} (8)

In step 240, the defocus amount DEF of the subject image plane with respect to the planned image formation plane can be obtained by the following equation using the shift amount x obtained by equation (5).
DEF = KX · PY · x (9)
In equation (9), PY is the detection pitch, and KX is a conversion coefficient determined by the size of the opening angle of the center of gravity of the pair of distance measurement pupils.

  Whether or not the calculated defocus amount DEF is reliable is determined as follows. As shown in FIG. 18B, when the degree of correlation between a pair of data is low, the value of the interpolated minimum amount C (x) of the correlation amount increases. Therefore, it is determined that the reliability is low when C (x) is equal to or greater than a predetermined value. Alternatively, in order to normalize C (x) with the contrast of data, if the value obtained by dividing C (x) by SLOP that is proportional to the contrast is equal to or greater than a predetermined value, it is determined that the reliability is low. Alternatively, if SLOP that is proportional to the contrast is equal to or less than a predetermined value, it is determined that the subject has low contrast and the reliability of the calculated defocus amount DEF is low. As shown in FIG. 18C, when the correlation between the pair of data is low and there is no drop in the correlation amount C (k) between the shift ranges kmin to kmax, the minimum value C (x) is obtained. In such a case, it is determined that the focus cannot be detected. When focus detection is possible, the defocus amount is calculated by multiplying the calculated image shift amount by a predetermined conversion coefficient.

  In step 250, the focus detection calculation process (correlation calculation process) is terminated, and the process returns to step 140 in FIG.

  Thus, according to one embodiment, a first data sequence An in which a plurality of data is arranged in a one-dimensional manner and a second data sequence Bn in which a plurality of data are arranged in a one-dimensional manner are converted into a primary The first data string An and the second data string Bn are relatively displaced while changing the displacement amount, and the cross-correlation amount between the first data string An and the second data string Bn is calculated. A cross-correlation operation for calculating a first displacement amount that obtains an extreme value (a drop in the cross-correlation amount), and a relative relationship between the first data sequence An and the first data sequence An while changing the displacement amount in one dimension. Autocorrelation calculation for calculating a second displacement amount for which an autocorrelation amount between the first data strings An is calculated to obtain an extreme value of the autocorrelation amount (a drop in the autocorrelation amount), and a second displacement amount And a calculation for selecting a first displacement amount that gives a true extreme value of the cross-correlation amount based on It was. That is, the relative relationship between the first displacement amount and the second displacement amount is calculated based on the number of first displacement amounts and the separation amount between the displacement amounts, and the number of second displacement amounts and the separation amount between the displacement amounts. Since the first displacement amount corresponding to the second displacement amount that gives the true extreme value of the autocorrelation amount is selected as the displacement amount that gives the true extreme value of the cross-correlation amount, a relative distortion has occurred. A displacement amount that gives a false extreme value of a cross-correlation amount even when a correlation calculation capable of detecting a relative displacement amount with respect to a pair of first data sequence and second data sequence corresponding to a pair of object images is used. It is possible to accurately calculate a displacement amount (displacement amount that gives a true cross-correlation) that gives a true extreme value of the cross-correlation amount by eliminating (a displacement amount that gives a false cross-correlation).

<< Other Embodiments of the Invention >>
In the above-described embodiment, a true correlation drop is selected and false by comparing the positional relationships of a plurality of drop points in the autocorrelation graph and the cross-correlation graph shown in FIGS. 17 (a) and 17 (b). Although the method for eliminating the drop in the correlation has been described, the true drop in the correlation can be selected by further performing a correlation operation between the autocorrelation graph and the cross-correlation graph.

  FIG. 19 is a flowchart showing another embodiment of the focus detection calculation process in step 130 of FIG. The body drive control device 214 starts this focus detection calculation process (correlation calculation process) from step 300. The processing from step 300 to step 320 is the same as the processing from step 300 to step 320 in FIG. When a pair of data strings corresponding to the functions as shown in FIGS. 16A and 16B are assumed, the graph of the autocorrelation function C1 (k) at the time when step 320 is completed is shown in FIG. And FIG. 20B is obtained as a graph of the cross-correlation function C2 (k).

  In step 330, a correlation calculation is performed between the cross-correlation function and the autocorrelation function, and a true correlation is determined from the calculation result. In the graph of the autocorrelation function C1 (k) shown in FIG. 20A, there are two drops U1 and U2, and as described above, the drop U1 shows a true correlation and the drop U2 shows a false correlation. .

  In the graph of the cross-correlation function C2 (k) shown in FIG. 20B, there are two drops V1 and V2, but it is impossible to identify which is a true correlation and which is a false correlation only from this graph. Therefore, a section W1 including two drops V1 and V2 of the cross-correlation function C2 (k) is set as a reference section, and the autocorrelation function C1 (k) with respect to the cross-correlation function C2 (k) in the reference section W1. Is relatively shifted by the shift value h, thereby calculating the degree of correlation between the two correlation functions.

Assuming that the data string corresponding to C1 (k) is C1n and the data string corresponding to C2 (k) is C2n, the correlation amount C3 (h) is calculated by the correlation calculation expression (10) similar to the above expression (3). Can be sought.
C3 (h) = Σ | C2i · C1i + s + h−C1i + h · C2i + s | (10)
In Expression (10), h is a relative shift value of the pair of data strings C1n and C2n, and C3 (h) is a correlation amount indicating the degree of correlation between the pair of data strings C1n and C2n at the shift value h. Further, the integration process (Σ) is performed over a predetermined section of data with respect to the data index i.

  FIG. 20C shows the correlation amount C3 (h) obtained by sequentially changing the shift value h, and has three drops Y1, Y2, and Y3. The drop Y1 is a drop due to the coincidence of the drop U2 portion of the autocorrelation function C1 (k) and the drop V1 portion of the cross-correlation function C2 (k), and the correlation value S1 is a relatively large value. The drop Y3 is a drop due to the coincidence of the drop U1 part of the autocorrelation function C1 (k) and the drop V2 part of the cross-correlation function C2 (k), and the correlation value S3 is a relatively large value.

  Furthermore, the drop Y2 corresponds to the coincidence of the drop U1 part of the autocorrelation function C1 (k) and the drop V1 part of the cross-correlation function C2 (k), and the drop U2 part of the autocorrelation function C1 (k) and the cross-correlation function C2 ( k) is a drop due to the coincidence of the V2 portion, and the correlation value S2 is smaller than the correlation values S1 and S3 of the other drops Y1 and Y2.

  Therefore, if a drop shift value h1 indicating the smallest correlation value is obtained in the correlation amount C3 (h) graph shown in FIG. 20C, the value h1 is substantially equal to the shift value indicating the true correlation. The shift value closest to the shift value h1 obtained in this way is selected from the correlation function C2 (k) shown in FIG. 20B, and is set as the true shift value (shift value k1 shown in FIG. 20B). .

  The processing of Step 340 and Step 350 is the same as the processing of Step 340 and Step 350 of FIG.

  By performing further correlation calculation between the autocorrelation graph and the cross-correlation graph as described above, it is possible to select a drop corresponding to the true correlation even when there are two or more drops in the cross-correlation graph. become.

  As described above, according to another embodiment, the first data string An in which a plurality of data is arranged in one dimension and the second data string Bn in which a plurality of data are arranged in one dimension are: The first data string An and the second data string Bn are relatively displaced while changing the amount of displacement in one dimension, and the cross-correlation amount between the first data string An and the second data string Bn is calculated to obtain a mutual relationship. A cross-correlation operation for calculating a first displacement amount that obtains an extreme value of correlation amount (decrease in cross-correlation amount) and a relative relationship between the first data sequence An and the first data sequence An while changing the displacement amount in one dimension. Autocorrelation calculation for calculating a second displacement amount for which an autocorrelation amount between the first data strings An is calculated to obtain an extreme value of the autocorrelation amount (a drop in the autocorrelation amount), and an autocorrelation amount The amount of displacement that gives the extreme value of the correlation amount is calculated by calculating the correlation amount between the value and the cross-correlation amount. Since the first displacement amount closest to the amount is selected as the displacement amount that gives the true extreme value of the cross-correlation amount, a pair of first data strings corresponding to a pair of object images that are relatively distorted, and Even when using a correlation operation that can detect the amount of displacement relative to the second data string, eliminate the amount of displacement that gives a false extreme value of the cross-correlation amount (the amount of displacement that gives a false cross-correlation). The amount of displacement that gives the true extreme value of the cross-correlation amount (the amount of displacement that gives the true cross-correlation) can be accurately calculated.

<< Modification of Embodiment >>
The arrangement of the focus detection areas in the image sensor is not limited to the arrangement shown in FIG. 2, and the focus detection areas can be arranged in the diagonal direction and in the horizontal and vertical directions at other positions.

  In the imaging device 212 shown in FIG. 3, the focus detection pixels 313 and 314 are provided with one photoelectric conversion unit in one pixel. However, a pair of photoelectric conversion units is provided in one pixel. May be. FIG. 21 is a partially enlarged view of the image sensor 212A corresponding to the image sensor 212 shown in FIG. 3, and the focus detection pixel 311 includes a pair of photoelectric conversion units in one pixel. The focus detection pixel 311 shown in the drawing performs a function corresponding to the pair of the focus detection pixel 313 and the focus detection pixel 314 shown in FIG.

  The focus detection pixel 311 includes a microlens 10 and a pair of photoelectric conversion units 13 and 14 as shown in FIG. The focus detection pixel 311 is not provided with a color filter in order to increase the amount of light, and its spectral characteristics are a combination of the spectral sensitivity of a photodiode that performs photoelectric conversion and the spectral characteristics of an infrared cut filter (not shown). Spectral characteristics (see FIG. 7). That is, the spectral characteristics are obtained by adding the spectral characteristics of the green pixel, the red pixel, and the blue pixel shown in FIG. 6, and the light wavelength region of the sensitivity includes the light wavelength regions of the sensitivity of the green pixel, the red pixel, and the blue pixel. ing.

  FIG. 23 is a diagram for explaining the focus detection operation of the pupil division method by the focus detection pixels of the image sensor 212A shown in FIG. In the figure, reference numeral 90 denotes an exit pupil set at a distance d in front of the microlens arranged on the planned imaging plane of the interchangeable lens. Here, the distance d is a distance determined according to the curvature and refractive index of the microlens, the distance between the microlens and the photoelectric conversion unit, and is hereinafter referred to as a distance measuring pupil distance. 91 is an optical axis of the interchangeable lens, 50 and 60 are microlenses, (53,54) and (63,64) are photoelectric conversion units of a pair of focus detection pixels, (73,74) and (83,84) are It is a light beam for focus detection.

  Reference numeral 93 denotes an area of the photoelectric conversion units 53 and 63 projected by the microlenses 50 and 60, and is hereinafter referred to as a distance measuring pupil. Similarly, 94 is an area of the photoelectric conversion units 54 and 64 projected by the microlenses 50 and 60, and is hereinafter referred to as a distance measuring pupil. In FIG. 23, a focus detection pixel (comprising a microlens 50 and a pair of photoelectric conversion units 53 and 54) on the optical axis 91 and an adjacent focus detection pixel (a microlens 60 and a pair of photoelectric conversion units 63, 54). 64). In the focus detection pixels arranged in the periphery on the imaging surface, the pair of photoelectric conversion units are connected to the microlenses from the pair of distance measuring pupils 93 and 94, respectively. Receives incoming light flux. The arrangement direction of the focus detection pixels is made to coincide with the arrangement direction of the pair of distance measurement pupils.

  The microlenses 50 and 60 are disposed in the vicinity of the planned imaging plane of the optical system, and the shape of the pair of photoelectric conversion units 53 and 54 disposed behind the microlens 50 disposed on the optical axis 91 is formed. Projection is performed on the exit pupil 90 separated from the microlenses 50 and 60 by the distance measurement pupil distance d, and the projection shape forms distance measurement pupils 93 and 94. Further, the shape of the pair of photoelectric conversion units 63 and 64 disposed behind the microlens 60 disposed adjacent to the microlens 50 is projected onto the exit pupil 90 separated by the distance measuring pupil distance d. The projection shape forms distance measuring pupils 93 and 94. That is, the microlens of each pixel and the photoelectric conversion unit are arranged so that the projection shapes (ranging pupils 93 and 94) of the focus detection pixels coincide on the exit pupil 90 at the distance measurement pupil distance d. The positional relationship is determined.

  The photoelectric conversion unit 53 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 73 passing through the distance measuring pupil 93 and traveling toward the microlens 50. Further, the photoelectric conversion unit 54 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 74 passing through the distance measuring pupil 94 and traveling toward the microlens 50. Similarly, the photoelectric conversion unit 63 outputs a signal corresponding to the intensity of the image formed on the microlens 60 by the focus detection light beam 83 passing through the distance measuring pupil 93 and traveling toward the microlens 60. The photoelectric conversion unit 64 outputs a signal corresponding to the intensity of the image formed on the microlens 60 by the focus detection light beam 84 that passes through the distance measuring pupil 94 and travels toward the microlens 60.

  A large number of such focus detection pixels are arranged in a straight line, and the output of the pair of photoelectric conversion units of each pixel is grouped into an output group corresponding to the distance measurement pupil 93 and the distance measurement pupil 94, whereby the distance measurement pupil 93 and Information on the intensity distribution of a pair of images formed on the focus detection pixel array by the focus detection light fluxes that pass through the distance measuring pupils 94 is obtained. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process), which will be described later, to this information, an image shift amount of a pair of images is detected by a so-called pupil division method. Further, by applying a predetermined conversion process to the image shift amount, the deviation of the current imaging plane (imaging plane at the focus detection position corresponding to the position of the microlens array on the planned imaging plane) with respect to the planned imaging plane (Defocus amount) is calculated.

  Next, in the imaging device 212 shown in FIG. 3, an example in which the imaging pixel 310 includes a Bayer color filter is shown, but the configuration and arrangement of the color filter are not limited to this, and the complementary color filter (green : G, yellow: Ye, magenta: Mg, cyan: Cy) may be employed. In the image sensor 212 shown in FIG. 3, an example in which the focus detection pixels 313 and 314 are not provided with a color filter is shown. However, one of the color filters of the same color as the image pickup pixel 310 (for example, a green filter) is used. The present invention can be applied even when it is provided.

  In the focus detection pixels 311, 313, and 314 shown in FIGS. 5 and 22 of the above-described embodiment, an example in which the shape of the photoelectric conversion unit is a semicircle or a rectangle is shown. The shape of the photoelectric conversion unit is not limited to these, and may be other shapes. For example, the shape of the photoelectric conversion unit of the focus detection pixel may be an ellipse or a polygon.

  Further, in the imaging device 212 shown in FIG. 3, the example in which the imaging pixels and the focus detection pixels are arranged in a dense square lattice arrangement is shown, but a dense hexagonal lattice arrangement may be used.

  In the above-described embodiment, the focus detection operation by the pupil division method using the microlens has been described. However, the present invention is not limited to the focus detection of such a method, and the polarizing element disclosed in Japanese Patent Laid-Open No. 2008-15157. The present invention is also applicable to a focus detection apparatus using a pupil division type phase difference detection method.

  In the above-described embodiment, the focus detection operation by the pupil division method using the microlens has been described. However, the present invention is not limited to such a focus detection method, but is used for focus detection by the re-imaging pupil division method. Is also applicable. Here, the focus detection operation of the re-imaging pupil division method will be described with reference to FIG. In FIG. 24, 191 is the optical axis of the interchangeable lens, 110 and 120 are condenser lenses, 111 and 121 are aperture masks, 112, 113, 122 and 123 are aperture openings, 114, 115, 124 and 125 are re-imaging lenses, Reference numerals 116 and 126 denote image sensors (CCD) for focus detection.

  Reference numerals 132, 133, 142, and 143 denote focus detection light beams, and 190 denotes an exit pupil set at a distance d5 in front of the planned imaging plane of the interchangeable lens. Here, the distance d5 is a distance determined according to the focal length of the condenser lenses 110 and 120 and the distance between the condenser lenses 110 and 120 and the aperture openings 112, 113, 122, and 123, and is measured below. This is called the pupillary distance. Reference numeral 192 denotes an area of the diaphragm apertures 112 and 122 projected by the condenser lenses 110 and 120, and hereinafter referred to as a distance measuring pupil. Similarly, 193 is a region of the aperture openings 113 and 123 projected by the condenser lenses 110 and 120, and is hereinafter referred to as a distance measuring pupil. The condenser lens 110, the diaphragm mask 111, the diaphragm apertures 112 and 113, the re-imaging lenses 114 and 115, and the image sensor 116 are a re-imaging pupil division method phase difference detection focus detection unit that performs focus detection at one position. Configure.

  FIG. 24 schematically illustrates a focus detection unit on the optical axis 191 and a focus detection unit outside the optical axis. By combining a plurality of focus detection units, it is possible to realize a focus detection apparatus that performs focus detection by pupil division phase difference detection of the re-imaging method at the three focus detection positions 101 to 103 shown in FIG.

  The focus detection unit including the condenser lens 110 is disposed between the condenser lens 110 disposed in the vicinity of the planned imaging plane of the interchangeable lens, the image sensor 116 disposed behind the condenser lens 110, and the condenser lens 110 and the image sensor 116. A pair of re-imaging lenses 114 and 115 for re-imaging the primary image formed in the vicinity of the planned imaging surface on the image sensor 116, and the vicinity of the pair of re-imaging lenses (front surface in the figure). The aperture mask 11 has a pair of aperture openings 112 and 113.

  The image sensor 116 is a line sensor in which a plurality of photoelectric conversion units are densely arranged along a straight line, and the arrangement direction of the photoelectric conversion units is made to coincide with the dividing direction of the pair of distance measuring pupils (= aperture aperture arrangement direction). . Information corresponding to the intensity distribution of the pair of images re-imaged on the image sensor 116 is output from the image sensor 116, and image shift detection calculation processing (correlation processing, phase difference detection described later) is performed on this information. By performing the processing, an image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method (re-imaging method). Further, the deviation (defocus amount) of the current imaging plane with respect to the planned imaging plane is calculated by multiplying the image shift amount by a predetermined conversion coefficient.

  The image sensor 116 is projected onto the planned imaging plane by the re-imaging lenses 114 and 115, and the detection accuracy of the defocus amount (image deviation amount) is determined by the detection pitch of the image deviation amount (in the case of the re-imaging method). This is determined by the arrangement pitch of the photoelectric conversion units projected on the planned imaging plane.

  The condenser lens 110 projects the aperture openings 112 and 113 of the aperture mask 111 as areas 192 and 193 on the exit pupil 190. Regions 192 and 193 are called distance measurement pupils. That is, a pair of images re-imaged on the image sensor 116 is formed by a light beam passing through the pair of distance measuring pupils 192 and 193 on the exit pupil 190. The light beams 132 and 133 that pass through the pair of distance measuring pupils 192 and 193 on the exit pupil 190 are referred to as focus detection light beams.

  Even in such a re-imaging pupil division method, the balance of the pair of images formed on the image sensor is lost due to vignetting of the distance measuring pupil, so the present invention is applied when processing the output signal of the image sensor. can do.

  Note that the imaging apparatus is not limited to a digital still camera or a film still camera in which an interchangeable lens is mounted on the camera body as described above. For example, the present invention can be applied to a lens-integrated digital still camera, film still camera, or video camera. Furthermore, the present invention can be applied to a small camera module built in a mobile phone, a surveillance camera, a visual recognition device for a robot, an in-vehicle camera, and the like.

  The present invention can also be applied to focus detection devices other than cameras, distance measuring devices, and stereo distance measuring devices. Furthermore, the present invention can also be applied to an apparatus that detects the movement of a subject image and camera shake by detecting the correlation between signals from image sensors having different times. Furthermore, the present invention can be applied to pattern matching between an image signal of an image sensor and a specific image signal.

  Furthermore, the present invention is not limited to the one that detects the correlation between the image signal data, but can also be applied to the one that detects the correlation between the data related to sound and generally the correlation between two signals.

  In the above-described embodiments and their modifications, all combinations of the embodiments or the embodiments and the modifications are possible.

Cross-sectional view of the camera showing the configuration of the camera of one embodiment The figure which shows the focus detection position on the photographing screen of the interchangeable lens Front view showing detailed configuration of image sensor Front view of imaging pixels Front view of focus detection pixels Diagram showing spectral characteristics of imaging pixels The figure which shows the spectral characteristic of the pixel for focus detection Cross-sectional view of imaging pixels Cross section of focus detection pixel The figure which shows the structure of the focus detection optical system of the pupil division type phase difference detection method using a micro lens The flowchart which shows the imaging operation of the digital still camera (imaging device) of one embodiment The flowchart which shows the detail of the focus detection calculation process in step 130 of FIG. Diagram for explaining vignetting of focus detection beam Figure when viewing the distance measuring pupil plane in the direction of the optical axis from the planned focal plane 13 and 14, a pair of images received by the focus detection pixel column near the position x0 (image height 0) and a pair received by the focus detection pixel column near the position x1 (image height h). Showing the intensity distribution of the image (vertical axis is light intensity, horizontal axis is position on the shooting screen) Diagram for explaining a malfunction that calculates a high degree of correlation for two different data strings The figure for demonstrating the correlation calculation method of one Embodiment with respect to two mutually different data strings The figure for demonstrating the reliability determination method of a focus detection result The flowchart which shows other embodiment of the focus detection calculation process in step 130 of FIG. The figure for demonstrating the correlation calculation method of other embodiment with respect to two mutually different data strings Partial enlarged view of image sensor of modification FIG. 21 is a front view of focus detection pixels arranged in an image sensor of the modification shown in FIG. The figure for demonstrating the focus detection operation | movement of a pupil division system by the pixel for a focus detection of the image pick-up element of the modification shown in FIG. Diagram for explaining focus detection operation of re-imaging pupil division method

Explanation of symbols

201; Digital still camera, 202; Interchangeable lens, 212, 212A; Image sensor, 214; Body drive control device

Claims (7)

First electric signal data string output means for outputting a first electric signal data string composed of a plurality of electric signal data;
A second electric signal data string output means for outputting a second electric signal data string composed of a plurality of electric signal data;
The first electrical signal data sequence and the second electrical signal data sequence are displaced relative to each other while changing the displacement amount, so that the mutual relationship between the first electrical signal data sequence and the second electrical signal data sequence is achieved. the correlation quantity calculated by the first correlation operation, a cross-correlation calculation means for calculating a plurality of first displacement amount corresponding to a plurality of extreme values and the plurality of extreme values of the cross correlation amount,
The first electrical signal data sequence and the first electrical signal data sequence are relatively displaced while changing a displacement amount, and an autocorrelation amount between the first electrical signal data sequences is calculated as the first correlation calculation. autocorrelation calculating means for calculating calculates, and a plurality of second displacement amount corresponding to a plurality of extreme values and the plurality of extreme values of the autocorrelation by a second correlation operation which is different from,
A first displacement amount that gives a true extreme value of the cross-correlation amount is selected from the plurality of first displacement amounts based on the plurality of extreme values of the autocorrelation amount and the separation amounts of the plurality of second displacement amounts. A correlation calculating device. Electricity for receiving a pair of light beams passing through a pair of regions of the pupil of the imaging optical system and generating a first electric signal data string and a second electric signal data string corresponding to the pair of images formed by the pair of light beams Signal data string generation means;
The first electrical signal data sequence and the second electrical signal data sequence are displaced relative to each other while changing the displacement amount, so that the mutual relationship between the first electrical signal data sequence and the second electrical signal data sequence is achieved. the correlation quantity calculated by the first correlation operation, a cross-correlation calculation means for calculating a plurality of first displacement amount corresponding to a plurality of extreme values and the plurality of extreme values of the cross correlation amount,
The first electrical signal data sequence and the first electrical signal data sequence are relatively displaced while changing a displacement amount, and an autocorrelation amount between the first electrical signal data sequences is calculated as the first correlation calculation. autocorrelation calculating means for calculating calculates, and a plurality of second displacement amount corresponding to a plurality of extreme values and the plurality of extreme values of the autocorrelation by a second correlation operation which is different from,
A first displacement amount that gives a true extreme value of the cross-correlation amount is selected from the plurality of first displacement amounts based on the plurality of extreme values of the autocorrelation amount and the separation amounts of the plurality of second displacement amounts. Sorting means to
And a focus detection unit that detects a focus adjustment state of the imaging optical system based on the first displacement selected by the selection unit. The correlation calculation device according to claim 1,
The selecting means includes the number of the first displacement amounts calculated by the cross-correlation calculating means and the distance between the first displacement amounts, the number of the second displacement amounts calculated by the autocorrelation calculating means, and Based on the distance between the second displacement amounts, a relative relationship between the first displacement amount and the second displacement amount is calculated, and corresponds to a second displacement amount that gives a true extreme value of the autocorrelation amount. A correlation calculation device, wherein the first displacement amount is selected as a first displacement amount that gives a true extreme value of the cross-correlation amount. First electric signal data string output means for outputting a first electric signal data string composed of a plurality of electric signal data;
A second electric signal data string output means for outputting a second electric signal data string composed of a plurality of electric signal data;
The first electrical signal data sequence and the second electrical signal data sequence are displaced relative to each other while changing the displacement amount, so that the mutual relationship between the first electrical signal data sequence and the second electrical signal data sequence is achieved. the correlation quantity calculated by the first correlation operation, a cross-correlation calculation means for calculating a plurality of first displacement amount corresponding to a plurality of extreme values and the plurality of extreme values of the cross correlation amount,
The first electrical signal data sequence and the first electrical signal data sequence are relatively displaced while changing a displacement amount, and an autocorrelation amount between the first electrical signal data sequences is calculated as the first correlation calculation. Is an autocorrelation calculation means for calculating by a different second correlation calculation;
By calculating a correlation amount between the autocorrelation amount and the cross-correlation amount to calculate a second displacement amount that can obtain an extreme value of the correlation amount, the second displacement amount is calculated from the plurality of first displacement amounts. And a selecting means for selecting the first displacement amount closest to the first displacement amount as a first displacement amount that gives a true extreme value of the cross-correlation amount. Electricity for receiving a pair of light beams passing through a pair of regions of the pupil of the imaging optical system and generating a first electric signal data string and a second electric signal data string corresponding to the pair of images formed by the pair of light beams Signal data string generation means;
The first electrical signal data sequence and the second electrical signal data sequence are displaced relative to each other while changing the displacement amount, so that the mutual relationship between the first electrical signal data sequence and the second electrical signal data sequence is achieved. the correlation quantity calculated by the first correlation operation, a cross-correlation calculation means for calculating a plurality of first displacement amount corresponding to a plurality of extreme values and the plurality of extreme values of the cross correlation amount,
The first electrical signal data sequence and the first electrical signal data sequence are relatively displaced while changing a displacement amount, and an autocorrelation amount between the first electrical signal data sequences is calculated as the first correlation calculation. Is an autocorrelation calculation means for calculating by a different second correlation calculation;
By calculating a correlation amount between the autocorrelation amount and the cross-correlation amount to calculate a second displacement amount that can obtain an extreme value of the correlation amount, the second displacement amount is calculated from the plurality of first displacement amounts. Sorting means that sorts the first displacement amount closest to the first displacement amount as a first displacement amount that gives a true extreme value of the cross-correlation amount;
And a focus detection unit that detects a focus adjustment state of the imaging optical system based on the first displacement selected by the selection unit. In the correlation calculation device according to any one of claims 1, 3, and 4,
In the first correlation calculation, the first electric signal data string is a function F (x) of a one-dimensional position x, and the second electric signal data string is a function G (x) of the one-dimensional position x. If
G (x) = (1 + δx) · M · F (x) (δ and M are constants)
The correlation calculation device is characterized by being a correlation calculation capable of calculating a cross-correlation amount between the first electric signal data string and the second electric signal data string. An imaging apparatus comprising the focus detection apparatus according to claim 2.

Images