JP2014164258A - Focus detection device and amount of image deviation detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a highly accurate focus detection device.SOLUTION: The focus detection device comprises: an image sensor; image synthesizing means for generating a synthesized image signal string by mutually adding and synthesizing a pair of relatively shifted image signal strings; contrast extraction means for generating a contrast signal string by extracting a plurality of contrast components from the synthesized image signal string; contrast evaluation means for calculating a contrast evaluation value of the synthesized image signal string on the basis of a non-linear contrast signal string having the contrast signal string converted; image deviation amount detection means for detecting the amount of a shift corresponding to an extremal value in a plurality of contrast evaluation values as the amount of an image deviation; and defocus amount calculation means for calculating the amount of a defocus of an optical system.

Description

  The present invention relates to a pupil division type phase difference detection type focus detection device and an image shift amount detection device.

  A focus detection apparatus using a so-called pupil division type phase difference detection method is known. (For example, refer to Patent Document 1). In the focus detection apparatus, a pair of image signals corresponding to a pair of images formed by a pair of focus detection light fluxes passing through the exit pupil of the optical system is generated. A correlation value representing the degree of coincidence between the pair of relatively shifted image signal patterns is calculated by performing a known correlation operation by relatively shifting the pair of image signals. A shift amount that maximizes the degree of coincidence between the pair of image signal patterns that are relatively shifted based on the correlation value is detected as a relative image shift amount between the pair of subject images. At the same time, the focus adjustment state of the optical system is detected according to the image shift amount. The focus adjustment state of the optical system is represented by the difference between the planned focal plane and the detected image plane, that is, the defocus amount.

JP 2007-233302 A

  In the conventional focus detection device of the pupil division type phase difference detection method, the image shift amount is detected based on the matching degree of a pair of image patterns. Therefore, when the identity of a pair of image patterns, that is, the identity of an image waveform (image pattern) other than the image formation position is reduced due to the aberration of the optical system, an image shift detection error occurs, resulting in There was a problem that the focus detection accuracy was lowered.

(1) A focus detection apparatus according to a first aspect of the present invention includes an image sensor that photoelectrically converts an image formed by each of light beams that pass through a pair of regions of an exit pupil of an optical system, and generates a pair of image signal sequences; A composite image signal sequence composed of a plurality of composite image signals corresponding to each shift amount of a plurality of shift amounts is generated by adding and synthesizing a pair of image signal sequences that are relatively shifted corresponding to each shift amount. And generating a contrast signal sequence composed of a plurality of contrast components corresponding to each shift amount by extracting a plurality of contrast components from the composite image signal sequence through a linear combination operation on the plurality of composite image signals. Extraction means and nonlinear contrast in which a contrast signal sequence is converted by nonlinear conversion of multiple contrast components based on a nonlinear function A contrast evaluation means for calculating a contrast evaluation value of the composite image signal sequence corresponding to each shift amount based on the signal sequence; and a plurality of contrast evaluation values obtained by calculating the contrast evaluation value corresponding to each shift amount Image shift amount detection means for detecting the shift amount corresponding to the extreme value in the contrast evaluation value as a relative image shift amount of the image, and defocus for calculating the defocus amount of the optical system based on the image shift amount And a quantity calculating means.
(2) An image shift amount detection device according to an eighth aspect of the present invention includes an image signal sequence generation unit that generates a pair of image signal sequences obtained by discretely sampling a pair of images at a predetermined spatial pitch, A pair of image signal sequences that are relatively shifted in accordance with the shift amount are added together to generate a composite image signal sequence composed of a plurality of composite image signals corresponding to the shift amounts of the plurality of shift amounts. Contrast extraction that generates a contrast signal sequence composed of a plurality of contrast components corresponding to each shift amount by extracting a plurality of contrast components from the combined image signal sequence through linear combination operation on the plurality of combined image signals with an image combining means And a non-linear contrast signal sequence in which the contrast signal sequence is converted by non-linear conversion of a plurality of contrast components based on a non-linear function. Accordingly, contrast evaluation means for calculating the contrast evaluation value of the composite image signal sequence corresponding to each shift amount, and a plurality of contrast evaluation values obtained by calculating the contrast evaluation value corresponding to each shift amount And an image shift amount detecting means for detecting a shift amount corresponding to the extreme value of the image as a relative image shift amount of the pair of images.

  ADVANTAGE OF THE INVENTION According to this invention, a highly accurate focus detection apparatus can be provided.

It is a cross-sectional view which shows the structure of a digital camera. It is a figure which shows the focus detection position on an imaging | photography screen. It is a front view which shows the detailed structure of an image pick-up element. It is a figure for demonstrating the mode of the imaging light beam which an imaging pixel receives. It is a figure for demonstrating the mode of the focus detection light beam which a focus detection pixel receives. It is a flowchart which shows the operation | movement by the body control apparatus of a digital camera. It is a flowchart detailing an image shift detection calculation process based on contrast evaluation for a pair of image signal sequences. It is the figure which showed typically how an imaging light beam and a pair of focus detection light beams converge in the vicinity of a planned focal plane. It is a figure which shows the example of the point image distribution (point image distribution function) of the point image which an imaging light beam forms on a plan focal plane. It is a figure which shows the example of a pair of point image distribution (point image distribution function) of the point image which a pair of focus detection light beam forms on a plan focal plane. FIG. 6 is a diagram illustrating a subject image when a monochrome edge subject is formed on the best image plane with a pair of focus detection light beams when the photographing optical system has no aberration. It is a figure which shows the to-be-photographed image formed in the best image surface by a pair of focus detection light beam when the imaging optical system with a big aberration is used. FIG. 6 is a diagram in which a pair of point image distributions are relatively displaced and overlapped by an image shift amount. It is a figure for demonstrating the detection principle of the image shift amount detection based on contrast evaluation. It is a figure showing the relationship between the sampling position based on a focus detection pixel pitch, and a composite image signal intensity | strength (composite image signal sequence). It is the figure which showed the image signal intensity | strength of a pair of image signal sequence in the case of the step-like pattern of level | step difference C1 as a pair of images, and those synthetic | combination image signal sequences. It is the figure which showed the image signal intensity | strength of a pair of image signal sequence in the case of the step-like pattern of level | step difference C1 as a pair of images, and those synthetic | combination image signal sequences. It is a figure which shows contrast evaluation value C (k) with respect to the shift amount k. It is a figure explaining the interpolation method (three-point interpolation) of the shift amount G. It is a figure which shows the example of nonlinear function H (x). It is a figure which shows the example of nonlinear function H (x). It is a figure which shows the example of nonlinear function H (x). It is a figure which shows the example of nonlinear function H (x). It is a figure which shows the example of nonlinear function H (x). It is a figure which shows the example of nonlinear function H (x). It is a figure which shows the example of nonlinear function H (x). It is a figure which shows the example of nonlinear function H (x).

  The imaging apparatus including the focus detection apparatus according to the first embodiment of the present invention will be described by taking an interchangeable lens digital camera as an example. FIG. 1 is a cross-sectional view illustrating a configuration of a digital camera 201 according to the present embodiment. A digital camera 201 according to the present embodiment includes an interchangeable lens 202 and a camera body 203, and the interchangeable lens 202 is attached to the camera body 203 via a mount unit 204. An interchangeable lens 202 having various photographing optical systems can be attached to the camera body 203 via a mount unit 204.

  The interchangeable lens 202 includes a lens 209, a zooming lens 208, a focusing lens 210, a diaphragm 211, a lens control device 206, and the like. The lens control device 206 includes a microcomputer (not shown), a memory, a lens drive control circuit, and the like. The lens control device 206 performs drive control for adjusting the focus of the focusing lens 210 and adjusting the aperture diameter of the aperture 211, and detecting the states of the zooming lens 208, the focusing lens 210, and the aperture 211. The lens control device 206 transmits lens information and receives camera information (defocus amount, aperture value, etc.) through communication with a body control device 214 described later. The aperture 211 forms an aperture having a variable aperture diameter at the center of the optical axis in order to adjust the amount of light and the amount of blur.

  The camera body 203 includes an imaging element 212, a body control device 214, a liquid crystal display element driving circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, an AD converter 221 and the like. In the imaging device 212, imaging pixels are arranged according to a two-dimensional array defined by rows and columns, and focus detection pixels are arranged at portions corresponding to the focus detection positions. Details of the image sensor 212 will be described later.

  The body control device 214 includes a microcomputer, a memory, a body drive control circuit, and the like. The body control device 214 repeatedly performs exposure control of the image sensor 212, readout of the pixel signal from the image sensor 212, focus detection calculation based on the pixel signal of the focus detection pixel, and focus adjustment of the interchangeable lens 202, and an image. It performs signal processing, display and recording, and camera operation control. The body control device 214 communicates with the lens control device 206 via the electrical contact 213 to receive lens information and transmit camera information.

  The liquid crystal display element 216 functions as an electronic view finder (EVF). The liquid crystal display element driving circuit 215 displays a through image on the liquid crystal display element 216 based on the image signal read from the image sensor 212, and the photographer can observe the through image through the eyepiece 217. The memory card 219 is an image storage that stores image data generated based on an image signal captured by the image sensor 212.

  The AD conversion device 221 performs AD conversion on the pixel signal output from the image sensor 212 and sends it to the body control device 214. The imaging device 212 may have a configuration in which the AD conversion device 221 is incorporated.

  A subject image is formed on the imaging surface of the imaging element 212 by the light beam that has passed through the interchangeable lens 202. This subject image is photoelectrically converted by the image sensor 212, and the pixel signals of the imaging pixels and focus detection pixels are sent to the body control device 214.

  The body control device 214 calculates the defocus amount based on the pixel signal (focus detection signal) from the focus detection pixel of the image sensor 212 and sends this defocus amount to the lens control device 206. Further, the body control device 214 processes the pixel signal (imaging signal) of the imaging pixel of the imaging element 212 to generate image data, stores the image data in the memory card 219, and reads the through image signal read from the imaging element 212. Is sent to the liquid crystal display element driving circuit 215, and a through image is displayed on the liquid crystal display element 216. Further, the body control device 214 sends aperture control information to the lens control device 206 to control the aperture of the aperture 211.

  The lens control device 206 updates the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the positions of the zooming lens 208 and the focusing lens 210 and the aperture value of the aperture 211 are detected, and lens information is calculated according to these lens positions and aperture values, or a lookup prepared in advance. Lens information corresponding to the lens position and aperture value is selected from the table.

  The lens control device 206 calculates a lens driving amount based on the received defocus amount, and drives the focusing lens 210 to the in-focus position according to the lens driving amount. In addition, the lens control device 206 drives the diaphragm 211 in accordance with the received diaphragm value.

  FIG. 2 is a diagram showing a focus detection position on the shooting screen, and an area for sampling an image on the shooting screen (focus detection area, focus detection) when focus detection is performed by a focus detection pixel array on the image sensor 212 described later. An example of (position) is shown. In this example, the focus detection area 101 is arranged at the center (on the optical axis) on the rectangular shooting screen 100. Focus detection pixels are linearly arranged in the longitudinal direction (horizontal direction) of the focus detection area 101 indicated by a rectangle.

  FIG. 3 is a front view showing a detailed configuration of the image sensor 212, and shows details of a pixel array in which the vicinity of the focus detection area 101 arranged in the horizontal direction in FIG. 2 is enlarged. Imaging pixels 310 are densely arranged on the imaging element 212 in a two-dimensional square lattice pattern. The imaging pixel 310 includes a red pixel (R), a green pixel (G), and a blue pixel (B), and is arranged according to a Bayer arrangement rule. In FIG. 3, focus detection pixels 315 and 316 for horizontal focus detection having the same pixel size as that of the imaging pixel 310 are alternately arranged on a straight line in the horizontal direction in which green pixels and blue pixels are supposed to be continuously arranged. Are arranged in succession.

  The shape of each microlens of the imaging pixel 310 and the focus detection pixels 315 and 316 is a shape obtained by cutting out a circular microlens originally larger than the pixel size into a square shape corresponding to the pixel size.

  As shown in FIG. 3, the imaging pixel 310 includes a rectangular microlens 10, a photoelectric conversion unit 11 in which a light receiving region is limited to a square by a light shielding mask, and a color filter. The color filters include three types of red (R), green (G), and blue (B), and have spectral sensitivity characteristics corresponding to the respective colors. In the image pickup device 212, image pickup pixels 310 having respective color filters are arranged in a Bayer array.

  The focus detection pixels 315 and 316 are provided with white filters that transmit all visible light in order to perform focus detection for all colors. The white filter has a spectral sensitivity characteristic such that the spectral sensitivity characteristics of the green pixel, the red pixel, and the blue pixel are added, and the light wavelength region exhibiting high sensitivity is each color filter in each of the green pixel, the red pixel, and the blue pixel. Includes a light wavelength region exhibiting high sensitivity.

  As shown in FIG. 3, the focus detection pixel 315 is a photoelectric conversion in which a light receiving region is limited to a left half of a square (a left half when a square is divided into two equal parts by a vertical line) using a rectangular microlens 10 and a light shielding mask. And a white filter (not shown).

  Further, as shown in FIG. 3, the focus detection pixel 316 is limited to the right half of the square (right half when the square is divided into two equal parts by a vertical line) by the rectangular microlens 10 and the light shielding mask. It comprises a photoelectric conversion unit 16 and a white filter (not shown).

  When the focus detection pixel 315 and the focus detection pixel 316 are displayed with the microlens 10 superimposed, the photoelectric conversion units 15 and 16 in which the light receiving area is limited to a half of a square by a light shielding mask are arranged in the horizontal direction.

  Further, when the remaining portion obtained by halving the square is added to the portion of the light receiving region limited to the half of the square described above, a square having the same size as the light receiving region of the imaging pixel 310 is obtained.

  In the configuration of the imaging pixel and the focus detection pixel as described above, under a general light source, the output level of the green imaging pixel and the output level of the focus detection pixel are substantially equal, and the red imaging pixel and the blue color are detected. The output level of the image pickup pixel becomes smaller than this.

  FIG. 4 is a diagram for explaining the state of the imaging light beam received by the imaging pixel 310 shown in FIG. 3, and takes a cross section of the imaging pixel array arranged in the horizontal direction. The photoelectric conversion units 11 of all the imaging pixels 310 arranged on the image sensor 212 receive the light flux that has passed through the openings of the light shielding masks arranged close to the photoelectric conversion unit 11. The shape of the light-shielding mask opening is projected by the microlens 10 of each imaging pixel 310 onto a region 97 common to all imaging pixels on the exit pupil 90 of the imaging optical system that is separated from the microlens 10 by the distance measuring pupil distance d. .

  Therefore, the photoelectric conversion unit 11 of each imaging pixel receives the light beam 71 passing through the region 97 and the microlens 10 of each imaging pixel, and a signal corresponding to the intensity of the image formed on each microlens 10 by the light beam 71. Is output.

  FIG. 5 is a view for explaining the state of the focus detection light beam received by the focus detection pixels 315 and 316 shown in FIG. 3 in comparison with FIG. 4, and is a cross-section of the focus detection pixel array arranged in the horizontal direction. Have taken.

  The photoelectric conversion units 15 and 16 of all the focus detection pixels 315 and 316 arranged on the image sensor 212 receive the light beam that has passed through the opening of the light shielding mask disposed in proximity to each of the photoelectric conversion units 15 and 16. To do. The shape of the light-shielding mask opening arranged close to the photoelectric conversion unit 15 is such that the focus detection pixel on the exit pupil 90 separated from the microlens 10 by the distance measurement pupil distance d by the microlens 10 of each focus detection pixel 315. 315 is projected onto a common area 95. Similarly, the shape of the light-shielding mask opening arranged close to the photoelectric conversion unit 16 is such that the focus detection on the exit pupil 90 separated from the microlens 10 by the distance measurement pupil distance d by the microlens 10 of each focus detection pixel 316. The image is projected onto an area 96 common to all the pixels 316. The pair of regions 95 and 96 is called a distance measuring pupil.

  Accordingly, the photoelectric conversion unit 15 of each focus detection pixel 315 receives the light beam 75 passing through the distance measuring pupil 95 and the microlens 10 of each focus detection pixel 315 and is formed on each microlens 10 by the light beam 75. A signal corresponding to the intensity of the image is output. The photoelectric conversion unit 16 of each focus detection pixel 316 receives a light beam 76 passing through the distance measuring pupil 96 and the microlens 16 of each focus detection pixel 316 and is formed on each microlens 10 by the light beam 76. A signal corresponding to the intensity of the image is output.

  A region where the distance measuring pupils 95 and 96 on the exit pupil 90 through which the light beams 75 and 76 received by the pair of focus detection pixels 315 and 316 pass is integrated on the exit pupil 90 through which the light beam 71 received by the imaging pixel 310 passes. The pair of light beams 75 and 76 are complementary to the light beam 71 on the exit pupil 90.

  In the above description, the light receiving area of the photoelectric conversion unit is regulated by the light shielding mask, but the shape of the photoelectric conversion unit itself may be the opening shape of the light shielding mask. In that case, the shading mask may be eliminated.

  In short, it is important that the photoelectric conversion unit and the distance measuring pupil have an optically conjugate relationship by the microlens.

  Further, the position of the distance measuring pupil (the distance measuring pupil distance) is generally set to be substantially the same as the exit pupil distance of the photographing optical system. When a plurality of interchangeable lenses are mounted, the distance measuring pupil distance is set to the average exit pupil distance of the plurality of interchangeable lenses.

  A large number of the pair of focus detection pixels 315 and 316 are arranged alternately and linearly, and the output of the photoelectric conversion unit of each focus detection pixel is collected into a pair of output groups corresponding to the distance measurement pupil 95 and the distance measurement pupil 96. . As a result, information (a pair of image signal sequences) relating to the intensity distribution of the pair of images formed on the horizontal focus detection pixel array by the pair of light beams passing through the distance measuring pupil 95 and the distance measuring pupil 96 is obtained. By applying an image shift detection calculation process (phase difference detection process), which will be described later, to this information (a pair of image signal sequences), an image shift amount of the pair of images is detected. Further, by performing a conversion operation using a conversion coefficient corresponding to the proportional relationship between the distance between the center of gravity of the pair of distance measurement pupils and the distance measurement pupil distance to the image shift amount, the planned imaging plane and pupil at the focus detection position are calculated. The deviation from the image plane detected by the division type phase difference detection method, that is, the defocus amount is calculated.

  In FIG. 5, for easy understanding, the pair of distance measuring pupils 95 and 96 are shown in a clear shape, and the pair of focus detection light beams 75 and 76 are expressed in a cone shape, and a cross section perpendicular to the optical axis 91. When the light beam is cut out, the light density is explained as if it were uniform on the cross section. However, in practice, the outer shapes of the pair of distance measuring pupils 95 and 96 are unclear depending on the aberration of the micro lens of the focus detection pixel. Further, the light density of the pair of focus detection light beams 75 and 76 in a cross section perpendicular to the optical axis 91 is not uniform, and has a distribution according to the optical characteristics of the focus detection optical system and the optical characteristics of the photographing optical system. Show.

  FIG. 6 is a flowchart showing an operation performed by the body control device 214 of the digital camera 201 of the present embodiment. The body control device 214 starts operation from step S110 when the power of the digital camera 201 is turned on in step S100. If it is necessary to change the aperture in step S110, the body control device 214 sends an aperture adjustment command to the lens control device 206 to cause the aperture adjustment. At the same time, the body control device 214 performs an imaging operation to read out the signal of the imaging pixel 310 from the imaging element 212 and display it on the liquid crystal display element 216. In subsequent step S120, the body control device 214 reads a pair of image signal sequences corresponding to the pair of subject images from the focus detection pixel sequence.

  In step S <b> 130, the body control device 214 performs an image shift detection calculation process based on contrast evaluation described later on the read pair of image signal sequences to calculate an image shift amount of the pair of image signal sequences, and the image shift. The amount is converted into a defocus amount, and the process proceeds to step S140. The image shift detection calculation processing based on contrast evaluation is an image in which a pair of image signal sequences are relatively shifted and added and synthesized to generate a synthesized image signal sequence and the contrast of the synthesized image signal sequence is maximized. This is a process for calculating the amount of deviation.

  In step S140, the body control device 214 determines whether or not the focus adjustment state of the photographic optical system is an in-focus state, that is, whether or not the calculated absolute value of the defocus amount is within a predetermined value. The predetermined value is determined to be 100 μm, for example, by experiment. When the body control device 214 determines that the focus adjustment state of the photographing optical system is not the in-focus state, the process proceeds to step S150. In step S150, the body control device 214 transmits the calculated defocus amount to the lens control device 206, and drives the focusing lens 210 of the interchangeable lens 202 shown in FIG. 1 to the in-focus position. Thereafter, the process returns to step S110 to repeat the above-described operation.

  Even when focus detection is impossible, the process branches to step S150, and the body control device 214 transmits a scan drive command to the lens control device 206, and moves the focusing lens 210 of the interchangeable lens 202 from the infinity position to the closest position. Drive to scan. Thereafter, the process returns to step S110 and the above-described operation is repeated.

  On the other hand, if it is determined in step S140 that the focus adjustment state of the photographing optical system is the in-focus state, the process proceeds to step S160. In step S160, the body control device 214 determines whether or not a shutter release has been performed by operating a shutter button (not shown). If it is determined that the shutter release has not been performed, the process returns to step S110 and the operation described above. Is repeated. In step S160, if the body control device 214 determines that the shutter release has been performed, in step S170, the body control device 214 causes the imaging device 212 to perform an imaging operation, and reads signals from the imaging pixels of the imaging device 212 and all focus detection pixels. .

  In step S180, the body control device 214 performs pixel interpolation on the imaging signal at each pixel position in the focus detection pixel row based on the signals of the imaging pixels around the focus detection pixel. In subsequent step S190, the body control device 214 stores the image data composed of the image pickup pixel signal and the interpolated signal in the memory card 219, and the process returns to step S110 and the above-described operation is repeated.

Prior to detailed description of the image shift detection calculation process (FIG. 7) based on contrast evaluation for a pair of image signal sequences in step S130 of FIG. 6, problems of the prior art will be described. First, a conventional image shift detection calculation process that performs image shift detection based on the degree of coincidence between a pair of image signal sequences will be described. In the conventional image shift detection calculation processing based on the matching degree of a pair of image signal sequences, a pair of image signal sequences A _{1 to} A _M and B _{1 to} B _M read from the focus detection pixel sequence (number of pixels 2M). On the other hand, a well-known correlation calculation (SAD: Sum of Absolute Difference) as shown in the following equation (1) is performed to calculate a correlation amount E (k) representing the degree of coincidence between a pair of image signal sequence patterns.
E (k) = Σ | A _n −B _{n + k} | (1)

In equation (1), the Σ operation is accumulated for variable n. The range of the variable n is limited to a range in which the image signal sequences A _n and B _{n + k} exist according to the image shift amount k. The image shift amount k is an integer and is a relative shift amount in units of the signal pitch of the pair of image signal sequences. The amount of correlation E (k) with respect to a plurality of shift amounts k is obtained by shifting the pair of image signal sequences by a predetermined amount relative to the calculation of equation (1), that is, by changing the image shift amount k within a predetermined range. Is calculated. In the expression (1), the value of the pair of image signal sequences has a smaller value of the correlation amount E (k) as the degree of coincidence is higher. Therefore, the correlation amount E (k) obtained with respect to a plurality of shift amounts k. The shift amount giving the minimum value is set as the image shift amount. The correlation calculation expression for detecting the degree of coincidence between the pair of image signal sequences is not limited to the expression (1), and any expression may be used as long as the degree of coincidence between the pair of image signal strings is calculated.

  The principle that the degree of coincidence between a pair of image signal sequences can be detected by the above-described conventional image shift detection calculation is based on the following concept. If the premise that the shape and waveform of the pair of subject image signals formed by the pair of focus detection light beams is the same, that is, if they match, the pattern of the pair of image signal trains is perfectly positioned at the time of focusing. And overlap. Therefore, if the premise that the pattern of the pair of image signal sequences formed by the pair of focus detection light beams is broken, the focus adjustment detected according to the image shift amount calculated by the conventional image shift detection calculation is made accordingly. The condition will cause an error.

  FIG. 8 shows a case where the best image plane is formed on the planned focal plane 98 and passes through a pair of areas 95 and 96 of the imaging light flux and the exit pupil passing through the exit pupil area 97 shown in FIGS. 4 and 5, respectively. It is the figure which showed typically how a pair of focus detection light beam which converges converges in the plan focal plane 98 vicinity. The best image plane is an imaging plane that has the highest image quality, such as sharpness or contrast, of the subject image formed by the photographing light flux and the pair of subject images formed by the pair of focus detection light fluxes. For example, in FIG. 8, if there is a point light source on the optical axis 91, a point image is formed on the optical axis 91 of the planned focal plane 98 corresponding to the point light source.

  In the case of an ideal non-aberration imaging optical system, a point image formed by a photographing light beam passing through the exit pupil region 97 is also a pair formed by a pair of focus detection light beams passing through the pair of exit pupil regions 95 and 96. Each of the point images is a complete point that does not spatially spread on the planned focal plane 98, and a pair of points formed by a pair of focus detection light beams that pass through the pair of regions 95 and 96 of the exit pupil. The spatial position of the image on the planned focal plane 98 also coincides. When a general subject is photographed using such an aberration-free photographing optical system, the shape of the pair of subject images formed on the best image plane by the pair of focus detection light beams completely coincides with each other. Since the positions of the pair of subject images also coincide, it is said that the image is in focus when the image displacement amount of the pair of subject images obtained by the image displacement detection calculation process based on the degree of coincidence of the pair of conventional image signal sequences is zero. Can be guaranteed.

  However, when the photographing optical system has optical aberration, a point image formed by the photographing light flux passing through the exit pupil region 97 is also formed by a pair of focus detection light fluxes passing through the pair of exit pupil regions 95 and 96. Each of the pair of point images is also a point image having a spatial spread on the planned focal plane 98.

  FIG. 9 shows the point image distribution 51 (point image distribution function) of the point image formed by the photographing light beam on the planned focal plane 98 in the state shown in FIG. 8, that is, in the state where the best image plane is formed on the planned focal plane 98. ), Which has a large peak at the center and a symmetric bottom at the periphery. On the other hand, FIG. 10 shows a pair of point image distributions (point image distributions) of point images formed by the pair of focus detection light beams on the planned focal plane 98 in the same state, that is, in a state where the best image plane is formed on the planned focal plane 98. The solid line represents the point image distribution 55 formed by the focus detection light beam passing through the region 95, and the broken line represents the point image distribution 56 formed by the focus detection light beam passing through the region 96. 9 and 10, the horizontal axis represents the horizontal position on the planned focal plane 98, and the vertical axis represents the image intensity. The peak positions of the point image distributions 51, 55, and 56 are the center of the image plane, that is, the position at which the optical axis 91 intersects the planned focal plane 98.

  Similar to the point image distribution 51, the point image distributions 55 and 56 have a large peak at the center and have a base at the periphery, but the bases are both asymmetric. While the right foot of the point image distribution 55 is large, there is almost no left foot. The left foot of the point image distribution 56 is large, while the right foot is scarce. Further, the pair of focus detection light beams has a complementary relationship with the photographic light beam, and a combination of the pair of focus detection light beams becomes a photographic light beam. Therefore, a pair represented by a point image distribution 55 and a point image distribution 56 is used. A point image distribution 51 is obtained by adding and synthesizing the subject image signals. As shown in FIGS. 9 and 10, when the point image is formed on the optical axis, the shape of the point image distribution 51 is left-right symmetric, and one of the point image distribution 55 and the point image distribution 56 is left and right. The shape matches when reversed. When the point image is on the periphery of the screen outside the optical axis, the shape of the point image distributions 51, 55, and 56 depends on the position shown in FIG. 9 and FIG. Further, since the shape of the point image distribution 51 is not symmetrical, the shape of the point image distribution 55 and the point image distribution 56 do not match even when one of the point image distribution 55 and the point image distribution 56 is reversed left and right.

  In general, when the amount of aberration of the photographic optical system is small or good, in the shape of the point image distributions 51, 55, and 56 on the best image plane, the spread of the skirt portion is small compared to the size of the peak portion, and a pair of The point image distributions 55 and 56 have almost the same shape, and the positions of the pair of point images are almost the same.

  In general, the distribution function of the image signal of the subject image formed by the imaging optical system having aberration is formed by the imaging optical system having aberration in the distribution function of the image signal of the subject image formed when there is no aberration. This is a convolution of the distribution function of the image signal of the point image.

  Therefore, when the amount of aberration is small, or when a general subject is photographed using a good photographing optical system, the shape of the pair of subject images formed on the best image plane by the pair of focus detection light beams is almost the same. At the same time, the positions of the pair of subject images also coincide. Therefore, the focus detection is performed according to the image shift amount calculated by the conventional image shift detection calculation process based on the premise that the image shift amount with the highest matching degree between the pair of subject images is 0. Does not cause a large error.

  However, when photographing a general subject using a photographing optical system having a large amount of aberration, the shape of the pair of subject images formed on the best image plane by the pair of focus detection light beams does not match. Therefore, if the focus detection calculated by the conventional image shift detection calculation process based on the premise that the image shift amount with the highest matching degree between the pair of subject images is 0, a large error occurs. .

  FIG. 11 shows a subject image when a black-and-white edge subject is formed on the best image plane with a pair of focus detection light beams when the photographing optical system has no aberration, and the focus detection that passes through the region 95 in FIG. The light beam forms a subject image 65, and the focus detection light beam passing through the region 96 forms a subject image 66. The position of the edge portion 45 of the subject image 65 and the position of the edge portion 46 of the subject image 66 coincide with each other. In such a case, an image shift detection calculation based on the degree of coincidence of a pair of subject image signals may be used. The image shift amount is calculated as 0.

  On the other hand, FIG. 12 shows a subject image formed on the best image plane by a pair of focus detection light beams when a photographing optical system with large aberration is used for a subject with the same black and white edge as FIG. Further, it is assumed that the distribution of a pair of point images formed on the best image plane by the pair of focus detection light beams that have passed through the photographing optical system is represented by point image distribution functions 55 and 56 shown in FIG. The subject image 67 is an edge image formed by the focus detection light beam passing through the region 95, and is an image obtained by convolving the point image distribution function 55 with the subject image 65 in the case of no aberration. The subject image 68 is an edge image formed by a focus detection light beam passing through the region 96, and is an image obtained by convolving the point image distribution function 56 with the subject image 66 in the case of no aberration.

  The pair of subject images 67 and 68 are originally the same subject image, but the pair of point image distributions formed by the pair of focus detection light beams are not the same, so that the shape of the pair of subject images, that is, the pair of subject image signals. The patterns are greatly different from each other. For example, the shape of the upper portion 41 of the edge portion 47 of the subject image 67 and the shape of the upper portion 42 of the edge portion 48 of the subject image 68 are greatly different. The shape of the lower portion 43 of the edge portion 47 of the subject image 67 and the shape of the lower portion 44 of the edge portion 48 of the subject image 68 are greatly different. In the state where the best image plane coincides with the planned focal plane, the detected image shift amount does not become zero even when the image shift detection is performed on the pair of subject images 67 and 68 having different shapes. For example, in this state, when the image shift amount Δ (Δ ≠ 0) is calculated by the image shift detection calculation based on the matching degree of the pair of subject image signals, the image plane corresponding to the image shift amount Δ is, for example, FIG. It becomes the surface 99.

  The cause of such an error (image shift amount Δ) is that the pair of point image distributions formed by the pair of focus detection light beams on the best image plane is not the same as described above. Although the peak positions of the point image distributions 55 and 56 in FIG. 10 coincide with each other on the best image plane, image deviation detection calculation processing based on the degree of coincidence of a pair of subject image signal patterns is performed on the point image distributions 55 and 56. The image shift amount Δ thus obtained does not become zero. When the point image distributions 55 and 56 are relatively displaced by the image shift amount Δ and superimposed, the result is as shown in FIG. That is, in the image shift detection calculation based on the matching degree of the pair of subject image signal patterns, the state shown in FIG. 13 is determined to be the highest matching degree of the point image distributions 55 and 56.

  As described above, in the image shift detection calculation based on the degree of coincidence between the pair of subject image signal patterns, since the identity of the pair of subject images is lost when the aberration of the photographing optical system is large, the image shift amount is detected. An error occurs.

  An image shift amount detection calculation process based on contrast evaluation (image quality) that enables highly accurate image shift amount detection even when the identity of a pair of subject image signal patterns is lost will be described.

  FIG. 14 is a diagram for explaining the detection principle of image shift amount detection based on contrast evaluation. 14 (d), 14 (e) and 14 (f) are superimposed by changing the relative positions of the pair of point image distributions 55 and 56 formed on the intended focal plane by the pair of focus detection light beams. Is displayed. A point image distribution 55 represented by a solid line is formed by the focus detection light beam passing through the region 95, and a point image distribution 56 represented by a broken line is formed by the focus detection light beam passing through the region 96. 14 (a), 14 (b) and 14 (c) show point image distributions 51a, 51b obtained by adding and synthesizing a pair of point image distributions by changing their relative positions and overlaying each other. 51c is shown. The relative position Pa of the pair of point image distributions in FIGS. 14 (a) and 14 (d), the relative position Pb of the pair of point image distributions in FIGS. 14 (b) and 14 (e), and FIG. The relative position Pc of the pair of point image distributions in (c) and 14 (f) is different from each other. In the image shift amount detection calculation process based on the contrast evaluation, the relative position of the pair of point image distributions is For example, the positions Pa, Pb, and Pc are changed in this order. That is, the point image distributions 51a, 51b, and 51c are intensity distributions of a plurality of composite image signals corresponding to a plurality of different shift amounts k.

  As shown in FIGS. 14B and 14E, the peak value of the point image distribution 51b of the composite subject image signal shows the highest value at the relative position Pb of the pair of point image distributions. It becomes the highest and is closest to the point image distribution (FIG. 9) by the photographing light flux when the best image plane coincides with the planned focal plane. Further, as the peak value of the point image distributions 51a and 51c of the composite subject image signal decreases as the distance from the relative position Pb of the pair of point image distributions to the position Pa or Pc increases, the image quality decreases.

  That is, in the case of a point image, a composite subject image is generated while sequentially changing the relative positions of a pair of point image distributions, and the peak value of the image signal of the composite subject image is maximized, resulting in the highest image quality. By setting the relative position to be the image shift amount, it is possible to accurately detect the image shift amount even when the identity of the pair of point image distributions is low.

  The image shift detection based on such a principle is extended to a general subject image by the same mechanism as described above. Since the pair of subject images formed by the pair of focus detection light beams are obtained by convolution of the point image distribution as described above with the subject image in the case of no aberration, the relative positions of the pair of subject images. By generating a composite image while sequentially changing the image, and setting the relative position where the contrast of the composite image is the best as the image shift amount, an accurate image shift amount can be detected even in a general subject image. It becomes possible.

  As described above, in image shift amount detection based on contrast evaluation, the pair of focus detection light beams are in a complementary relationship with the photographing light beam, that is, the pair of focus detection light beams is equivalent to the photographing light beam. By using this, by adding and synthesizing a pair of subject images formed by the pair of focus detection light beams while relatively displacing them, a composite subject image equivalent to the subject image formed by the photographing light beams is generated, The displacement amount at which the contrast evaluation value of the composite subject image is an extreme value, that is, the maximum value or the minimum value is defined as the image shift amount.

  Evaluation of the contrast of the subject image in image shift amount detection based on contrast evaluation is similar in that it performs image quality evaluation equivalent to focus detection by a so-called contrast detection method, but differs in the following points. That is, in contrast detection focus detection, it is necessary to scan the photographic optical system in the direction of the optical axis in order to detect image quality peaks by changing the image quality. In detecting the amount of deviation, it is not necessary to scan the photographic optical system in the optical axis direction in order to change the image quality. In image shift amount detection based on contrast evaluation, it is only necessary to relatively displace a pair of subject image signals. In image shift amount detection based on contrast evaluation, the relative displacement of a pair of subject image signals plays a role equivalent to the scanning drive in the optical axis direction of the imaging optical system in focus detection of the contrast detection method. There is an effect that it is not necessary to perform scanning driving in the optical axis direction of the photographing optical system every time focus detection is performed.

  Next, the image shift detection calculation process based on contrast evaluation for the pair of image signal sequences in step S130 of FIG. 6 will be described in detail with reference to the flowchart of FIG.

In step S200, the initial value of the relative shift amount k of the pair of image signal sequences A _{1 to} A _M and B _{1 to} B _M read from the focus detection pixel sequence (number of pixels 2M) is set to k = −5. Set.

In step S210, the pair of image signal sequences A _{1 to} A _M and B _{1 to} B _M are relatively shifted by the shift amount k. That is, the signal A _N of the image signal sequence A and the signal B _{N + k} of the image signal sequence B correspond with the shift amount k.

In step S220, a pair of image signal sequences A _{1 to} A _M and B _{1 to} B _M that are relatively shifted by the shift amount k are added and combined by equation (2), and M + 1−2 | shown in FIG. k | generates a composite image signal sequence F (n, k) composed of composite image signals (n = | k |, | k | +1,..., M-1- | k |, M- | k). |).
F (n, k) = A _n + B _{n + k} (2)

  In FIG. 15, the horizontal axis represents the sampling position based on the focus detection pixel pitch, and the vertical axis represents the composite image signal intensity (composite image signal sequence). A composite image signal sequence F (n, k) obtained by sampling and outputting by spatially discretely sampling the intensity of the composite image signal indicated by the solid line 1510 at the focus detection pixel pitch is indicated by a circle.

In step S230, the first-order difference processing of equation (3), which is a linear combination operation, is performed on the composite image signal sequence F (n, k), and a high-frequency contrast component is generated from the composite image signal sequence F (n, k). To extract. A contrast signal sequence P (n, k) composed of M−2 | k | contrast components thus obtained is generated.
P (n, k) = F (n, k) -F (n-1, k) (3)

In step S240, the contrast signal sequence P (n, k) is nonlinear by a quadratic function (square function: y = H (x) = x ² ) that is a nonlinear function H (x) as shown in equation (4). Conversion is performed to generate a non-linear contrast signal sequence Q (n, k).
Q (n, k) = H (P (n, k)) = (P (n, k)) ² (4)

  Here, the reason why the non-linear contrast signal sequence Q (n, k) is generated by performing the non-linear conversion by the non-linear function as shown in the equation (4) on the contrast signal sequence P (n, k) will be described.

  FIGS. 16A and 16B show a pair of image signal sequences in the case of a stepped pattern having a step C1 as a pair of images. The pair of image signal strings are relatively shifted by the shift amount k1, and the position of the step C1 is different in each image signal string. FIG. 16C shows a composite image signal sequence composed of a plurality of composite image signals obtained by adding and synthesizing the pair of image signal sequences of FIGS. 16A and 16B. It becomes a step-like pattern having a step C1 at two places. If the contrast evaluation value C (k) of the composite image signal sequence is calculated by integrating the absolute values of the contrast signal sequence P (n, k) (C (k) = Σ | P (n, k) |), The contrast evaluation value C (k1) of the composite image signal sequence of 16 (c) is obtained as follows. That is, the contrast evaluation value C (k1) of the composite image signal sequence in FIG. 16C integrates the absolute value of the step (first-order difference) of the composite image signal sequence over the section where the composite image signal sequence exists. Therefore, C (k1) = 2 × | C1 |.

  17 (a) and 17 (b) show a pair of image signal sequences in the case where the pair of images is a stepped pattern having a step C1 as in FIGS. 16 (a) and 16 (b). In the state where the image signal trains are relatively shifted by the shift amount k2, the positions of the steps C1 of the respective image signal trains coincide with each other. FIG. 17C shows a combined image signal sequence obtained by adding and synthesizing the pair of image signal sequences of FIGS. 17A and 17B, and has a step C2 (= 2 × C1) at one place. Pattern. When the contrast evaluation value C (k2) of the composite image signal sequence in FIG. 17C is calculated by integrating the absolute values of the steps of the composite image signal sequence as they are over the section where the composite image signal sequence exists, C ( k2) = | 2 × C1 | = 2 × | C1 | = C (k1). That is, as shown in FIG. 18A, the contrast evaluation value C (k) with respect to the shift amount k is constant regardless of the shift amount k, and the contrast evaluation is performed by relatively shifting the pair of image signal sequences. Even if the value is calculated, the peak or bottom of the contrast evaluation value cannot be detected, and therefore the amount of image shift cannot be detected.

  In the above description, the contrast evaluation value C (k) has been described as being obtained by integrating the absolute values of the steps of the composite image signal sequence. However, the steps are the contrast components of the composite image signal sequence, and actually (3 ) In the first-order difference calculation of the equation. The contrast component of the composite image signal sequence can generally be obtained by N-th order difference processing (N is a positive integer) of the composite image signal sequence. However, when the contrast component is extracted by N-th order difference processing of the composite image signal sequence. However, a similar phenomenon occurs when the absolute value of the contrast component is integrated as it is to calculate the contrast evaluation value C (k).

  The mathematical reason for such a result is that the addition / synthesis processing of the composite image signal sequence is a linear combination of the pair of image signal sequences (addition of a pair of image signal sequences) and the composite image signal sequence. This is because the Nth-order difference processing for extracting the contrast component is also a linear combination of the signal sequence of the composite image signal sequence (calculation for multiplying each signal by a predetermined coefficient). That is, for the image patterns as shown in FIGS. 16 and 17, when the contrast evaluation value C (k) is calculated by integrating the absolute values of the contrast components extracted by the N-th order difference process on the composite image signal sequence. Since the contrast evaluation value C (k) becomes equal to the sum of the contrast evaluation values for each of the pair of signal sequences (the linear evaluation of the contrast evaluation values for each of the pair of signal sequences). The contrast evaluation value C (k) becomes constant regardless of the shift amount k.

Such a problem can be solved by performing nonlinear conversion once as shown in the equation (4) before integrating the contrast components. That is, when the contrast evaluation value C (k) is calculated by accumulating Q (n, k) obtained by nonlinearly transforming the contrast signal sequence with a square function as shown in equation (4), (C (k) = ΣQ (n, k) = Σ | P (n, k) | ² ), which is as follows. Contrast evaluation value C (k2) = 4 × C1 2 next to the composite picture signal sequence when the step portion of the pair of image match as in FIG. 17 (c), the pair of images as shown in FIG. 16 (c) greater than the contrast evaluation value C (k1) = 2 × C1 2 for composite image signal sequence when the step portion does not match. Accordingly, the contrast evaluation value C (k) does not become constant regardless of the shift amount k, and the contrast evaluation value C (k) is obtained at the shift amount where the stepped portions of the pair of images coincide and the contrast of the composite image signal sequence becomes the highest. Becomes a peak, and the image shift amount can be detected based on the shift amount.

The contrast evaluation value C (k) of the composite image signal sequence shown in FIG. 16C is compared with the contrast signal sequence x = P (n) for the step pattern of the step C1 as shown in FIGS. 16A and 16B. obtained by integrating the square function y = ^{x 2} of k). When the x = C1 is y = C1 ^2, the composite image signal sequence of the step C1 as shown in FIG. 16 (c) Since two, the C (k1) = 2y = 2C1 2. The contrast evaluation value C (k) of the composite image signal sequence shown in FIG. 17C is compared with the contrast signal sequence x = P (n) for the step pattern of the step C1 as shown in FIGS. 17A and 17B. obtained by integrating the square function y = ^{x 2} of k). When the x = 2C1 is y = 4C1 ^2, since one point is stepped 2C1 synthetic image signal sequence as shown in FIG. 17 (c), the C (k2) = y = 4C1 2. Since C (k2) = y = 4C1 ² is larger than C (k1) = 2y = 2C1 ² , the composite image signal sequence is shown in FIGS. 16 (a), 16 (b), 17 (a) and 17 (b). ), The contrast evaluation value C (k) is an extreme value (in this case, a peak in the shift amount k = k2, as shown in FIG. 18B). ) Is obtained.

Returning to the description of FIG. 7, in step S250, the signals constituting the non-linear contrast signal sequence Q (n, k) are integrated by equation (5) to calculate the contrast evaluation value C (k). In the equation (5), Σ represents an integration with respect to n.
C (k) = ΣQ (n, k) (5)

  In step S260, it is checked whether or not the shift amount k has reached 5. If not, the shift amount k is incremented and updated. The process returns to step S210, and step S210 is performed on the updated shift amount k. The contrast evaluation value C (k) is calculated through the process of step S250.

  When the shift amount k becomes 5 in step S260, all the contrast evaluation values C (k) for each shift amount of shift amount k = −5 to 5 are calculated, and the process branches to step S280. Then, based on the contrast evaluation value C (k) obtained discretely corresponding to the shift amount k = −5 to 5 in integer units, the contrast evaluation value when the continuous shift amount is assumed is maximized. The shift amount G (decimal unit) is obtained by interpolation.

FIG. 19 is a diagram for explaining an interpolation method (three-point interpolation) of the shift amount G, where the horizontal axis is the shift amount k (−5, −4,..., 4, 5), and the vertical axis is the contrast. It is an evaluation value. Further, the contrast evaluation value C (k) corresponding to the integer shift amount k is indicated by ●, and the contrast evaluation value C (2) is maximized at the shift amount k = 2. Further, the contrast evaluation values of the shift amount k = 1 immediately before the shift amount k = 2 and the shift amount k = 3 immediately after the shift amount k = 2 are C (1) and C (3), respectively, and C (1)> Suppose C (3). When the shift amount G (decimal unit) is interpolated, a straight line passing through the contrast evaluation values C (1) and C (3) and a straight line having a slope opposite to the straight line and passing through the contrast evaluation value C (2) It is assumed that the coordinates of the intersections of the above are the maximum contrast evaluation value C (G) (indicated by ◯) and the shift amount G when a continuous shift amount is assumed. For generalization, the contrast evaluation value C (kj) is maximized at an integer shift amount kj, and the contrast evaluation values C (kj−1) and C (kj + 1) at shift amounts kj−1 and kj + 1 are continuous. The shift amount G with which the maximum contrast evaluation value C (G) when the shift amount is assumed can be obtained using the three-point interpolation method of equations (6) to (9).
G = kj−D / SLOP (6)
C (G) = C (kj) + | D | (7)
D = {C (kj−1) −C (kj + 1)} / 2 (8)
SLOP = MAX {| C (kj + 1) -C (kj) |, | C (kj-1) -C (kj) |} (9)

  Whether the shift amount G calculated by the equation (6) is reliable depends on whether the contrast evaluation value C (G) calculated by the equation (7) is below a predetermined threshold and / or by the equation (9). The determination is made based on a criterion such that the calculated SLOP falls below a predetermined threshold. In FIG. 19, when no contrast evaluation value peak exists, it is determined that focus detection is impossible.

When the shift amount G is calculated and it is determined that the shift amount G is reliable, the shift amount G is converted into the image shift amount Sf by the equation (10) in step S290. In equation (10), the detection pitch PY is twice the sampling pitch of the same kind of focus detection pixels, that is, the pitch of the imaging pixels.
Sf = PY × G (10)

Further, the image shift amount Sf is multiplied by a predetermined conversion coefficient Kd to be converted into a defocus amount Df.
Df = Kd × Sf (11)

  In equation (11), the conversion coefficient Kd is a conversion coefficient corresponding to the proportional relationship between the distance between the center of gravity of the pair of distance measurement pupils 95 and 96 and the distance measurement pupil distance, and changes according to the aperture F value of the optical system.

The above is the details of the image shift detection calculation process based on the contrast evaluation. A digital camera 201 including a focus detection device in the present embodiment includes an image sensor 212 and a body control device 214. In step S210, the body control device 214 performs image shift processing for relatively shifting the pair of image signal sequences A _{1 to} A _M and B _{1 to} B _M. In step S220, the body control device 214 performs image composition processing for generating a composite image signal sequence F (n, k) by adding and synthesizing the pair of image signal sequences A _{1 to} A _M and B _{1 to} B _M to each other. . In step S230, the body control device 214 performs a contrast extraction process for generating a contrast signal sequence P (n, k) by extracting a plurality of contrast components from the composite image signal sequence F (n, k). In step S240, the body control device 214 performs nonlinear conversion processing for converting the contrast signal sequence P (n, k) into the nonlinear contrast signal sequence Q (n, k). In step S250, the body control device 214 performs a contrast evaluation process for calculating the contrast evaluation value C (k) of the composite image signal sequence F (n, k). In step S280, the body control device 214 performs image shift amount detection processing for detecting the shift amount G corresponding to the extreme value C (G) among the plurality of contrast evaluation values C (k) as the image shift amount Sf. The body control device 214 performs a defocus amount calculation process for calculating the defocus amount Df of the interchangeable lens 202 in step S290.

  By detecting the image shift amount based on the contrast evaluation value in this way, it is possible to detect the image shift amount accurately even when the identity of a pair of images is lost due to optical aberration, and to evaluate the contrast. Since the contrast component of the composite image signal sequence is nonlinearly converted in the value calculation, it is possible to reliably detect the image shift amount even for the stepped image pattern as shown in FIGS. .

In the digital camera 201 described above, the pair of image signal sequences A _{1 to} A _M and B _{1 to} B _M are discrete from the pair of subject images 67 and 68 at a detection pitch PY that is a sampling pitch by the same kind of focus detection pixels. This is a signal sequence obtained by sampling. The plurality of shift amounts k are discrete values in units of detection pitch PY. The body control device 214 detects the contrast evaluation value C (kj) indicating the extreme value in the contrast evaluation value C (k), and the shift amount kj and the shift amount kj corresponding to the contrast evaluation value C (G). Based on the two contrast evaluation values C (kj−1) and C (kj + 1) at the two shift amounts kj−1 and kj + 1 increased or decreased by the pitch PY, the image shift amount Sf is detected with an accuracy equal to or less than the detection pitch PY. As a result, the amount of image shift can be detected with higher accuracy.

In step S230 in FIG. 7, the first-order difference processing of equation (3) is performed on the composite image signal sequence F (n, k). As described above, from the composite image signal sequence F (n, k). As a linear combination calculation process for extracting a high-frequency contrast component and generating a contrast signal sequence P (n, k), an N-th order difference calculation process for a positive integer N can be used. For example, when the second-order difference process is used, the equation (12) is obtained.
P (n, k) = − F (n−1, k) + 2 × F (n, k) −F (n + 1, k) (12)

  When the difference interval is the same, the second-order difference processing is preferable because the efficiency of extracting high-frequency components is higher than the first-order difference processing.

The nonlinear function H (x) in step S240 in FIG. 7 is a simple square function (y = H (x) = x ² ), but in reality, the output range of the output y is set to the input range of the input x. In some cases, it is more convenient to match (for example, the number of operation bits of the CPU is limited). In such a case, the nonlinear function H (x) can be adjusted as appropriate. For example, when the input range of the input x is 0 to 100 and the output range of the output y is 0 to 100, the nonlinear function H (x) can be determined as shown in the equation (13), and the nonlinear function H (x ) Is represented by the graph of FIG.
^{y = H (x) = x} 2/100 (13)

  In the following description, it is assumed that the sign of the input range of the input x is plus and the nonlinear function H (x) is a function in which the sign of the output range of the output y is plus.

  The nonlinear function H (x) is not limited to the above-mentioned square function, and various variations can be considered. However, in order to ensure the steadiness of the contrast evaluation (the characteristic that the contrast evaluation value increases or decreases when the contrast is high does not change regardless of the size of the contrast component), the nonlinear function H (x) The magnitude relation of the contrast component after conversion needs to be always constant with respect to the value of the contrast component before nonlinear conversion. This is equivalent to the fact that the magnitude relationship at any input x in the input range of the input x of the nonlinear function H (x) is a nonlinear function that is always preserved or inverted in the output y. That is, in the nonlinear function H (x), when arbitrary inputs x1, x2 (x1 <x2) are set, H (x1) <H (x2) or H (x1)> H (x2) always holds. H (x1) = H (x2) does not hold. This means that the nonlinear function H (x) is a monotonically increasing function or a monotonically decreasing function in the input range of the input x. In other words, when the first derivative of the nonlinear function H (x) is h (x), the first derivative h (x) is h (x)> 0 or h (x) <in the input range of the input x. 0. That is, h (x) ≠ 0 for any x.

  Similarly, in order to ensure the continuity of the contrast evaluation, the magnitude relationship at the arbitrary input x in the input range of the input x is also related to the first derivative with respect to the first derivative h (x) of the nonlinear function H (x). It is desirable that the value of the function h (x) is always saved or inverted. That is, in the first derivative h (x), when arbitrary inputs x1 and x2 (x1 <x2), h (x1) <h (x2) or h (x1)> h (x2) is always established. However, h (x1) = h (x2) does not hold. This means that the first derivative h (x) is a monotonically increasing function or a monotonically decreasing function in the input range of the input x. In other words, when the second derivative of the nonlinear function H (x) is r (x), the second derivative r (x) is r (x)> 0 or r (x) <in the input range of the input x. 0. That is, r (x) ≠ 0 for any x.

  For the nonlinear function H (x) in the equation (13), the first derivative h (x) = x / 50> 0 (x = 0 to 100), the second derivative r (x) = 1/50> 0. (X = 0 to 100). Therefore, the conditions of the nonlinear function H (x): monotonically increasing (first derivative h (x)> 0) and the first derivative h (x): monotonically increasing (second derivative r (x)> 0) Satisfied. Nonlinear function H (x): monotonically increasing (first derivative h (x)> 0) and first derivative h (x): monotonically increasing (second derivative r (x)> 0) are satisfied. In the case of the nonlinear function H (x), the graph of the contrast evaluation value C (k) has a characteristic (peak value) having a peak (extreme value) at the shift amount at which the contrast of the composite image signal sequence is maximum as shown in FIG. Convex characteristics).

  Also, the condition of nonlinear function H (x): monotonically decreasing (first derivative h (x) <0) and first derivative h (x): monotonically increasing (second derivative r (x)> 0) is satisfied. In the case of the non-linear function H (x), the graph of the contrast evaluation value C (k) has a bottom (extreme value) in the shift amount at which the contrast of the composite image signal sequence as shown in FIG. (Concave characteristic). For a step pattern in which the composite image signal sequence has steps as shown in FIGS. 16 (a), 16 (b), 17 (a) and 17 (b), for example, a nonlinear function (first-order derivative) shown in FIG. An example in which nonlinear transformation is performed using the function h (x)> 0 and the second derivative r (x) <0) will be described. Assuming that the step pattern step value is 30, in the case where there are two steps in the composite image signal sequence as shown in FIG. 16C, according to FIG. 25, the y value when x = 30. Is about 55, and the contrast evaluation value C (k1) is a value obtained by multiplying the value of y by the number of steps, that is, a value obtained by doubling the value. When there is one step having a step value of 60 in the composite image signal sequence as shown in FIG. 17C, according to FIG. 25, the contrast evaluation value C (k2) is twice the step. The value of y at x = 60, that is, about 77. Since C (k1)> C (k2), the graph of the contrast evaluation value C (k) has a concave characteristic.

Nonlinear function H (x): monotonically increasing (first derivative h (x)> 0) and first derivative h (x): monotone decreasing (second derivative r (x) <0) are satisfied. In the case of the non-linear function H (x), and the non-linear function H (x): monotonic decrease (first derivative h (x) <0) and the first derivative h (x): monotonic decrease (second derivative r ( In the case of the nonlinear function H (x) that satisfies the condition x) <0), the graph of the contrast evaluation value C (k) is a shift that maximizes the contrast of the composite image signal sequence as shown in FIG. It becomes a characteristic (concave characteristic) having a bottom (extreme value) in quantity. The following table summarizes the relationship between the above-mentioned conditions and the contrast evaluation value characteristics. Even when the graph of the contrast evaluation value C (k) has a concave characteristic, the shift amount G (decimal unit) that minimizes the contrast evaluation value when a continuous shift amount is assumed is obtained by three-point interpolation. It can be obtained by a technique.

Nonlinear function H (x): monotonically increasing (first derivative h (x)> 0) and first derivative h (x): monotonically increasing (second derivative r (x)> 0) are satisfied. Examples of the nonlinear function H (x) include formula (14) (FIG. 21), formula (15) (FIG. 22), formula (16) (FIG. 23), formula (17) (FIG. 24). The EXP () function in equation (15) is a power function of the Napier number e, and the LOG () function in equation (16) is a common logarithm with base 10.
y = H (x) = 750 / (15-x / 10) -50 (14)
y = H (x) = 100 × (EXP (95 + x / 20) −EXP (95)) / EXP (100) (15)
y = H (x) = 100-50 * LOG (100-99 * x / 100) (16)
y = H (x) = 800 * (1-COS (x / 200)) (17)

Nonlinear function H (x): monotonically increasing (first derivative h (x)> 0) and first derivative h (x): monotone decreasing (second derivative r (x) <0) are satisfied. As an example of the nonlinear function H (x), there is a formula (18) (FIG. 25). The SQRT () function in the equation (18) is a root (square root) function.
y = H (x) = 10 × SQRT (x) (18)

Nonlinear function H (x): monotonically decreasing (first derivative h (x) <0) and first derivative h (x): monotonically increasing (second derivative r (x)> 0) are satisfied. As an example of the nonlinear function H (x), there is an equation (19) (FIG. 26).
y = H (x) = 100−10 × SQRT (x) (19)

Nonlinear function H (x): monotonic decrease (first derivative h (x) <0) and first derivative h (x): monotonic decrease (second derivative r (x) <0) are satisfied. As an example of the nonlinear function H (x), there is an equation (20) (FIG. 27).
^{y = H (x) = 100} -x 2/100 (20)

  The above-described nonlinear function and its first derivative are both monotone functions within the range of values that can be taken by the absolute values of the plurality of contrast components. Therefore, it is possible to ensure the continuity of the contrast evaluation and to obtain an image shift amount accurately.

In steps S210 to S250 of FIG. 7, the process of calculating the contrast evaluation value C (k) has been described in the following steps in order to facilitate understanding of the essential contents of the present invention. That is, (i) the pair of image signal sequences A _{1 to} A _M and B _{1 to} B _M are relatively shifted by a shift amount k. (Ii) A pair of image signal sequences A _{1 to} A _M and B _{1 to} B _M that are relatively shifted by the shift amount k are added and synthesized by the equation (2) to obtain a synthesized image signal sequence F (n, k ) Is generated. (Iii) Contrast signal sequence P obtained by performing first-order difference processing of equation (3) on composite image signal sequence F (n, k) and extracting high-frequency contrast components from composite image signal sequence F (n, k) (N, k) is generated. (Iv) Non-linear conversion is performed on the contrast signal sequence P (n, k) by a quadratic function (square function: H (x) = x ² ) that is a non-linear function H (x) as shown in equation (4). , A non-linear contrast signal sequence Q (n, k) is generated. (V) The contrast evaluation value C (k) is calculated by integrating the nonlinear contrast signal sequence Q (n, k) according to the equation (5).

However, in the actual processing for calculating the contrast evaluation value C (k), the composite image signal sequence F (n, k), the contrast signal sequence P (n, k), which are intermediate products in computation, and the nonlinear contrast It is not always necessary to explicitly generate the signal sequence Q (n, k). For example, as shown in the equation (21), the contrast signal sequence P (n, k) can be directly calculated without explicitly generating the composite image signal sequence F (n, k).
P (n, k) = (A _n + B _{n + k} ) − (A _n−1 + B _{n−1 + k} ) (21)

For example, as shown in the equation (22), the nonlinear contrast signal sequence Q (n, k) can be directly calculated without explicitly generating the contrast signal sequence P (n, k).
Q (n, k) = ((A _n + B _{n + k} ) − (A _n−1 + B _{n−1 + k} )) ² (22)

Alternatively, as shown in the equation (23), the contrast evaluation value C (k) can be directly calculated from a pair of image signal sequences without explicitly generating any intermediate signal sequence.
C (k) = Σ ((A _n + B _{n + k} ) − (A _n−1 + B _{n−1 + k} )) ² (23)

  That is, in the present invention, whether or not to explicitly generate the composite image signal sequence F (n, k), contrast signal sequence P (n, k), and nonlinear contrast signal sequence Q (n, k) is essential. is not. The essential point is the process of calculating the contrast evaluation value C (k) by nonlinearly transforming and integrating the contrast components of the information that is added and synthesized by relatively shifting the pair of image signal sequences.

  In the above-described embodiment, each of the pair of focus detection pixels 315 and 316 has one photoelectric conversion unit and individually receives the pair of focus detection light beams 75 and 76. It is also possible to change the configuration so that the pair of photoelectric conversion units individually receive the pair of focus detection light beams 75 and 76.

  In the embodiment described above, the focus detection operation by the pupil division phase difference detection method using the microlens has been described as an example. However, the present invention is not limited to the focus detection of such a method, and a well-known re-imaging pupil. The present invention can also be applied to focus detection using a divided phase difference detection method.

  In the re-imaging pupil division phase difference detection method, a pair of subjects formed by a pair of focus detection light beams that pass through a pair of distance measuring pupils using a pair of separator lenses for a subject image formed on the primary image plane An image is re-imaged on a pair of image sensors, and an image shift amount of the pair of subject images is detected based on outputs of the pair of image sensors. Accordingly, when the optical characteristics of the photographing optical system are not good, the identity of the signal patterns (shapes) of the pair of subject images is lost, and the degree of coincidence of the pair of subject image signals is deteriorated. Since problems similar to those of the divided phase difference detection method occur, accurate focus adjustment can be performed by using the image shift detection calculation processing based on contrast evaluation according to the present invention.

  Note that the imaging device to which the focus detection device is applied is not limited to the digital camera 201 having a configuration in which the interchangeable lens 202 is attached to the camera body 203 as described above. For example, the present invention can be applied to a lens-integrated digital camera or video camera. Furthermore, the present invention can be applied to a small camera module built in a mobile phone, a surveillance camera, a visual recognition device for a robot, an in-vehicle camera, and the like.

  The present invention is not limited to a so-called pupil division phase difference detection type focus detection apparatus that detects the defocus amount of the photographing optical system by the TTL method. For example, the present invention can be applied to a so-called external light type phase difference detection distance measuring device having a pair of distance measuring optical systems separately from the photographing optical system. By applying the present invention to a pair of image signal sequences generated by a pair of image sensors spatially and discretely sampling a pair of images formed by a pair of ranging optical systems at a predetermined pitch. It is also possible to detect the image shift amount of the pair of image signal sequences and calculate the subject distance based on the image shift amount. In this way, even when there is a slight difference in aberration characteristics between the pair of distance measuring optical systems, highly accurate image shift detection can be achieved, so the aberration characteristics of the pair of distance measuring optical systems coincide with each other with high precision. This makes it easier to manufacture and lowers costs.

  Further, the image shift detection for a pair of images according to the present invention is not limited to the above-described focus detection and distance measurement. For example, in an imaging apparatus that includes an imaging optical system and an image sensor that spatially samples an image formed by the imaging optical system and generates an image signal sequence at a predetermined frame interval, image shift detection according to the present invention is performed. By applying two-dimensionally to two image signal sequences (a pair of image signal sequences) in different frames, it is possible to detect the amount of image shift between the two image signal sequences. The image shift amount can be recognized as a shake amount of the imaging apparatus, or can be recognized as a movement (motion vector) of a subject image between frames. Further, the present invention can be applied to two image signal sequences (a pair of image signal sequences) generated completely independently. For example, in order to detect a specific pattern, the image signal sequence is compared with an observed reference image signal sequence, and applied to so-called template matching that detects the position and presence of a specific pattern in the reference image signal sequence. Can do.

  Such an image shift amount detection device as an application example includes an image sensor that generates a pair of image signal sequences obtained by discretely sampling a pair of images at a predetermined spatial pitch. The image shift amount detection device includes an image shift processing unit that relatively shifts a pair of image signal sequences corresponding to a plurality of shift amounts. The image shift amount detection device adds and synthesizes a pair of image signal sequences that are relatively shifted by the image shift processing unit, thereby synthesizing a plurality of combined image signals for each shift amount. It has an image composition processing part which generates an image signal sequence. The image shift amount detection device extracts a plurality of contrast components from a composite image signal sequence by performing a linear combination operation on a plurality of composite image signals, thereby extracting the plurality of contrast components from the plurality of contrast components for each shift amount. A contrast extraction processing unit for generating a contrast signal sequence. The image shift amount detection device includes a nonlinear conversion processing unit that converts a contrast signal string into a nonlinear contrast signal string by nonlinearly transforming a plurality of contrast components based on a nonlinear function. The image shift amount detection device includes a contrast evaluation processing unit that calculates the contrast evaluation value of the composite image signal sequence for each shift amount based on the nonlinear contrast signal sequence. This image shift amount detection device uses a pair of shift amounts corresponding to extreme values among a plurality of contrast evaluation values obtained by calculating a contrast evaluation value for each shift amount corresponding to a plurality of shift amounts. An image shift amount detection processing unit that detects the relative image shift amount of the first image. According to the image shift amount detection device including these configurations, the image shift amount can be detected with high accuracy.

10 microlens, 11, 15, 16 photoelectric conversion unit,
41 Upper part, 44 Lower part, 45, 46, 47, 48 Edge part,
51, 55, 56 Point image distribution, 65, 66, 67, 68 Subject image,
71, 75, 76 Luminous flux, 90 Exit pupil, 91 Optical axis, 95, 96 Distance pupil,
97 area common to all imaging pixels, 98 planned focal plane, 99 plane,
100 shooting screen, 101 focus detection area,
201 digital camera, 202 interchangeable lens, 203 camera body,
204 mount unit, 206 lens control device, 208 zooming lens,
209 lens, 210 focusing lens, 211 aperture, 212 imaging device,
214 body control device, 215 liquid crystal display element driving circuit, 216 liquid crystal display element,
217 eyepiece, 219 memory card, 221 AD converter,
310 imaging pixels, 315, 316 focus detection pixels,
1510 Solid line

Claims (8)

An image sensor that photoelectrically converts an image formed by each of light beams passing through a pair of regions of the exit pupil of the optical system, and generates a pair of image signal sequences;
A composite image signal composed of a plurality of composite image signals corresponding to each shift amount of the plurality of shift amounts by adding and synthesizing the pair of image signal sequences relatively shifted corresponding to the plurality of shift amounts. An image composition means for generating a sequence;
Contrast extraction means for generating a contrast signal sequence composed of the plurality of contrast components corresponding to the shift amounts by extracting a plurality of contrast components from the composite image signal sequence through a linear combination operation on the plurality of composite image signals. When,
Based on the nonlinear contrast signal sequence obtained by converting the contrast signal sequence by nonlinearly converting the plurality of contrast components based on a nonlinear function, the contrast evaluation value of the composite image signal sequence corresponds to each shift amount. Contrast evaluation means for calculating
An image in which a shift amount corresponding to an extreme value among a plurality of contrast evaluation values obtained by calculating the contrast evaluation value corresponding to each shift amount is detected as a relative image shift amount of the image. A deviation amount detecting means;
A focus detection apparatus comprising: a defocus amount calculation unit that calculates a defocus amount of the optical system based on the image shift amount. The focus detection apparatus according to claim 1,
The focus detection apparatus according to claim 1, wherein the nonlinear function is a monotone function in a range of values that can be taken by absolute values of the plurality of contrast components. The focus detection apparatus according to claim 2,
The focus detection apparatus according to claim 1, wherein the first derivative of the nonlinear function is a monotone function in a range of values that can be taken by absolute values of the plurality of contrast components. The focus detection apparatus according to claim 3,
The focus detection apparatus, wherein the nonlinear function is a quadratic function. In the focus detection apparatus according to any one of claims 1 to 4,
The focus detection apparatus according to claim 1, wherein the linear combination calculation is an N-th order difference calculation for a positive integer N. In the focus detection apparatus according to any one of claims 1 to 5,
The focus evaluation apparatus, wherein the contrast evaluation unit calculates the contrast evaluation value by integrating signals constituting the non-linear contrast signal sequence. In the focus detection apparatus according to claim 1,
The pair of image signal sequences is a signal sequence obtained by discretely sampling the image at a predetermined spatial pitch,
The plurality of shift amounts are discrete values in the predetermined spatial pitch unit,
The image shift amount detection means includes a contrast evaluation value indicating the extreme value among the plurality of contrast evaluation values, a shift amount corresponding to the contrast evaluation value, and an increase / decrease by the predetermined spatial pitch with respect to the shift amount. A focus detection apparatus that detects the image shift amount with an accuracy equal to or less than the predetermined spatial pitch based on the two contrast evaluation values for the two shift amounts. Image signal sequence generation means for generating a pair of image signal sequences obtained by discretely sampling a pair of images at a predetermined spatial pitch;
A composite image signal composed of a plurality of composite image signals corresponding to each shift amount of the plurality of shift amounts by adding and synthesizing the pair of image signal sequences relatively shifted corresponding to the plurality of shift amounts. An image composition means for generating a sequence;
Contrast extraction means for generating a contrast signal sequence composed of the plurality of contrast components corresponding to the shift amounts by extracting a plurality of contrast components from the composite image signal sequence through a linear combination operation on the plurality of composite image signals. When,
Based on the nonlinear contrast signal sequence obtained by converting the contrast signal sequence by nonlinearly converting the plurality of contrast components based on a nonlinear function, the contrast evaluation value of the composite image signal sequence corresponds to each shift amount. Contrast evaluation means for calculating
A shift amount corresponding to an extreme value among a plurality of contrast evaluation values obtained by calculating the contrast evaluation value corresponding to each shift amount is detected as a relative image shift amount of the pair of images. An image shift amount detecting device comprising: an image shift amount detecting means for performing the operation.

Images