JP2014038191A - Correlation calculation device and focus detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To shorten a correlation calculation time to the extent that correlation calculation accuracy can be secured.SOLUTION: A correlation calculation device according to the present invention comprises: first correlation calculation means that calculates a first local correlation value by multiplication between a plurality of pieces of first signal data and a plurality of pieces of second signal data, integrates the first local correlation value and thereby calculates a first integrated correlation value; logarithmic conversion means that generates a plurality of pieces of third signal data and a plurality of pieces of fourth signal data by logarithmically converting the plurality of pieces of first signal data and the plurality of pieces of second signal data; and second correlation calculation means that calculates a second local correlation value by addition between the plurality of pieces of third signal data and the plurality of pieces of fourth signal data, integrates the second local correlation value and thereby calculates a second integrated correlation value.

Description

  The present invention relates to a correlation calculation device and a focus detection device.

  As one of devices including a correlation calculation device, a pupil detection type phase difference detection type focus detection device is known. In the focus detection device, a pair of signal data sequences corresponding to a pair of images formed by a pair of focus detection light beams passing through the exit pupil of the optical system is generated, and correlation is performed while relatively shifting the pair of signal data sequences. By performing the operation, a correlation value between the pair of signal data strings is calculated. A shift amount at which the correlation value is an extreme value is detected as a relative image shift amount of the pair of images, and a focus adjustment state of the optical system is detected according to the image shift amount. The focus adjustment state of the optical system is expressed as a difference between the planned focal plane and the detected image plane, that is, a defocus amount.

  In such a pupil division type phase difference detection type focus detection apparatus, when a pair of focus detection light beams undergo non-uniform vignetting, a signal level between a pair of signal data is determined according to the difference in the amount of received light. An imbalance occurs. A correlation calculation method for detecting a defocus value with high accuracy even when such signal level imbalance occurs is disclosed (for example, see Patent Document 1).

  For example, in Equations (8) and (9) of Patent Document 1, a local data pair (first data and second data located in the vicinity thereof) of a pair of signal data strings and the other signal data. First multiplication data obtained by multiplying the first data and the fourth data between the pair of local data in the column (third data and fourth data in the vicinity thereof), second data and third data, And the second multiplication data multiplied by. At the same time, the local correlation value is calculated by taking the absolute value of the difference between the first multiplication data and the second multiplication data, and further obtained by integrating the local correlation value over a predetermined section of the signal data string. The integrated correlation value obtained is used as a correlation value between a pair of signal data strings.

JP 2007-333720 A

  However, the conventional correlation calculation device has a problem that the calculation time is long.

(1) The correlation calculation device according to the present invention includes a plurality of first signal data forming a local partial array in a first signal data array including a plurality of first signal data and a first signal data including a plurality of second signal data. A first local correlation value is calculated by multiplying a plurality of second signal data having a local partial arrangement corresponding to a plurality of first signal data having a local partial arrangement in the two-signal data arrangement. In addition, the degree of correlation between the first signal data array and the second signal data array is obtained by integrating the first local correlation values calculated each time the plurality of first signal data and the plurality of second signal data are changed. First correlation calculating means for calculating a first integrated correlation value representing the third signal data, and third signal data comprising a plurality of third signal data by logarithmically converting the plurality of first signal data and the plurality of second signal data, respectively. Logarithm conversion means for generating a fourth signal data array comprising an array and a plurality of fourth signal data, a plurality of third signal data forming a partial partial array in the third signal data array, and a fourth signal data array A second local correlation value is calculated by performing addition with a plurality of fourth signal data forming a local partial arrangement corresponding to a plurality of third signal data forming a local partial arrangement, The second integration representing the degree of correlation between the third signal data array and the fourth signal data array by integrating the second local correlation value calculated each time the three signal data and the plurality of fourth signal data are changed. And a second correlation calculation means for calculating a correlation value.
(2) The correlation calculation device according to the present invention includes a first signal data array composed of a plurality of first signal data obtained by sampling the spatial intensity distribution of the first image at a plurality of spatial coordinate points arranged at predetermined intervals. , A plurality of first signal data obtained at a plurality of spatial coordinate points having a first positional relationship in the vicinity of the first spatial coordinate point among the plurality of spatial coordinate points and a spatial intensity distribution of the second image are predetermined. In a second signal data array composed of a plurality of second signal data obtained by sampling at a plurality of spatial coordinate points arranged at intervals, a second positional relationship with respect to a first spatial coordinate point among the plurality of spatial coordinate points The first product data and the second product data are obtained by multiplying a plurality of second signal data obtained at a plurality of spatial coordinate points in the first positional relationship in the vicinity of the second spatial coordinate point at And the first product data and The first difference data is calculated by performing subtraction with the second product data, and the first difference data is subjected to non-linear transformation and the same encoding to obtain a plurality of first signal data in the vicinity of the first spatial coordinate point A first local correlation value between a plurality of second signal data in the vicinity of the second spatial coordinate point is calculated, and the first local correlation value is calculated by moving the first spatial coordinate point over a predetermined spatial coordinate area. First correlation calculation means for calculating a first integrated correlation value in the second positional relationship between the first signal data array and the second signal data array by integrating, a plurality of first signal data, and a plurality of first signal data The second signal data is logarithmically converted to calculate a plurality of third signal data and a plurality of fourth signal data, and a third signal data array including a plurality of third signal data and a fourth signal data including a plurality of fourth signal data are calculated. Logarithm to generate signal data array In the third signal data array, a plurality of third signal data obtained at a plurality of spatial coordinate points in the first positional relationship in the vicinity of the first spatial coordinate point, and a fourth signal data array, By performing addition with a plurality of fourth signal data obtained at a plurality of spatial coordinate points having the first positional relationship in the vicinity of the second spatial coordinate point having the second positional relationship with respect to the one spatial coordinate point The first sum data and the second sum data are calculated, and the second difference data is calculated by further subtracting the first sum data and the second sum data. To calculate a second local correlation value between the plurality of third signal data in the vicinity of the first space coordinate point and the plurality of fourth signal data in the vicinity of the second space coordinate point. Move the coordinate point over a predetermined spatial coordinate area A second correlation calculating means for calculating a second integrated correlation value in the second positional relationship between the third signal data array and the fourth signal data array by integrating the function values; and whether a predetermined condition is satisfied Accordingly, the selection means for selecting one of the calculation of the first integrated correlation value by the first correlation calculation means and the calculation of the second integration correlation value by the second correlation calculation means, and the first correlation calculation by the selection means When the calculation of the first integrated correlation value by the means is selected, the first integrated correlation value corresponding to the second positional relationship is calculated every time the second positional relationship is changed over a predetermined range, and the first integrated correlation value is calculated. A spatial relative displacement amount between the first signal data array and the second signal data array is calculated based on the second positional relationship when the value indicates an extreme value, and the second integration by the second correlation calculation unit is performed by the selection unit. When the calculation of the correlation value is selected, the second Each time the positional relationship is changed over a predetermined range, a second integrated correlation value corresponding to the second positional relationship is calculated, and the first signal is based on the second positional relationship when the second integrated correlation value shows an extreme value. Relative displacement amount detection means for calculating a spatial relative displacement amount between the data array and the second signal data array is provided.
(3) The correlation calculation device according to the present invention includes a plurality of first signal data forming a local partial array in a first signal data array including a plurality of first signal data, and a second signal data including a plurality of second signal data. A local partial array is obtained by multiplying a plurality of second signal data forming a local partial array corresponding to a plurality of first signal data forming a local partial array in the two-signal data array. In the first signal data array and the second signal data array, a first local correlation value is calculated between the plurality of first signal data forming the plurality of first signal data and the plurality of second signal data forming the local partial array. The first signal data array is obtained by accumulating the first local correlation values calculated each time the plurality of first signal data forming the local partial array and the plurality of second signal data forming the local partial array are changed. And second A first correlation calculation means for calculating a first integrated correlation value representing a degree of correlation between the two signal data arrays, and a plurality of first signal data and a plurality of second signal data by logarithmically converting each of the plurality Logarithmic conversion means for generating a third signal data array composed of third signal data and a fourth signal data array composed of a plurality of fourth signal data, and a plurality of third signals forming a local partial array in the third signal data array By performing addition between the signal data and a plurality of fourth signal data forming a local partial array corresponding to a plurality of third signal data forming a local partial array in the fourth signal data array, Calculating a second local correlation value between the plurality of third signal data forming the local partial array and the plurality of fourth signal data forming the local partial array, and in the third signal data array and By integrating a plurality of third signal data forming a local partial array and a plurality of fourth signal data forming a local partial array in the signal data array, each time the second local correlation value calculated is integrated, And a second correlation calculation means for calculating a second integrated correlation value representing a degree of correlation between the third signal data array and the fourth signal data array.

  According to the correlation calculation device of the present invention, the correlation calculation time can be shortened within a range in which the correlation calculation accuracy is ensured.

It is a figure which shows the structure of a digital still camera. It is a figure which shows the focus detection position on an imaging screen. It is a front view which shows the detailed structure of an image pick-up element. It is a figure explaining the focus detection method by a pupil division system. It is a front view which shows the projection relationship in an exit pupil plane. It is the figure which showed the intensity distribution (light quantity) of the image signal in a focus detection position on the vertical axis, and the position deviation in the focus detection position on the horizontal axis. 5 is a flowchart showing an imaging operation of the digital still camera 210. It is a figure which shows the data flow of correlation calculation. It is a figure which shows the data flow of correlation calculation. It is a figure which shows the data flow of correlation calculation. It is the graph which showed the relationship of logarithmic transformation when a horizontal axis is input x and a vertical axis | shaft is output y. It is a graph at the time of converting signal waveform I0 of input data sequence Xi into signal waveform I1 of output data sequence Yi using logarithmic conversion T1. It is a graph at the time of converting the signal waveform I0 of the input data sequence Xi into the signal waveform I2 of the output data sequence Yi using logarithmic conversion T2. It is a graph at the time of converting the signal waveform I3 of the input data sequence Xi into the signal waveform I4 of the output data sequence Yi using logarithmic conversion T3. It is the graph which showed a pair of input signal waveform and the calculation result of correlation calculation. It is a flowchart which shows the detail of a focus detection operation | movement. It is a flowchart which shows the detail of a 2nd correlation calculation process. It is a flowchart which shows the detail of a 2nd correlation calculation process. It is a flowchart which shows the detail of a focus detection operation | movement. It is a figure which shows the data flow of correlation calculation. It is a figure which shows the image which expanded the face image vicinity in a basic image. It is a figure which shows the image which expanded the face image vicinity of the basic low-pass image. It is a figure which shows the image which expanded the area | region vicinity of the partial image in a comparison object image. It is a figure which shows the image which expanded the partial image vicinity in the comparison object low pass image. It is a figure which shows the correspondence of the calculation formula of a local correlation value, and each pixel data used for the calculation by the formula. It is the figure which showed typically the calculation of the local correlation value in the correlation pixel position. It is a front view which shows the detailed structure of an image pick-up element. It is a figure which shows the focus detection apparatus of a re-imaging pupil division system. It is a figure which shows the ranging device of an external light triangulation system.

  A digital still camera 201 will be described as an imaging apparatus having a focus detection device including a correlation calculation device according to an embodiment of the present invention. FIG. 1 is a diagram illustrating a configuration of a digital still camera 201. The digital still camera 201 includes an interchangeable lens 202 and a camera body 203. The interchangeable lens 202 is attached to the mount portion 204 of the camera body 203.

  The interchangeable lens 202 includes lenses 205, 206, and 207, a diaphragm 208, a lens drive control device 209, and the like. A lens 206 is a zooming lens, and a lens 207 is a focusing lens. The lens drive control device 209 includes a CPU and its peripheral components, and controls driving of the focusing lens 207 and the aperture 208, position detection of the zooming lens 206, focusing lens 207 and the aperture 208, and camera drive control of the camera body 203. Transmission of lens information and reception of camera information by communication with the apparatus 212 are performed.

  The camera body 203 includes an image sensor 211, a camera drive control device 212, a memory card 213, an LCD driver 214, an LCD 215, an eyepiece lens 216, and the like. The image sensor 211 is disposed on the planned image plane (planned focal plane) of the interchangeable lens 202, captures the subject image formed by the interchangeable lens 202, and outputs an image signal. An imaging pixel (hereinafter simply referred to as an imaging pixel) is two-dimensionally arranged in the imaging element 211, and a focus detection pixel (hereinafter, referred to as a focus detection position) is replaced with an imaging pixel in a portion corresponding to the focus detection position. Simply referred to as a focus detection pixel). A plurality of focus detection pixels are linearly arranged to form a focus detection pixel row.

  The camera drive control device 212 includes peripheral components such as a CPU and a memory. The camera drive control device 212 controls drive of the image sensor 211, processing of the captured image, focus detection and focus adjustment of the interchangeable lens 202, control of the aperture 208, display control of the LCD 215, communication with the lens drive control device 209, and camera Performs overall sequence control. The camera drive control device 212 communicates with the lens drive control device 209 via an electrical contact 217 provided on the mount unit 204.

  The camera drive control device 212 constitutes a correlation calculation device, an image shift detection device, and a focus detection device. The camera drive control device 212 includes a logarithmic lookup table to be described later, and includes logarithmic conversion processing, local correlation value calculation processing, correlation value integration processing, data value distribution detection processing, intermediate processing calculation processing, image shift detection processing, and defocusing. Perform quantity detection processing.

  The memory card 213 is an image storage that stores captured images. The LCD 215 is used as a display for a liquid crystal viewfinder (EVF: electronic viewfinder). The photographer can visually recognize the captured image displayed on the LCD 215 via the eyepiece lens 216.

  The subject image that has passed through the interchangeable lens 202 and formed on the image sensor 211 is photoelectrically converted by the image sensor 211 and output as an image. The image output is sent to the camera drive control device 212. The camera drive control device 212 calculates the defocus amount at the focus detection position based on the output of the focus detection pixel, and sends this defocus amount to the lens drive control device 209. The camera drive control device 212 sends an image signal generated based on the output of the imaging pixel to the LCD driver 214. The camera drive control device 212 displays an image on the LCD 215 based on the image signal. At the same time, the camera drive control device 212 stores the image signal in the memory card 213.

  The lens drive control device 209 detects the positions of the zooming lens 206, the focusing lens 207, and the diaphragm 208, and calculates lens information based on these detection positions, or detects a position from a lookup table prepared in advance. Select lens information according to. The lens drive control device 209 sends the lens information to the camera drive control device 212. The lens drive control device 209 calculates a lens drive amount based on the defocus amount received from the camera drive control device 212, and drives and controls the focusing lens 207 based on the lens drive amount.

  FIG. 2 shows a focus detection position (focus detection area, focus detection area) on the imaging screen G set on the planned imaging plane of the interchangeable lens 202. Focus detection positions G1 to G5 are set on the imaging screen G, and focus detection pixels of the imaging element 211 are linearly arranged in the longitudinal direction of the focus detection positions G1 to G5 on the imaging screen G. That is, the focus detection pixel row on the image sensor 211 samples an image at the focus detection positions G1 to G5 in the subject image formed on the shooting screen G. The photographer manually selects an arbitrary focus detection position from the focus detection positions G1 to G5 according to the shooting composition.

  FIG. 3 is a front view showing a detailed configuration of the image sensor 211, and is a partially enlarged view in which the periphery of one focus detection position G1 on the image sensor 211 is enlarged. The image sensor 211 includes an imaging pixel 310 and a focus detection pixel 311 for focus detection.

  The imaging pixel 310 includes the microlens 10, the photoelectric conversion unit 11, and a color filter (not shown). The focus detection pixel 311 includes a microlens 10 and a pair of photoelectric conversion units 12 and 13. The photoelectric conversion unit 11 of the imaging pixel 310 is designed in such a shape that the microlens 10 receives all the light beams that pass through the exit pupil (for example, F1.0) of the bright interchangeable lens 202. The pair of photoelectric conversion units 12 and 13 of the focus detection pixel 311 are designed in such a shape that the microlens 10 receives all the light beams passing through a specific exit pupil (for example, F2.8) of the interchangeable lens.

  The imaging pixel 310 arranged two-dimensionally on the imaging element 211 includes a color filter of any one of red (R), green (G), and blue (B). Each color filter has different spectral sensitivity characteristics. The imaging pixels 310 including any one of R, G, and B color filters are arranged in a Bayer array as shown in FIG.

  The focus detection pixel 311 is not provided with a color filter in order to increase the amount of light, and its spectral sensitivity characteristic is a total of the spectral sensitivity of a photodiode that performs photoelectric conversion and the spectral sensitivity of an infrared cut filter (not shown). Spectral sensitivity characteristics. The spectral sensitivity of the focus detection pixel 311 has a spectral sensitivity characteristic that is obtained by adding the spectral sensitivities of the green pixel G, red pixel R, and blue pixel B of the imaging pixel 310. The light wavelength region of the sensitivity is the green pixel G, The light wavelength region of the sensitivity of the red pixel R and the blue pixel B is included.

  The focus detection pixels 311 are arranged densely in a straight line without gaps in the rows or columns where the B filters and G filters of the imaging pixels 310 at the focus detection positions G1 to G5 shown in FIG. In this way, when the pixel signal at the position of the focus detection pixel 311 is calculated by pixel interpolation, it can be made inconspicuous to the human eye even if a slight error occurs. This is because the human eye is more sensitive to red than blue, and the density of green pixels is higher than that of blue and red pixels, so the contribution to image degradation for a single pixel defect in green pixels is small. .

  In an image sensor in which the image pickup pixels of the complementary color filter are developed two-dimensionally, the focus detection pixels 311 are arranged at pixel positions where cyan and magenta including a blue component whose output error is relatively inconspicuous should be arranged.

  In the imaging pixel 310, the microlens 10 is disposed in front of the photoelectric conversion unit 11 for imaging (the direction in which the optical system is located), and the photoelectric conversion unit 11 is projected forward by the microlens 10. The photoelectric conversion unit 11 is formed on a semiconductor circuit substrate (not shown) on the image sensor 211, and a color filter (not shown) is disposed between the microlens 10 and the photoelectric conversion unit 11.

  In the focus detection pixel 311, the microlens 10 is disposed in front of the focus detection photoelectric conversion units 12 and 13 (the direction in which the optical system is located), and the photoelectric conversion units 12 and 13 are projected forward by the microlens 10. . The photoelectric conversion units 12 and 13 are formed on a semiconductor circuit substrate (not shown) on the image sensor 211.

  A focus detection method based on the pupil division method will be described with reference to FIG. In FIG. 4, the microlens 50 of the focus detection pixel 311 disposed on the optical axis 91 of the interchangeable lens 202, a pair of photoelectric conversion units 52 and 53 disposed behind the microlens 50, and the interchangeable lens 202 The focus detection method will be described by taking the micro lens 60 of the focus detection pixel 311 arranged outside the optical axis 91 and the pair of photoelectric conversion units 62 and 63 arranged behind the micro lens 60 as an example. The exit pupil 90 of the interchangeable lens 202 is set at a position of a distance d4 in front of the microlenses 50 and 60 arranged on the planned imaging plane of the interchangeable lens 202. The distance d4 is a value determined according to the curvature and refractive index of the microlenses 50 and 60, the distance between the microlenses 50 and 60 and the photoelectric conversion units 52, 53, 62 and 63, and the like in this specification. This is called the distance measurement pupil distance.

  The microlenses 50 and 60 are arranged on the planned image plane of the interchangeable lens 202, and the microlens 50 on the optical axis 91 causes the pair of photoelectric conversion units 52 and 53 to be shaped from the microlens 50 by a projection distance d4. Projected onto the spaced exit pupil 90, the projection shape forms distance measuring pupils 92 and 93. The microlenses 60 outside the optical axis 91 project the shapes of the pair of photoelectric conversion units 62 and 63 onto the exit pupil 90 separated by the projection distance d4, and the projection shapes form distance measuring pupils 92 and 93. That is, the projection direction of each pixel is determined so that the projection shape (ranging pupils 92 and 93) of the photoelectric conversion unit of each focus detection pixel matches on the exit pupil 90 at the projection distance d4.

  The photoelectric conversion unit 52 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 72 passing through the distance measuring pupil 92 and traveling toward the microlens 50. The photoelectric conversion unit 53 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 73 passing through the distance measuring pupil 93 and traveling toward the microlens 50. The photoelectric conversion unit 62 outputs a signal corresponding to the intensity of the image formed on the microlens 60 by the focus detection light beam 82 passing through the distance measuring pupil 92 and traveling toward the microlens 60. The photoelectric conversion unit 63 outputs a signal corresponding to the intensity of the image formed on the microlens 60 by the focus detection light beam 83 passing through the distance measuring pupil 93 and traveling toward the microlens 60. The arrangement direction of the focus detection pixels 311 is made to coincide with the division direction of the pair of pupil distances.

  A large number of such focus detection pixels are arranged in a straight line, and the output of the pair of photoelectric conversion units of each pixel is grouped into an output group corresponding to the distance measurement pupils 92 and 93, whereby the pair of distance measurement pupils 92 and 93 is formed. Information regarding the intensity distribution of a pair of images formed on the focus detection pixel row by the focus detection light fluxes that pass through each can be obtained. Further, by performing image shift detection calculation processing (correlation processing, phase difference detection processing) described later on this information, it is possible to detect the image shift amounts of a pair of images in the so-called pupil division method. By multiplying this image shift amount by a predetermined conversion coefficient, the deviation of the current imaging plane (imaging plane at the focus detection position corresponding to the position of the microlens array on the planned imaging plane) with respect to the planned imaging plane ( Defocus amount) can be calculated.

  In FIG. 4, the first focus detection pixel (including the microlens 50 and the pair of photoelectric conversion units 52 and 53) on the optical axis 91 and the first focus detection pixel (the microlens 60 and the pair of photoelectric conversions) adjacent to the first focus detection pixel. In the other focus detection pixels, similarly, a pair of photoelectric conversion units receive light beams coming from the pair of distance measurement pupils to the respective microlenses.

  FIG. 5 is a front view showing a projection relationship on the exit pupil plane. In FIG. 4, the circumscribed circles of the distance measuring pupils 92 and 93 obtained by projecting the pair of photoelectric conversion units 12 and 13 from the focus detection pixel 311 onto the exit pupil plane 90 by the microlens 10 are predetermined when viewed from the imaging plane. An aperture F value (hereinafter referred to as a distance measuring pupil F value). The predetermined aperture F value is, for example, F2.8. When the photoelectric conversion unit 11 of the imaging pixel 310 is projected onto the exit pupil plane 90 by the microlens 10, a region 94 shown in FIG. 5 is obtained, and the region 94 is a wide region including the distance measuring pupils 92 and 93.

  In FIG. 5, the positional relationship between the center of the region 95 corresponding to the aperture opening of the interchangeable lens 202 indicated by the broken line and the center of the circumscribed circle of the distance measuring pupils 92 and 93 is the position of the exit pupil unique to the interchangeable lens 202. It changes depending on the position of the focus detection pixel 311 on the imaging screen G, that is, the distance from the optical axis, and does not necessarily match. When the size of the exit pupil of the interchangeable lens 202 is smaller than the size of the circumscribed circle of the distance measuring pupils 92 and 93 and the centers of the exit pupil and the circumscribed circle do not coincide with each other, the pair of distance measuring pupils 92 and 93 are passed. Therefore, the light flux of the pair of images formed by these light fluxes does not match, resulting in distortion.

  FIG. 6 shows the intensity distribution (light quantity) of the image signal at one focus detection position on the vertical axis and the position deviation within the focus detection position on the horizontal axis. The position deviation within the focus detection position corresponds to the positions of a plurality of focus detection pixels belonging to one focus detection position on the image sensor 211 shown in FIG. 3, for example. A pair of image signals 400 and 401 in the case where vignetting is not generated in the focus detection light beam are obtained by simply shifting the same image signal function horizontally to each other as shown in FIG. When vignetting occurs in the focus detection light beam, the amount of the focus detection light beam passing through the distance measuring pupil changes depending on the focus detection position and the position deviation within the focus detection position. For this reason, the pair of image signals 402 and 403 are as shown in FIG. 6B, and the same signals are not relatively shifted.

<Imaging operation>
FIG. 7 is a flowchart showing the imaging operation of the digital still camera 210. The camera drive control device 212 repeatedly executes this imaging operation when the camera is turned on in step S100. In step S <b> 110, the camera drive control device 212 thins out and reads out the data of the imaging pixel 310 from the imaging device 211 and displays it on the electronic viewfinder LCD 215. When the data of the imaging pixel 310 is thinned and read out, the display quality can be improved by performing the thinning out reading with a setting that does not include the focus detection pixel 311 as much as possible. Conversely, by performing thinning readout so as to include the focus detection pixel 311 and displaying it on the electronic viewfinder LCD 215 without correcting the focus detection pixel output, the focus detection position can be displayed recognizable to the user. .

  In step S120, the camera drive control device 212 reads data from the focus detection pixel array. When an arbitrary focus detection position is selected in advance from the focus detection positions G1 to G5 shown in FIG. 2 by a focus detection position selection operation member (not shown), focus detection corresponding to the selected focus detection position is performed. Data is read from the pixel column. In the subsequent step S130, the camera drive control device 212 performs an image shift detection calculation process, that is, a correlation calculation process, which will be described later, based on a pair of image data corresponding to the focus detection pixel column, calculates an image shift amount, Calculate the focus amount. In step S140, the camera drive control device 212 checks whether or not the focus is close, that is, whether or not the calculated absolute value of the defocus amount is within the focus determination reference value.

  If it is determined in step S140 that the camera drive control device 212 is not near the focus, the process proceeds to step S150. In step S150, the camera drive control device 212 transmits the calculated defocus amount to the lens drive control device 209, and drives the focusing lens 207 of the interchangeable lens 202 to the in-focus position via the lens drive control device 209. This process returns to step S110 and the above operation is repeated. Even when the focus cannot be detected, this process branches to step S150, and the camera drive control device 212 transmits a scan drive command to the lens drive control device 209, and the interchangeable lens 202 is transmitted via the lens drive control device 209. The focusing lens 207 is scan-driven between the infinite position and the closest position, and the process returns to step S110 and the above operation is repeated.

  If it is determined in step S140 that the camera drive control device 212 is near the in-focus state, the process proceeds to step S160. In step S160, the camera drive control device 212 determines whether or not a shutter release has been performed by operating a release button (not shown). If not, the process returns to step S110 and the above operation is repeated. When the shutter release is performed, the process proceeds to step S170, and the camera drive control device 212 transmits an aperture adjustment command to the lens drive control device 209, and the aperture 208 of the interchangeable lens 202 is set via the lens drive control device 209. Then, the control F value determined by the exposure calculation by the camera drive control device 212 or the F value manually set by the user is set. When the aperture control is finished, the camera drive control device 212 causes the imaging device 211 to perform an imaging operation, and reads image data from the imaging pixel 310 and all the focus detection pixels 311 of the imaging device 211.

  In step S <b> 180, the camera drive control device 212 interpolates the pixel data at each pixel position in the focus detection pixel row based on the data of the focus detection pixel 311 and the data of the surrounding imaging pixels 310. In step S190, the camera drive control device 212 stores the image data including the data of the imaging pixel 310 and the interpolated focus detection pixel position data in the memory card 213, and the process returns to step S110 and the above operation is repeated. It is.

<Focus detection operation>
Prior to the detailed description of the focus detection operation in step S130 of FIG. 7, when the identity of a pair of image signals is broken as shown in FIG. Whether the image shift amount of the image signal is accurately detected will be briefly described by taking the technique disclosed in Japanese Patent Application Laid-Open No. 2007-333720 as an example.

A pair of image signals whose identity is lost due to different distortions are divided into a first signal data string A (data A _{1 to} A _N , N is the number of data) and a second signal data string B (data B _{1 to} B _N ). Then, as disclosed in equation (8) of the specification of Japanese Patent Application Laid-Open No. 2007-333720, in the prior art, the integrated correlation value C (k) at the shift amount k is obtained by the following equation (1). ing. Data A _{1 to} A _N correspond to signal values output from N focus detection pixels included in one of a pair of focus detection pixel arrays arranged at a predetermined pixel pitch, and data B _{1 to} B _N corresponds to the signal value of the image signal output from each of the N focus detection pixels included in the other of the pair of focus detection pixel columns.
C (k) = Σ | (A _n × B _{n + 1 + k} ) − (B _{n + k} × A _{n + 1} ) | (1)

In equation (1), the Σ operation represents the integration for n, and the range that n takes in the integration is limited to the range in which the data A _n , A _{n + 1} , B _{n + k} , B _{n + 1 + k} exist according to the shift amount k. The data _An and A _{n + 1} are respectively obtained by the nth and (n + 1) th adjacent focus detection pixels 311 out of the N focus detection pixels 311 included in one of the pair of focus detection pixel columns described above. This corresponds to the signal value of the output image signal. The data B _{n + k} and B _{n + 1 + k} are in a positional relationship of the N focus detection pixels 311 included in the other of the above-described pair of focus detection pixel columns and separated from the nth focus detection pixel 311 by the shift amount k. Corresponds to the signal values of the image signals respectively output by the (n + k) th and (n + 1 + k) th adjacent focus detection pixels 311.

FIG. 8 shows a data flow of the correlation calculation of equation (1). Each of the first signal data string A and the second signal data string B is a one-dimensional data array. And the attention data A _n in the first signal data string A, in the second signal data string B which the amount of shift k, data position is shifted one with respect to data B _{n + k} corresponding to the target data A _n and multiplying the data _{B n + 1 + k,} the first operation data _{(an × B n + 1 +} k), i.e., obtaining a first product data. At the same time, the data _{B n + k} of the second signal data string B, with respect to the attention data _{A n} of the first signal data string A is multiplied by the data _{A n + 1} that data position is shifted one second operation data _{(B n + k} × A _{n + 1} ), that is, second product data is obtained. An integrated correlation value C (k) between the first signal data string A and the second signal data string B is calculated based on the sum of absolute values of differences between the first calculation data and the second calculation data. That is, in the equation (1), the local correlation value corresponding to n and k is obtained by subtracting the difference data ((A _n × B) by multiplying the product of A _n and B _{n + 1 + k} and the product of B _{n + k} and A _{n + 1. n + 1 + k} ) − (B _{n + k} × A _{n + 1} )) is calculated as an absolute value, and the integrated correlation value C (k) is calculated by integrating the local correlation values thus calculated for n.

The local correlation value in the above-described equation (1) is between the partial array (A _n , A _{n + 1} ) in the first signal data string A and the partial array (B _{n + k} , B _{n + 1 + k} ) in the second signal data string B. It can be considered as a local correlation value. The positional relationship between the elements A _n and A _{n + 1} of the partial array adjacent to each other is equal to the positional relationship of the elements B _{n + k} and B _{n + 1 + k} of the partial array adjacent to each other. It becomes. Further, the positional relationship between the element A _n of the partial array and the element B _{n + k} of the partial array is k data in terms of the data interval.

  By performing the correlation calculation as in equation (1), even when the pair of image signals are subjected to different distortion changes, the accurate image shift amount of the pair of image signals before being subjected to the distortion change is calculated. Is possible.

However, in the correlation calculation as in equation (1), since multiplication of a pair of image signal data is used, there are the following problems.
1. In general, the calculation time of the multiplication operation is longer than that of the addition / subtraction operation, so that the focus detection response is lowered.
2. Since the data bit length of the operation result is double that of the original data due to the multiplication operation of the two data, the processing system of the operation result becomes complicated and the processing time becomes longer.

  According to the present invention, even when a pair of image signals have undergone different distortion changes, the advantage of the prior art that it is possible to calculate an accurate image shift amount of a pair of image signals before undergoing the distortion change. The above-mentioned problems can be solved while maintaining. Hereinafter, the description will proceed to the correlation calculation using the logarithmic transformation used in the present invention.

In the equation (1), when the first calculation data (A _n × B _{n + k} ) and the second calculation data (B _{n + k} × A _{n + 1} ) are equal, the correlation is high, that is, the first calculation data and the second calculation data Assuming that the difference is 0, a local correlation value is calculated. Therefore, after the same function conversion is performed on the first calculation data and the second calculation data, the local correlation value may be calculated assuming that the correlation is high when the converted first calculation data and the second calculation data are equal. It will be. By performing this function conversion, if the multiplication operation can be changed to the addition / subtraction operation, the operation time and the bit length can be shortened. A logarithmic function can be considered as a function for performing such characteristic conversion. That is, when logarithmic conversion of multiplication of a plurality of data is performed, the sum of logarithmic conversions of each data is obtained, and therefore, a correlation operation expression using logarithmic conversion corresponding to the expression (1) is expressed by the following expression (2). In the equation (2), Log () represents a natural logarithm, but the base of the logarithm is not limited to the Napier number.
C (k) = Σ | Log (A _n × B _{n + 1 + k} ) −Log (B _{n + k} × A _{n + 1} ) |
= Σ | (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log (A _{n + 1} )) | (2)

(2) Multiplication can be converted into addition and subtraction by logarithmic transformation as shown in equation (2). That is, in the equation (2), the local correlation value | (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log (A _{n + 1} )) | is the logarithmic transformation data constituting the third signal data sequence LA. Log (A ₁ ) to Log (A _N ), local partial arrays (Log (A _n ), Log (A _{n + 1} )) and logarithmically transformed data Log (B ₁ ) constituting the fourth signal data sequence LB ) To Log (B _N ), local partial sequences (Log (B _{n + k} ), Log (B _{n + 1 + k} )) corresponding to the local partial sequences (Log (A _n ), Log (A _{n + 1} )) It can be calculated only by the addition / subtraction operation between. By this addition / subtraction operation, the first sum data (Log (A _n ) + Log (B _{n + 1 + k} )) and the second sum data (Log (B _{n + k} ) + Log (A _{n + 1} )) are calculated, and the first sum data (Log ( A _n ) + Log (B _{n + 1 + k} )) and the second sum data (Log (B _{n + k} ) + Log (A _{n + 1} )) are subtracted from the difference data ((Log (A _n ) + Log (B _{n + 1 + k} )) − (Log ( B _{n + k} ) + Log (A _{n + 1} ))) is calculated. The local correlation value is calculated by taking the absolute value of the difference data ((Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log (A _{n + 1} ))), and the calculated local correlation value is calculated. An integrated correlation value C (k) is calculated by integrating n.

The logarithmic transformation data Log (A _n ), Log (B _{n + 1 + k} ), Log (B _{n + k} ), and Log (A _{n + 1} ) used in the equation (2) need to be calculated by performing logarithmic transformation each time when calculating the local correlation value. Absent. Before the calculation of the expression (2), for example, when image data corresponding to the first signal data sequence A and the second signal data sequence B, which are a pair of image signals, is read from the image sensor 211, the first A logarithmic data string obtained by logarithmically converting the signal data string A and the second signal data string B can be prepared. As a result, the calculation amount of the equation (2) can be greatly reduced compared to the equation (1), and the calculation time can be shortened.

For example, image shifting is performed 100 times (k = 0 to 100), the number of data of the first signal data string A is 101 (n = 1 to 100, n + 1 = 2 to 101), and the number of data of the second signal data string B is 201. As (1 ≦ n + k ≦ 200, 2 ≦ n + 1 + k ≦ 201), every time the shift amount k takes each value from 0 to 100, the calculation amount when the integrated correlation value is calculated using the equation (1) and ( 2) When the calculation amount of the integrated correlation value is calculated using the equation, the calculation amount is compared as follows. In the following comparison, it is considered that the expression (1) includes two multiplications and the expression (2) includes three additions / subtractions. Compared with multiplication, addition / subtraction has a significantly shorter calculation time. Therefore, compared with calculation of the integrated correlation value according to equation (1), calculation of the integrated correlation value according to equation (2) is 20000 × (multiplication time−addition / subtraction time) − The calculation time is shortened by 302 × logarithmic conversion time.
(1) Formula: Number of multiplications 2 × 100 × 100 = 20000 times
Number of subtractions 100 × 100 = 10000 times (2) Formula: Logarithmic conversion times 101 + 201 = 302 times
Number of additions / subtractions 3 × 100 × 100 = 30000 times

FIG. 9 shows a data flow of the correlation calculation of equation (2). As described above, (2) and the third signal data string LA consisting of converting data LA _n for each data A _n log transformed in advance the first signal data string A before performing the calculation of equation, the second signal data string A fourth signal data string LB composed of converted data LB _n obtained by logarithmically converting each data B _n of B is prepared.

Third signal data and the attention data LA _n in the column LA, the shift amount k shifted by the fourth signal data string data data position is shifted one for data LB n _{+ k} corresponding to the target data LA _n in LB Add LB _{n + 1 + k} . From the addition result, the sum of the data LB _{n + k} of the fourth signal data sequence LB and the data LA _{n + 1} whose data position is shifted by one from the data of interest LA _{n of} the third signal data sequence LA is subtracted. By summing up the absolute values of the subtraction results for n, the integrated correlation value C (k) in the shift amount k between the third signal data string LA and the fourth signal data string LB is calculated. That is, in equation (2), the local correlation value corresponding to n and k is calculated as the absolute value of the addition / subtraction result between LA _n and LB _{n + 1 + k} , LB _{n + k} and LA _{n + 1} which are logarithmic transformation data, The accumulated correlation value C (k) is calculated by accumulating the correlation values for n.

  In the equation (2), instead of performing logarithmic conversion by function calculation, logarithmic conversion lookup table is used to further increase the logarithmic conversion time. In this case, the calculation time is further shortened. The

Further, the expression (2) can be modified as follows.
C (k) = Σ | (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log (A _{n + 1} )) |
= Σ | (Log (A _n ) −Log (A _{n + 1} )) − (Log (B _{n + k} ) −Log (B _{n + 1 + k} )) | (3)

In the expression (3), the first term (Log (A _n ) −Log (A _{n + 1} )) in the absolute value calculation is calculation data consisting only of the first signal data string A, and the second term (Log (B _{n + k).} ) −Log (B _{n + 1 + k} ) is operation data consisting only of the second signal data string B. Prior to the operation of the expression (3), n is sequentially changed (Log (A _n ) −Log (A _{n + 1} )). If the data sequence of ( ₁ ) and the data sequence of (Log (B _n ) −Log (B _{n + 1} )) are calculated in advance as intermediate processing data, the amount of computation can be further reduced and the computation time can be shortened as follows. That is, compared with the equation (1), the equation (3) can shorten the calculation time by 20000 × multiplication time−302 × logarithmic conversion time−300 × addition / subtraction time.
(3) Formula: Logarithmic conversion count 101 + 201 = 302 times
Intermediate processing count (addition / subtraction count) 100 + 200 = 300 times
Number of additions / subtractions 100 × 100 = 10000 times

FIG. 10 shows a data flow of the correlation calculation of equation (3). For each data LA _{n of} the third signal data string LA in advance, a fifth signal data string RA consisting of intermediate processing data RA _n (= LA _n −LA _{n + 1} ) and each data LB _n of the fourth signal data string LB A sixth signal data string RB including intermediate processing data RB _n (= LB _n −LB _{n + 1} ) is prepared.

Data RB _{n + k} corresponding to the attention data RA _n in the sixth signal data string RB shifted by the shift amount _k is subtracted from the attention data RA _{n in} the fifth signal data string RA. The integrated correlation value C (k) in the shift amount k between the fifth signal data string RA and the sixth signal data string RB is calculated by summing up the absolute values of the calculation results for n. That is, in equation (3), the local correlation value corresponding to n and k is calculated as the absolute value of the difference between the intermediate processing data RA _n and RB _{n + k,} and the calculated local correlation value is integrated over n to calculate the integrated correlation. A value C (k) is calculated.

  FIG. 11 is a graph showing the relationship of logarithmic transformation when the horizontal axis is input x (0 to 255) and the vertical axis is output y (up to 255), and is the logarithmic transformation T0 that is the simplest logarithmic transformation. = 255 × Log (x) / Log (255). In the case of such logarithmic transformation T0, the smaller the data value of the input x, the larger the first derivative for the input x of the logarithmic transformation T0. Since the image sensor data is shot noise, the smaller the data value, the worse the S / N ratio. Therefore, when logarithmic transformation T0 is performed, noise in a region where the data value is small may increase. Therefore, it is preferable to reduce the primary differential coefficient in a range where the input data value is small. When the input data value range for image shift detection corresponds to a value range where the data value is medium to large, the above-described primary differential coefficient may be small. It is preferable that the resolution is improved and the calculation accuracy is improved by increasing the primary differential coefficient. In the case of logarithmic transformation T0, it is preferable to prevent this because the output data diverges infinitely when the input data is 0.

  In order to reduce the primary differential coefficient in the portion where the input data value is small, the logarithmic transformation T1 = 255 × (Log (x) −Log (10)) / (Log (255) −Log (10) shown in FIG. )) May be performed.

  The minimum input data value at which the S / N ratio of the input data is greater than or equal to a predetermined value is x0, for example 10, and logarithm conversion is performed so that the maximum value of the input data range becomes the maximum value of the output data range. The output data corresponding to the data may be clipped to 0 as invalid data. In this case, unlike the logarithmic transformation T1, when the input data value is smaller than 10, the output data is 0 without becoming a large value with a minus sign. Therefore, it is not difficult to match the bit length of the input data and the bit length of the output data. There is no possibility that the output data diverges infinitely when the input data is 0.

For example, when the input data x is 8 bits (0 to 255) and the output data y is 8 bits (0 to 255), the logarithmic conversion relationship between the input data x and the output data y is as follows.
When x> x0: y = 255 × (Log (x) −Log (x0)) / (Log (255) −Log (x0)) When valid data x ≦ x0: y = 0 invalid data
... (4)

  If, for example, x0 = 10 in the equation (4), a logarithmic transformation T1 'obtained by clipping the logarithmic transformation T1 by 0 is obtained. FIG. 12 is a graph when the signal waveform I0 of the input data string Xi (40> minimum value> 10, maximum value ≦ 255) is converted into the signal waveform I1 of the output data string Yi using the logarithmic transformation T1, and the input data The value range of the output data and the value range of the output data are almost the same, and no 0 clip occurs in the output data.

  For example, when the value of input data is 40 or less and the amount of noise is large, x0 = 40 in equation (4), so that logarithmic transformation T2 in which the input data is 40 or less and the output data is clipped 0 (FIG. 11). Logarithmic conversion is performed using. FIG. 13 is a graph when the signal waveform I0 of the same input data string Xi as that of FIG. 12 is converted into the signal waveform I2 of the output data string Yi using logarithmic conversion T2, and when the value of the input data is 40 or less. The output data is clipped by 0 (see a part S2 of the signal waveform I2 in FIG. 13).

By adopting the logarithmic transformation of the formula (5) instead of the formula (4), when the range of the input data is narrow, the range of the output data can be expanded to reduce the error in the logarithmic transformation. .
When x> x0: y = M × (Log (x) −Log (x0)) / (Log (M) −Log (x0)) When valid data x ≦ x0: y = 0 invalid data
... (5)

  For example, if x0 = 40 and M = 400 in the equation (5), the logarithmic transformation T3 of FIG. 11 is obtained. In FIG. 14, the signal waveform I3 of the input data string Xi (minimum value> 50> x0, maximum value <180 <255, range width about 100) is converted into the signal waveform I4 of the output data string Yi using logarithmic transformation T3. It is a graph of the case. As shown in FIG. 14, the width of the value range of the signal waveform I4 is expanded to a value exceeding 150, while the width of the value range of the signal waveform I3 is about 100.

  As described above, logarithmic transformations (clip values, input / output value ranges) with different characteristics are prepared, and appropriate logarithmic transformations are selected according to the characteristics of input data (minimum value, maximum value, data value histogram, etc.) It is useful to use as

  In addition, the SN ratio of input data due to thermal noise changes according to the charge accumulation time of the image sensor, or the SN ratio of input data due to shot noise changes according to the sensitivity setting of the image sensor. An appropriate logarithmic transformation may be selected and used according to the above. When the charge accumulation time is long or the sensitivity setting is high, x0 is increased, for example, the clip is made different according to the charge accumulation time or sensitivity setting.

  For example, the following points are preferably taken into account when logarithmic conversion is performed by converting some data as shown in equation (4) to invalid data. That is, with respect to the first signal data sequence LA and the second signal data sequence LB generated by performing such logarithmic conversion on the first signal data sequence A and the second signal data sequence B, When the integration value of local correlation values is calculated to obtain the integrated correlation value C (k), if invalid data is included in the data used for the calculation of the local correlation value, the following invalidation processing is performed. It is preferable to eliminate the possibility that an error will occur in the value of the integrated correlation value C (k) and the accuracy of the image shift detection will be reduced.

  FIGS. 15B and 15D are graphs showing a pair of input signal waveforms on which correlation calculation is performed. The vertical axis indicates the data value, and the horizontal axis indicates the data position. FIGS. 15A, 15C, and 15E are graphs showing the calculation results of the correlation calculation. The vertical axis represents the integrated correlation value C (k), and the horizontal axis represents the shift amount k. In FIG. 15B, the pair of input signal waveforms Ia1 and Ib1 are signal waveforms obtained by multiplying the signal waveform with the shift amount k = 0 by a different gain due to vignetting or the like, and the correlation calculation result is the shift amount k. It is expected that the highest correlation is obtained at = 0, that is, the integrated correlation value c (k) takes a minimum value. When the conventional correlation calculation shown in Equation (1) is performed on the pair of input signal waveforms Ia1 and Ib1 shown in FIG. 15B, the graph of the integrated correlation value C (k) shown in FIG. The integrated correlation value C (k) obtained is a minimum value when the shift amount k = 0.

  When logarithmic transformation is performed on the pair of input waveforms Ia1 and Ib1 in FIG. 15B using the logarithmic transformation T2 in FIG. 11, the pair of input signal waveforms Ia2 and Ib2 in FIG. A part of the data (data value is 40 or less) of the input signal waveform Ia1 becomes invalid data (data value 0) in the input signal waveform Ia2. When the correlation calculation shown in the equation (2) is performed on the input signal waveform Ia2 including invalid data and Ib2 not including invalid data in FIG. 15D, the integrated correlation value C shown in FIG. The graph of (k) is obtained, and the integrated correlation value C (k) is not a minimum value at one point of the shift amount k but becomes broad. Therefore, even if the shift amount k indicating the maximum correlation is forcibly determined, the shift amount determined by the influence of noise may have a large error and a large fluctuation. In such a case, that is, when invalid data is included in the data used for calculating the local correlation value in the integration of the local correlation value in the expression (2), the value of the local correlation value is set to 0. It is effective to invalidate as.

For example, each data An of the first signal data string A and each data B _n of the second signal data string B are logarithmically converted by logarithmic conversion T2 (x0 = 40) _expressed by the equation (4), and converted data LA _{n is obtained.} generating a fourth signal data string LB to the third signal data string LA made consisting of converted data LB _n from. Next, the local correlation value E (n) is calculated by the following formula.
When any of LA _n , LB _{n + 1 + k} , LB _{n + k} , and LA _{n + 1} is invalid data (0) E (n) = 0
In other cases, E (n) = | (LA _n + LB _{n + 1 + k} ) − (LB _{n + k} ) + LA _{n + 1} ) |
... (6)

Further, the integrated correlation value C (k) is obtained by integrating the local correlation value E (n) with respect to n as follows.
C (k) = ΣE (n) (7)

  In equation (7), when the local correlation value C (k) is integrated, the invalidated local correlation value E (n) is 0, so that it is not integrated. In this way, since the integrated correlation value C (k) is a value obtained by integrating the local correlation values calculated using only valid data, highly accurate calculation that is not affected by invalid data becomes possible.

  The input signal waveform Ia2 including invalid data generated by performing logarithmic conversion on the pair of input waveforms Ia1 and Ib1 (FIG. 15B) using the logarithmic conversion T2 (Equation (4)) does not include invalid data. When the integrated correlation value C (k) is performed with respect to Ib2 (FIG. 15D) by the expressions (6) and (7), the graph of the integrated correlation value C (k) shown in FIG. And the integrated correlation value C (k) becomes a minimum value when the shift amount k = 0.

  When invalid data is included in the data used to calculate the local correlation value described above, the method of invalidating the local correlation value as 0 uses intermediate processing data as shown in FIG. It can also be used when calculating the integrated correlation value.

For example, in FIG. 10, when calculating the fifth signal data sequence RA including the intermediate processing data RA _n (= LA _n −LA _{n + 1} ) for each data LA _n of the third signal data sequence LA, LA _n or LA _{n + 1} Is invalid data (0), the intermediate processing data RA _{n is set} to invalid data (0). The intermediate processing data _RB n for each data LBn fourth signal data string _{LB (= LB n -LB n +} 1) when calculating the sixth signal data string RB consisting of, LB _n or _{LB n + 1} of any invalid data In the case of (0), the intermediate process data RB _n is set as invalid data (0).

Next, the local correlation value F (n) is calculated by the following equation.
When either RA _n or RB _{n + k} is invalid data (0) F (n) = 0
In other cases, F (n) = | RA _n −RB _{n + k} |
... (8)

Further, the integrated correlation value C (k) is obtained by integrating the local correlation value F (n) with respect to n as follows.
C (k) = ΣF (n) (9)

  In the expression (9), when the local correlation value C (k) is integrated, the invalidated local correlation value F (n) is 0, so that it is not integrated. In this way, since the integrated correlation value C (k) is a value obtained by integrating the local correlation values calculated using only valid intermediate processing data, high-precision calculation that is not affected by invalid intermediate processing data can be performed. It becomes possible.

  When there are a large number of invalid local correlation values when the integrated correlation values are obtained by integrating the local correlation values using the expressions (6) and (7) or (8) and (9). Since the integrated correlation value becomes relatively small and the behavior is as if the degree of correlation is high, an error may occur. Such an error can be prevented by normalizing the integrated correlation value in accordance with the number of integrations of the effective local integrated value used in calculating the integrated correlation value.

For example, in equation (7), when the total number of integrations is Na and the number of invalid local correlation values is Nb, the effective number of local integration values is (Na−Nb). The accumulated correlation value C (k) obtained by the equation is normalized to the accumulated correlation value D (k) as in the following equation (10).
D (k) = C (K) × Na / (Na−Nb) (10)

  In the equation (10), the integrated correlation value C (k) is normalized and increased as the number of invalid local correlation value integrations increases, that is, as the number of effective local correlation value integrations decreases. Such an error can be prevented.

  In the correlation calculation device according to the present embodiment, the correlation calculation process using the logarithmic transformation described above is performed. The correlation calculation process using logarithmic transformation has an advantage that the calculation time can be shortened compared to the correlation calculation process using multiplication. If the data value is within an appropriate level, the calculation accuracy can be maintained. In this correlation calculation device, if the data value is not at an appropriate level and some data is invalidated during logarithmic conversion, the calculation accuracy does not deteriorate according to the ratio of the number of invalid data to the total number of data. The correlation calculation processing to be performed preferably includes a first correlation calculation using multiplication and a second correlation calculation using logarithmic transformation including addition corresponding to the multiplication. Furthermore, it is preferable that the first and second correlation calculations are appropriately selected and used according to the situation.

  FIG. 16 is a flowchart showing details of the focus detection operation in step S130 in the imaging operation shown in FIG.

In step S300, the camera drive control device 212 starts focus detection calculation processing. In step S301, the camera drive control device 212 performs the following (11) with respect to a pair of signal data sequences (α _{1 to} α _M , β _{1 to} β _M : M is the number of data) output from the focus detection pixel sequence. ) Apply high frequency cut filter treatment as shown in the equation. Thus, the camera drive control device 212 generates the first signal data string A (data A _{1 to} A _N : N is the number of data) and the second signal data string B (data B _{1 to} B _N ), and the signal data string Noise components and high-frequency components that adversely affect correlation processing are removed. In the formula (11), n = 1 to N.
An = α _n + 2 × α _{n + 1} + α _{n + 2} ,
_{Bn = β n + 2 × β} n + 1 + β n + 2
(11)

  Note that the high-frequency cut filter processing in step S301 can be omitted if the calculation time is to be shortened or if it has already been greatly defocused and it is known that there are few high-frequency components.

  In step S302, the camera drive control device 212 checks whether both the average value of the first signal data string A and the average value of the second signal data string B exceed a predetermined threshold. If the predetermined threshold value is exceeded, the process branches to the second correlation calculation in step S303, and the calculation time is shortened by the correlation calculation by logarithmic transformation. If the predetermined threshold value is not exceeded, the process branches to the first correlation calculation in step S304, and the calculation accuracy is maintained by the correlation calculation by multiplication. The predetermined threshold described above can be set to 0.1 × MaxD, for example, when the data range of both the first signal data string A and the second signal data string B is 0 to MaxD.

  Note that the determination condition in the branch determination process in step S302 is not limited to the determination condition based on the average value as described above, and may be another determination condition. If the calculation accuracy by the second correlation calculation process can be ensured according to the factors affecting the data characteristics and noise characteristics of the first signal data string A and the second signal data string B, the second correlation calculation process is performed. If the selection is intended to shorten the calculation time and it is difficult to ensure the calculation accuracy by the second correlation calculation process, it is sufficient that the calculation accuracy can be ensured by selecting the first correlation calculation process. For example, the number of data that falls below a predetermined threshold is counted, and if the ratio of the counted number of data to the total number of data exceeds a predetermined value, the process branches to the first correlation calculation process. You may make it branch to a process. Further, the field luminance is measured by a photometric sensor (not shown), and if the measured luminance value exceeds a predetermined value, the process branches to the second correlation calculation process, and if not, the process branches to the first correlation calculation process. It may be. Alternatively, depending on the sensitivity setting of the image sensor, if the sensitivity exceeds a predetermined value, the process may branch to the first correlation calculation process, and if not, the process may branch to the second correlation calculation process.

  In step S303, the camera drive control device 212 performs second correlation calculation (correlation calculation processing by logarithmic transformation) shown in steps S310 to S399 of FIG. 17 described later on the first signal data string A and the second signal data string B. To calculate the defocus amount at the focus detection position, and the process proceeds to step S305.

  In step S304, the camera drive control device 212 performs a first correlation calculation (correlation calculation of equation (1)) on the first signal data string A and the second signal data string B to determine the defocus amount at the focus detection position. Then, the process proceeds to step S305.

  In step S305, the camera drive control device 212 ends the focus detection calculation process (correlation calculation process) and returns.

FIG. 17 is a flowchart showing details of the second correlation calculation process (correlation calculation process by logarithmic transformation) in step S303 of FIG. In step S310, the camera drive control device 212 starts the second correlation calculation process. In step S320, the camera drive control device 212 performs the data A _{1 to} A _N of the first signal data sequence A and the second signal data sequence B. The data value distribution of the data B _{1 to} B _N is detected. Specifically, the minimum value, maximum value, histogram mode value, etc. are detected.

  In step S330, the camera drive control device 212 performs logarithmic conversion with a plurality of logarithmic lookups that are less affected by noise and have a wider output data value range than the input data value range in accordance with the detected data value distribution. Select from the table LUT. The plurality of logarithmic lookup tables are, for example, lookup tables LUT corresponding to the above-described T1 ', T2, and T3, and are stored in the camera drive control device 212. Output data for input data below a predetermined value is invalid data.

In step S340, using the logarithmic lookup table LUT selected in step S330, the camera drive control device 212 uses the data A _{1 to} A _N of the first signal data sequence A and the data B _{1 of} the second signal data sequence B. ˜B _N are converted into each data LA _{n of} the third signal data string LA and each data LB _n of the fourth signal data string LB.

In step S350, the camera drive control device 212 first calculates data of the third signal data sequence LA in order to calculate the correlation amount between the third signal data sequence LA and the fourth signal data sequence LB at the shift amount k (integer). The data LB _n of the fourth signal data string LB is shifted relative to LA _n by a predetermined shift amount k.

  In step S360, the camera drive control device 212 performs the second correlation calculation between the data of the third signal data sequence LA and the data of the fourth signal data sequence LB shifted by the shift amount k, that is, The calculation of the local correlation value by the equation (6) and the integration of the local correlation value by the equation (7) are performed to calculate the integrated correlation value C (k) at the predetermined shift amount k.

  In step S370, the camera drive control device 212 repeats the processing of steps S350 and S360 while changing the relative shift amount k within a predetermined range, and the integrated correlation value C (k) corresponding to each shift amount k. Is calculated.

  In step S380, the camera drive control device 212 calculates a shift amount k0 that minimizes the integrated correlation value C (k) based on the integrated correlation value C (k) calculated corresponding to each shift amount k. The camera drive control device 212 uses the accumulated correlation values C (k0-1), C (k0), and C (k0 + 1) of the shift amount k0 and the shift amounts (k0-1) and (k0 + 1) before and after the shift amount k0, A shift amount km giving the highest correlation is obtained by interpolation based on a known three-point interpolation operation. If the minimum shift amount k0 is not found, or if the integrated correlation value C (k0) is larger than a predetermined threshold value, it is determined that focus detection is impossible.

In step S <b> 390, the camera drive control device 212 calculates the image shift amount represented by the product of the shift amount km and the detection pitch P, which is the pitch of the focus detection pixels, from the planned image formation on the subject image plane according to the following equation (12). It is converted into a defocus amount DEF for the surface. In equation (12), the defocus amount DEF is calculated based on the image shift amount (P × km) and the conversion coefficient G determined by the size of the opening angle of the center of gravity of the pair of distance measuring pupils.
DEF = G × P × km (12)

  In step S399, the focus detection calculation process (correlation calculation process) ends and returns.

  FIG. 17 described above is a flowchart corresponding to the data flow shown in FIG. 9 and the arithmetic processing corresponding to the equations (6) and (7). However, the data flow shown in FIG. It can be changed so as to correspond to the arithmetic processing corresponding to the equations (8) and (9). FIG. 18 shows a flowchart in which such changes are made to steps S350 to S370 in FIG.

In step S341 in FIG. 18, the camera drive control device 212 generates a fifth signal data sequence RA including the intermediate processing data RA _n calculated by addition / subtraction between the data LA _{n of} the third signal data sequence LA, A sixth signal data string RB composed of intermediate processing data RB _n calculated by addition / subtraction of each data LB _n of the signal data string LB is generated. However, when invalid data is included in any of the data used for calculating the intermediate process data, the intermediate process data is invalid data (0).

In step S351, the camera drive control device 212 first calculates data of the fifth signal data sequence RA in order to calculate the correlation amount between the fifth signal data sequence RA and the sixth signal data sequence RB at the shift amount k (integer). The data RB _n of the sixth signal data string RB is shifted relative to RA _n by a predetermined shift amount k.

  In step S361, the camera drive control device 212 performs the second correlation calculation between the data of the fifth signal data sequence RA and the data of the sixth signal data sequence RB shifted by the shift amount k, that is, (8 The local correlation value is calculated by the equation (9) and the local correlation value is integrated by the equation (9), and the integrated correlation value C (k) at the predetermined shift amount k is calculated.

  In step S371, the camera drive control device 212 repeats the processing in steps S351 and S361 while changing the relative shift amount k within a predetermined range, and the integrated correlation value C (k) corresponding to each shift amount k. Is calculated.

  FIG. 16 is an operation flowchart in the case where the focus detection calculation is performed only at the selected focus detection position. Since the correlation calculation using the logarithmic conversion data described above can reduce the correlation calculation time while maintaining the calculation accuracy, it is more effective when performing focus detection at a plurality of focus detection positions. For example, focus detection is performed at all five focus detection positions G1 to G5 on the imaging screen G in FIG. 2 to calculate the defocus amount at each focus detection position, and calculation at each of the five focus detection positions G1 to G5. The defocus amount obtained by averaging the plurality of defocus amounts acquired by the above may be calculated.

  FIG. 19 is a flowchart showing the focus detection operation described above as the focus detection operation in step S130 in the imaging operation shown in FIG. In step S400, focus detection calculation processing, that is, correlation calculation processing is started. In step S410, the camera drive control device 212 performs high-frequency cut filter processing on the pair of signal data sequences output from the focus detection pixel sequence in the same manner as in step S301 in FIG. A second signal data string B is generated.

  In step S420, the camera drive control device 212 checks whether or not the focus detection position to be processed is the selected focus detection position. If the focus detection position is the selected focus detection position, the process branches to step S440. If the focus detection position to be processed is not the selected focus detection position, the process branches to step S430. To do. Whether the focus detection position to be processed is the selected focus detection position is, for example, whether the focus detection position to be processed is a focus detection position designated by the user via a user input device (not shown). It is decided according to whether or not. Alternatively, based on the estimation that the subject image is included in the focus detection position G1 arranged at the center among the five focus detection positions G1 to G5 on the imaging screen G in FIG. May be determined according to whether or not the focus detection position G1.

  In step S430, the camera drive control device 212 performs the second correlation calculation (correlation calculation process by logarithmization) shown in steps S310 to S399 of FIG. 17 on the first signal data string A and the second signal data string B. The defocus amount at the focus detection position is calculated, and the process proceeds to step S450.

  In step S440, the camera drive control device 212 performs a first correlation calculation (correlation calculation of equation (1)) on the first signal data string A and the second signal data string B, thereby defocusing the focus detection position. And the process proceeds to step S450.

  In step S450, the camera drive control device 212 repeats the processes in steps S420 to S440 for the five focus detection positions in FIG.

  In step S460, the camera drive control device 212 performs a weighted average process on the defocus amounts of the five focus detection positions to calculate a final defocus amount. In the weighted averaging process, the defocus amount result at the selected focus detection position is obtained by applying a relatively larger weight to the defocus amount at the selected focus detection position than the defocus amount at the other focus detection position. Preferential treatment.

  In step S470, the focus detection calculation process (correlation calculation process) ends and returns.

  In this way, the selected focus detection position can be reliably detected by the first correlation calculation in which the calculation accuracy is ensured, and at the other plurality of focus detection positions, the calculation time is calculated. Since the second correlation calculation can be performed, the calculation time for the entire focus detection operation can be shortened.

  In the focus detection operation shown in FIGS. 16 to 19, there are a first correlation calculation that can reliably detect a focus but has a long calculation time, and a second correlation calculation that has a short calculation time while ensuring calculation accuracy under a predetermined condition. It is selected appropriately according to the situation. Therefore, in the focus detection apparatus having the correlation calculation apparatus of the present embodiment, the performance as the entire focus detection operation can be improved.

The camera drive control device 212 included in the digital still camera 201 including the correlation operation device according to the present embodiment has a local partial arrangement (A _n , A _{n + 1} ) in the first signal data sequence A composed of data A _{1 to} A _N. A plurality of first signal data A forming a local partial array (A _n , A _{n + 1} ) in a second signal data sequence B composed of a plurality of data A _n and A _{n + 1} and a plurality of data B _{1 to} B _{N By} multiplying a plurality of data B _{n + k} and B _{n + 1 + k} forming a local partial array (B _{n + k} , B _{n + 1 + k} ) corresponding to _{n 1} and A _{n + 1} , a local partial array (A _n , A _{n n + 1)} forming a plurality of data _{a n, a} n _{+ 1} and the local partial sequence _(B n + _k, a plurality of data _B n + _k forming the _{B n + 1 + k),} the local between _{B n + 1 + k} Sekichi _{| (A n × B n +} 1 + k) - (B n + k × A n + 1) | is calculated.

At the same time, the camera drive control device 212 includes a plurality of data A _n , A _{n + 1,} and a local partial array (A _n , A _{n + 1} ) in the first signal data sequence A and the second signal data sequence B. The local correlation value | (A _n × B _{n + 1 + k} ) − (B _{n + k} × A _{n + 1} ) | calculated each time a plurality of data B _{n + k} and B _{n + 1 + k} forming a general partial array (B _{n + k} , B _{n + 1 + k} ) are changed As a result, the integrated correlation value C (k) = Σ | (A _n × B _{n + 1 + k} ) − (B _{n + k} × A _{n + 1} ) representing the degree of correlation between the first signal data sequence A and the second signal data sequence B. | Is calculated.

The camera drive control device 212 includes a plurality of logarithmic conversion data Log (A ₁ ) to Log (A _N ) by logarithmically converting the plurality of data A _{1 to} A _N and the plurality of data B _{1 to} B _N respectively. A fourth signal data string LB composed of the third signal data string LA and a plurality of logarithmic conversion data Log (B ₁ ) to Log (B _N ) is generated.

The camera drive control device 212 has a plurality of logarithmic transformation data Log (A _n ), Log (A _{n + 1} ) forming a local partial array (Log (A _n ), Log (A _{n + 1} )) in the third signal data sequence LA. Corresponding to a plurality of logarithmic transformation data Log (A _n ), Log (A _{n + 1} ) forming a local partial array (Log (A _n ), Log (A _{n + 1} )) in the fourth signal data string LB. Local partial array by performing addition between a plurality of logarithmic transformation data Log (B _{n + k} ) and Log (B _{n + 1 + k} ) forming a partial array (Log (B _{n + k} ), Log (B _{n + 1 + k} )) (Log (A _n ), Log (A _{n + 1} )) and a plurality of logarithmic transformation data Log (A _n ), Log (A _{n + 1} ) and local partial arrays (Log (B _{n + k} ), L local correlation value | (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) between a plurality of logarithmic transformation data Log (B _{n + k} ) and Log (B _{n + 1 + k} ) forming og (B _{n + 1 + k} )) ) + Log (A _{n + 1} )) |

At the same time, the camera drive control device 212 performs a plurality of logarithmic transformations forming a local partial array (Log (A _n ), Log (A _{n + 1} )) in the third signal data sequence LA and the fourth signal data sequence LB. A plurality of logarithmic transformation data Log (B _{n + k} ) and Log (B _{n + 1 + k} ) forming data Log (A _n ), Log (A _{n + 1} ), and local partial array (Log (B _{n + k} ), Log (B _{n + 1 + k} )) By integrating the local correlation value | (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log (A _{n + 1} )) | calculated each time the change is made, the third signal data string LA and the third signal data LA integrated correlation value representing the correlation between the 4 signal data string _{LB C (k) = Σ |} (Log (a n) + Log (B n + 1 + k)) - (Log (B n + k) _{Log (A n + 1))} | is calculated.

  The digital still camera 201 including the correlation calculation device according to the present embodiment includes such a camera drive control device 212, so that the calculation time is within a range in which the calculation accuracy is ensured in the correlation calculation for focus detection. There is an effect that can be shortened.

The camera drive control device 212 included in the digital still camera 201 including the correlation calculation device according to the present embodiment samples one intensity of a pair of images by a plurality of focus detection pixels 311 arranged at a predetermined pixel pitch. In the first signal data string A composed of a plurality of data A _{1 to} A _N obtained in this manner, a plurality of adjacent data for one data in the vicinity of the nth focus detection pixel 311 among the plurality of focus detection pixels 311. A plurality of data A _n , A _{n + 1} obtained by the focus detection pixel 311 and a plurality of data obtained by sampling the other intensity of the pair of images with the plurality of focus detection pixels 311 arranged at a predetermined pixel pitch. in the second signal data string B including data _B 1 .about.B _n, a shift amount with respect to the n-th focus detection pixels 311 among the plurality of focus detection pixels 311 Only at a position spaced relationship (n + k) th multiple data B _{n + k} obtained at a plurality of focus detection pixels 311 in one data adjacencies in the vicinity of the focus detection pixels _311, multiplied with the _B n + 1 _{+ k} To calculate the first product data (A _n × B _{n + 1 + k} ) and the second product data (B _{n + k} × A _{n + 1} ).

The camera drive control device 212 subtracts the first product data (A _n × B _{n + 1 + k} ) and the second product data (B _{n + k} × A _{n + 1} ) to obtain difference data ((A _n × B _{n + 1 + k} ) − (B _{n + k).} × A _{n + 1} )) is calculated and the absolute value of the difference data is calculated to obtain a plurality of data A _n , A _{n + 1 in} the vicinity of the nth focus detection pixel 311 and the vicinity of the (n + k) th focus detection pixel 311. The local correlation value | (A _n × B _{n + 1 + k} ) − (B _{n + k} × A _{n + 1} ) | between the plurality of data B _{n + k} and B _{n + 1 + k} is calculated.

The camera drive control device 212 moves the variable n of the nth focus detection pixel 311 from n = 1 to N to obtain the local correlation value | (A _n × B _{n + 1 + k} ) − (B _{n + k} × A _{n + 1} ) | By integrating, the integrated correlation value C (k) = Σ | (A _n × B _{n + 1 + k} ) − () in the positional relationship separated by the shift amount k between the first signal data sequence A and the second signal data sequence B. _{Bn + k} * _{An + 1} ) | is calculated.

The camera drive control device 212 performs logarithmic conversion on the data A _{1 to} A _N and the plurality of data B _{1 to} B _N , respectively, to generate a plurality of log conversion data Log (A ₁ ) to Log (A _N ) and a plurality of log conversion data. _{Log (B 1) ~Log (B} N) is calculated, and a plurality of log-transformed data _{Log (a 1) ~Log (a} N) and a third signal data string LA and a plurality of log-transformed data Log _(B 1) A fourth signal data string LB composed of ~ Log (B _N ) is generated.

In the third signal data array LA, the camera drive control device 212 has a plurality of logarithmic conversion data Log () obtained by a plurality of focus detection pixels 311 adjacent to one data in the vicinity of the nth focus detection pixel 311. A _n ), Log (A _{n + 1} ), and the fourth signal data string LB, in the vicinity of the (n + k) th focus detection pixel 311 that is in a positional relationship separated from the nth focus detection pixel 311 by the shift amount k. The first sum data (Log (B ( _{n + k} )) is added to a plurality of logarithm conversion data Log (B _{n + k} ) and Log (B _{n + 1 + k} ) obtained by a plurality of focus detection pixels 311 that are adjacent to each other. A _n ) + Log (B _{n + 1 + k} )) and second sum data (Log (B _{n + k} ) + Log (A _{n + 1} )) are calculated.

The camera drive control device 212 further subtracts the first sum data (Log (A _n ) + Log (B _{n + 1 + k} )) and the second sum data (Log (B _{n + k} ) + Log (A _{n + 1} )) to obtain difference data ( (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log (A _{n + 1} ))) is calculated.

The camera drive control device 212 takes the absolute value of the calculated second difference data ((Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log (A _{n + 1} ))) to obtain the n th Logarithm conversion data Log (A _n ), Log (A _{n + 1} ) in the vicinity of the focus detection pixel 311, and logarithm conversion data Log (B _{n + k} ), Log (in the vicinity of the (n + k) th focus detection pixel 311. B _{n + 1 + k)} local correlation value between _{| (Log (a n) +} Log (B n + 1 + k)) - (Log (B n + k) + Log (a n + 1)) | is calculated.

The camera drive control device 212 moves the variable n of the n-th focus detection pixel 311 over n = 1 to N to obtain a local correlation value | (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log (A _{n + 1} )) | by integrating the accumulated correlation value C (k) = Σ in the positional relationship separated by the shift amount k between the third signal data sequence LA and the fourth signal data sequence LB. | (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log (A _{n + 1} )) | is calculated.

The camera drive control device 212 determines that the integrated correlation value C (k) = Σ depending on whether the average value of the first signal data string A and the average value of the second signal data string B both exceed a predetermined threshold value. | (A _n × B _{n + 1 + k} ) − (B _{n + k} × A _{n + 1} ) | and the integrated correlation value C (k) = Σ | (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log Either _{one of} (A _{n + 1} )) | is calculated.

When the calculation of the integrated correlation value C (k) = Σ | (A _n × B _{n + 1 + k} ) − (B _{n + k} × A _{n + 1} ) | is selected, the camera drive control device 212 sets the shift amount k over a predetermined range. Every time the change is made, the integrated correlation value C (k) = Σ | (A _n × B _{n + 1 + k} ) − (B _{n + k} × A _{n + 1} ) | corresponding to the shift amount k is calculated. The camera drive control device 212 calculates the first correlation value C (k) = Σ | (A _n × B _{n + 1 + k} ) − (B _{n + k} × A _{n + 1} ) | An image shift amount (P × km) between the 1 signal data string A and the second signal data string B is calculated.

When the calculation of the integrated correlation value C (k) = Σ | (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log (A _{n + 1} )) | Is an integrated correlation value C (k) = Σ | (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) corresponding to the shift amount k every time the shift amount k is changed over a predetermined range. + Log (A _{n + 1} )) | The camera drive control device 212 uses the integrated correlation value C (k) = Σ | (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log (A _{n + 1} )) | Based on the value k0 of the shift amount k, an image shift amount (P × km) between the first signal data string A and the second signal data string B is calculated.

The digital still camera 201 including the correlation calculation device according to the present embodiment includes such a camera drive control device 212, so that the integrated correlation value C (k) = Σ | ( A _n × B _{n + 1 + k} ) − (B _{n + k} × A _{n + 1} ) | and the integrated correlation value C (k) = Σ | (Log (A _n ) + Log (B _{n + 1 + k} )) − (Log (B _{n + k} ) + Log (A) The calculation of _{n + 1} )) | is appropriately selected depending on the situation, and there is an effect that the reduction of the correlation calculation time and the maintenance of the correlation calculation accuracy are balanced.

<Modification of correlation calculation>
The second correlation calculation using the logarithmic transformation performed by the correlation calculation apparatus in the above-described embodiment is not limited to the expressions (2) and (3) corresponding to the expression (1). Any correlation calculation may be used as long as it is a type of correlation calculation that calculates an integrated correlation value obtained by integrating local correlation values including differences. High-speed arithmetic is realized by decomposing the multiplication into addition by logarithmic transformation. At the same time, problems such as an increase in noise due to logarithmization are solved by invalidating output data for input data that is equal to or smaller than a predetermined value during logarithmic conversion. Further, by not using local correlation values including invalidation data in the calculation of the integrated correlation value, it is possible to prevent a decrease in correlation calculation accuracy due to invalid data being included in logarithmic data.

  Below, the modification of the 2nd correlation calculation using the logarithmic transformation corresponding to the 1st correlation calculation using multiplication is given. However, for the sake of brevity, the invalidation process (0 clip process) and the adjustment calculation part of the output data value range are omitted.

(Variation 1) A correlation calculation expression obtained by generalizing the subscript number 1 to an integer s in expression (1) is shown as expression (12) as a first correlation calculation expression, and a second correlation calculation expression corresponding to this is expressed as ( 13)
C (k) = Σ | (A _n × B _{n + s + k} ) − (B _{n + k} × A _{n + s} ) | (12)
C (k) = Σ | (Log (A _n ) + Log (B _{n + s + k} )) − (Log (B _{n + k} ) + Log (A _{n + s} )) | (13)

(Modification 2) A correlation operation expression obtained by generalizing the numerical value 1 of the subscript in the expression (14) of the specification of JP-A-2007-333720 to an integer s is shown as an expression (14) as a first correlation operation expression, A second correlation calculation expression corresponding to this is shown in Expression (15).
C (k) = Σ | (A _n ² × B _{n−s + k} × B _{n + s + k} ) − (B _{n + k} ² × A _n−s × A _{n + s} ) |
(14)
C (k) = Σ | (2 × Log (A _n ) + Log (B _{n−s + k} ) + Log (B _{n + s + k} )) − (2 × Log (B _{n + k} ) + Log (A _n−s ) + Log (A _{n + s} )) ｜
... (15)

(Modification 3) The first correlation calculation formula disclosed in formula (5) of the specification of Japanese Patent Application Laid-Open No. 2009-258231 is shown as formula (16) as the first correlation calculation formula, and the second correlation corresponding to this formula The arithmetic expression is shown in Expression (17).
C (k) = Σ | (A _n × An + s + t × Bn + t + k × Bn + s + k) − (An + s × A _{n + t} × B _{n + k} × B _{n + k + s + t} ) | (16)
C (k) = Σ | (Log (A _n ) + Log (A _{n + s + t} ) + Log (B _{n + t + k} ) + Log (B _{n + s + k} ))-(Log (An + s) + Log (An + t) + Log (Bn + k) + Log + B _{+ k +} B _{+ k +} B _{+ k +} B ₊ ... (17)

(Modification 4) The above-described correlation calculation is a correlation calculation for data arranged in a one-dimensional space. However, the correlation calculation performed by the correlation calculation apparatus according to the present invention is not limited to the correlation calculation for data arranged in a one-dimensional space.

For example, two-dimensional image data A _{i, j} and B _{i, j} are represented using indices i and j representing coordinates in a two-dimensional space, and an absolute value of a difference of multiplication as in the following equation (18) is integrated. It is also possible to calculate the correlation value C (k, h) of the two-dimensional image data A _{i, j} and B _{i, j in} the relative two-dimensional image shift amount (k, h) by performing one correlation calculation. Is possible. A second correlation calculation formula for the two-dimensional image data corresponding to this is shown in Formula (19).
C (k, h) = ΣΣ | (A _{i, j} × A _{i + s, j + s} × B _{i + s + k, j + h} × B _{i + k, j + s + h} ) − (A _{i + s, j} × A _{i, j + s} × B _{i + k, j + h} × B _{i + s + k, j + s + h} ) | (18)
C (k) = ΣΣ | (Log (A _{i, j} ) + Log (A _{i + s, j + s} ) + Log (B _{i + s + k, j + h} ) + Log (B _{i + k, j + s + h} )) − (Log (A _{i + s, j} ) + Log (A _{i , j + s} ) + Log (B _{i + k, j + h} ) + Log (B _{i + s + k, j + s + h} )) |
= ΣΣ | (Log (A _{i, j} ) + Log (A _{i + s, j + s} ) −Log (A _{i + s, j} ) −Log (A _{i, j + s} )) − (Log (B _{i + k, j + h} ) + Log (B _{i + s + k, j + s +} ) −Log (B _{i + s + k, j + h} ) −Log (B _{i + k, j + s + h} )) | (19)

  FIG. 26 shows the data flow of the second correlation calculation when s = 1 in the above equation (19) and the Log calculation is simply described as L.

(Modification 5) The above-described correlation calculation is used for image shift detection for focus detection. However, the correlation calculation performed by the correlation calculation apparatus according to the present invention is not limited to the use of the image shift detection calculation of the focus detection calculation. For example, the basic image and the comparison target image which are image data arranged in a two-dimensional space It may be applied to the calculation of the correlation value C in the template matching. Calculation of the integrated correlation value C in template matching between the face image in the basic image and the comparison target image and the partial image by the control device of the correlation calculation device will be described with reference to FIGS.

  Four two-dimensional images, that is, a face image 21 in the basic image 20, a partial image 33 of the comparison target image 30, a face image 41 of the basic low-pass image 20a obtained by subjecting the basic image 20 to low-pass filter processing, The partial image 43 of the comparison target low-pass image 30a obtained by performing the low-pass filter process on the comparison target image 30 is used for calculating the integrated correlation value C.

  FIG. 21 is an image obtained by enlarging the vicinity of the face image 21 in the basic image 20, and shows the face image 21 superimposed on a two-dimensional pixel arrangement indicated by square grids. The low-pass filter is applied to the pixel data of this image by two-dimensionally scanning the vicinity of the face image 21 in units of pixels.

  FIG. 22 is an enlarged image of the vicinity of the face image of the basic low-pass image generated as a result of the low-pass filter processing as described above, and a face image 41 with a high-frequency component lost and generated. In FIG. 22, the central portion of the region 1110 including the mouth and the central portion of the region 1120 including both eyes in the face image 41 are regions having the highest density, and are away from the central portion of the region 1110 and the central portion of the region 1120. As the concentration gradually decreases. The density of the area 1130 around both eyes is lower than the density of the area 1120, the density of the area 1140 including the central part of the face is lower than the density of the area 1130, and the density of the area 1150 near the face outline outside the area 1140. The density is lower than the density in region 1140. For convenience of drawing, the face image 41 of FIG. 22 is divided into regions 1110 to 1150 and the above description has been made. However, the decrease in density in the direction from the mouth and both eyes toward the face contour is actually Is continuous.

  FIG. 23 is an image obtained by enlarging the vicinity of a partial image 33 in the comparison target image 30. In FIG. 23, a partial image 33 is shown superimposed on a two-dimensional pixel arrangement indicated by square grids. By scanning the partial image 33 two-dimensionally with a low-pass filter in units of pixels, low-pass filter processing is performed on the pixel data of this image.

  FIG. 24 is an enlarged image of the vicinity of the partial image 33 in the comparison target low-pass image generated as a result of the low-pass filter processing as described above, and a partial image 43 in which a high-frequency component is lost and blurred is generated. In FIG. 24, in the partial image 43, the central portion of the region 1410 and the central portion of the region 1430 are regions having the highest density, and the concentration gradually decreases as the distance from the central portion of the region 1410 and the central portion of the region 1430 increases. . The density of the area 1420 around the area 1410 is lower than the density of the area 1410, the density of the area 1440 around the area 1430 is lower than the density of the area 1430, and the density of the area 1450 outside the areas 1420 and 1440 is the density of the area 1420. And the density of the area 1460 outside the area 1450 is lower than the density of the area 1450. For the convenience of drawing, the partial image 43 in FIG. 24 is divided into regions 1410 to 1460, and the above description has been made. However, from the central portion of the region 1410 and the central portion of the region 1430 to the vicinity of the outer periphery of the partial image 43 as described above. The decrease in concentration in the direction toward is actually continuous.

  Four two-dimensional images, that is, a face image 21 in the basic image 20, a partial image 33 of the comparison target image 30, a face image 41 of the basic low-pass image 20a obtained by subjecting the basic image 20 to low-pass filter processing, In the partial image 43 of the comparison target low-pass image 30a obtained by performing the low-pass filter process on the comparison target image 30, the pixel position on the two-dimensional image is represented by a two-dimensional position variable in the horizontal direction and the vertical direction. . A reference pixel position is referred to as a pixel origin. The pixel origin is set at the pixel position of the pixel address (0, 0). As shown in FIG. 21, in the face image 21, the reference pixel position 22 is set as the pixel origin. As shown in FIG. 23, in the partial image 33, the reference pixel position 32 corresponding to the reference pixel position 22 of the face image 21 is set as the pixel origin. As shown in FIG. 22, in the face image 41 of the basic low-pass image 20a, the reference pixel position 22a similar to that of the face image 21 is set as the pixel origin. In the partial image 43 of the comparison low-pass image 30a, the reference pixel position 32a similar to that of the partial image 33 is set as the pixel origin, as shown in FIG.

FIG. 25 shows two-dimensional pixel data of the face image 21, the partial image 33, the face image 41 of the basic low-pass image 20a, and the partial image 43 of the comparison target low-pass image 30a when the pixel origin is determined in this way. As shown, A _xy , B _xy , E _xy , and F _xy respectively. The pixel address (x, y) represents the pixel position in the horizontal direction and the vertical direction with respect to the pixel origin described above in each of the face image 21, the partial image 33, the face image 41, and the partial image 43. 21 to 24, the face image 21, the partial image 33, the face image 41, and the partial image 43 are not rectangular images. However, in order to simplify the description, the face image 21, the partial image 33, and the face image are illustrated in FIG. 41 and the partial image 43 are rectangular images. A local correlation value D (i, j) at a specific correlation pixel position 23 when the pixel address (x, y) takes a predetermined value (i, j) is given by the equation (20).
D (i, j) = | A _ij × F _ij −B _ij × E _ij | (20)

  As expressed in the equation (20), the local correlation value D (i, j) is product data obtained by multiplying the pixel data of the face image 21 and the pixel data of the partial image 43 at a specific correlation pixel position 23. And the value calculated based on the difference between the product data obtained by multiplying the pixel data of the partial image 33 and the pixel data of the face image 41 at the same correlation pixel position 23, and the correlation between the two product data Value. FIG. 25 shows the correspondence between the expression (20), which is a calculation formula for the local correlation value D (i, j), and each pixel data used for the calculation according to the expression (20). The pixel address (x, y) = (i, j) of the correlation pixel position 23 is defined by the above-described pixel origin and the size of each of the face image 21, the partial image 33, the face image 41, and the partial image 43. It is a position variable that takes a value in the range. Here, it is assumed that the size of the face image 21 and the size of the partial image 33 are equal to each other. If the size of the face image 21 and the size of the face image 41 after the low-pass filter processing are equal to each other, and the size of the partial image 33 and the size of the partial image 43 after the low-pass filter processing are equal to each other, All match.

FIG. 26 is a diagram schematically showing the calculation of the local correlation value D (i, j) by the equation (20) at the correlation pixel position 23 of the pixel address (i, j), and the vertical axis indicates the pixel data value. The horizontal axis is the pixel position. Even when the face represented by the pixel data value A _xy of the face image 21 is the same as the face represented by the pixel data value B _xy of the partial image 33, each pixel data value A _xy and There is a level difference (magnification fluctuation) that varies locally between _Bxy . In FIG. 26, the pixel data value E of the face image 41 at the correlation pixel position 23 of the pixel address (i, j) is different from the pixel data value A _ij of the face image 21 at the correlation pixel position 23 of the pixel address (i, j). _ij is a local average value of pixel data values around W around the correlated pixel position 23 of the pixel address (i, j). In FIG. 26, the pixel data of the partial image 43 at the pixel position of the pixel address (i, j) with respect to the pixel data value B _ij of the partial image 33 at the pixel position of the same pixel address (i, j) as the correlation pixel position 23. The value F _ij is a local average value of pixel data values around W around the pixel position of the pixel address (i, j).

Accordingly performs multiplying and level correction pixel data values _{F ij} of the partial image 43 in the pixel data values _{A ij} of the face image 21, multiplied by the pixel data values _{E ij} of the face image 41 to the pixel data value _{B ij} of the partial image 33 Level correction. By performing such level correction, the level of the corrected face image data (A _ij × F _ij ) and the value of the corrected partial image data (B _ij × E _ij ) are equal to each other. . By performing a difference calculation between the corrected face image data (A _ij × F _ij ) value and the partial image data (B _ij × E _ij ) value, the two images (the face image 21 and the partial image 33) are calculated. Even when there is a level difference due to illumination fluctuation or the like, an accurate local correlation value can be calculated.

  In the conventional difference sum calculation, the difference between the pixel data of the face image in the basic image and the pixel data of the partial image in the comparison target image is calculated. In the present modification, the product of the pixel data of the face image in the basic image and the pixel data of the partial image in the comparison target low-pass image, and the pixel data of the partial image in the comparison target image in Expression (20) The difference between the product of the basic low-pass image and the pixel data of the face image is calculated. The pixel data level at each pixel position of the partial image in the comparison target low-pass image corresponds to an average pixel data level around each pixel position of the partial image in the comparison target image. The pixel data level at each pixel position of the face image in the basic low-pass image corresponds to an average pixel data level around each pixel position of the face image in the basic image. In the template matching by the control device of the correlation calculation device in the present modification, a uniform illuminance difference or relatively low frequency illumination between the face image and the partial image is calculated when the local correlation value D (i, j) is calculated. Even when there is unevenness, the data level difference is corrected for each pixel. Therefore, it is possible to calculate a correlation value having high robustness with respect to a uniform illuminance difference or relatively low frequency illumination unevenness.

  The control device of the correlation calculation device in the present modification checks whether or not the scanning of the correlation pixel position 23 in the face image 21 is completed, and if the scanning is not completed, the scanning of the correlation pixel position 23 is performed. Until the process is completed, the local correlation values sequentially calculated are accumulated each time the correlation pixel position 23 is changed by scanning.

When the scanning of the correlation pixel position 23 in the face image 21 is completed, the integrated value of the correlation value at the reference pixel position 32 represented by the pixel address (N, M) designated by the scanning of the comparison target image 30 is obtained. The calculation ends. In the present modification, the integrated value of the correlation values is used as it is as the integrated correlation value C (N, M) between the face image 21 and the comparison target image 30 in the basic image. That is, the integrated correlation value C (N, M) is obtained by the first correlation calculation expression expressed by the expression (21). However, the integration calculation ΣΣ represents the integration for the position variables i and j. Since the range of possible values of the position variables i and j coincides with the above-described range of the definition area, that is, the range of pixels in which the face image exists is limited. When the integrated correlation value C (N, M) is small, the degree of correlation between the face image 21 in the basic image and the comparison target image 30 is high.
C (N, M) = ΣΣ | A _{i, j} × F _{i, j} −B _{i, j} × E _{i, j} | (21)

As described above, the position variables i and j are indices representing the coordinates of the two-dimensional space, the two-dimensional image data E _{i, j} of the face image 41 of the basic low-pass image 20a, and the partial image 43 of the comparison low-pass image 30a. 2-dimensional image data F _{i of, j} is 2-dimensional image data a _i of the face image 21 in the basic image _{20, j,} and the two-dimensional image data B _i of the comparison image in the basic image _20, with respect to _j , Respectively, by applying a low-pass filter process. A second correlation calculation expression corresponding to the first correlation calculation expression (21) for calculating the integrated correlation value C (N, M) is represented by the expression (22).
C = ΣΣ | (Log (A _{i, j} ) + Log (F _{i, j} )) − (Log (B _{i, j} ) + Log (E _{i, j} )) | (22)

When multiplication is used to calculate the correlation value of two-dimensional data, the number of multiplication operations increases approximately in proportion to the square of the number of data, so the calculation time increases significantly. However, the second correlation operation using logarithmic transformation is performed. If used, the calculation time can be greatly shortened, and the application of the present invention to two-dimensional data is extremely effective. When the predetermined condition is satisfied, the second correlation calculation formula (22) is used in the case of the integrated correlation value C (N, M), and when the predetermined condition is not satisfied, in the case of the integrated correlation value C (N, M). The first correlation calculation formula (21) is used. For example, the average value of the two-dimensional image data A _{i, j} of the face image 21 in the basic image 20, or the data value of the two-dimensional image data A _{i, j} of the face image 21 in the basic image 20 and the basic image When the average value of the two-dimensional image data B _{i, j} of the comparison target images in 20 is equal to or greater than a predetermined value, the above-described predetermined condition is satisfied.

(Modification 6) Furthermore, the present invention can be applied to the detection of the degree of correlation of three-dimensional spatial data using the same concept as the above-described detection of the degree of correlation of two-dimensional spatial data.

(Modification 7) In the above-described correlation calculation, the local correlation value is calculated as the absolute value of the difference between the two calculation data, but is not limited to the absolute value. Any non-linear calculation for unifying the signs of the local correlation values may be used. For example, the local correlation value may be a square calculation value of a difference between two calculation data.

(Modification 8) In the imaging device 211 illustrated in FIG. 3, an example in which a pair of photoelectric conversion units 12 and 13 is provided for each focus detection pixel 311 is shown. FIG. 27 shows focus detection pixels 313 and 314 having one photoelectric conversion unit in one pixel. The focus detection pixel 313 includes the microlens 10 and the photoelectric conversion unit 16. The focus detection pixel 314 includes the microlens 10 and the photoelectric conversion unit 17. The photoelectric conversion units 16 and 17 are projected onto the exit pupil of the interchangeable lens by the microlens 10 to form distance measurement pupils 92 and 93 shown in FIG. Therefore, the focus detection pixels 313 and 314 can obtain a pair of image outputs used for focus detection. FIG. 27 shows an example of an image sensor 211B in which focus detection pixels 313 and 314 are alternately arranged in a line. The adjacent focus detection pixels 313 and 314 form a pair of focus detection pixels, and correspond to the focus detection pixels 311 of the image sensor 211 shown in FIG.

(Modification 9) In the embodiment and the modification described above, the correlation calculation device according to the present invention has been described by taking a pupil detection method using a microlens as an example. The correlation calculation device according to the present invention is not limited to the focus detection device of the above-described type, but can also be applied to a focus detection device of a re-imaging pupil division method, and the effects as described above can be achieved.

  In FIG. 28, the focus detection device of the re-imaging pupil division method to which the correlation calculation device according to the present modification is applied has an optical axis 191 of an interchangeable lens, condenser lenses 110 and 120, aperture masks 111 and 121, aperture apertures 112 and 113. , 122 and 123, re-imaging lenses 114, 115, 124 and 125, and image sensors (CCD) 116 and 126 for focus detection. The focus detection light beams 132, 133, 142, and 143 pass through an exit pupil 190 that is set at a position of a distance d5 forward from the scheduled imaging surface of the interchangeable lens. The distance d5 is a distance measurement pupil distance determined according to the focal length of the condenser lenses 110 and 120, the distance between the condenser lenses 110 and 120, and the aperture openings 112, 113, 122, and 123, and the like. The apertures 112 and 122 are projected by the condenser lenses 110 and 120 to form a region (ranging pupil) 192, and the apertures 113 and 123 are projected by the condenser lenses 110 and 120 to thereby produce a region (measurement). Distance pupil) 193 is formed.

  Condenser lens 110, stop mask 111, stop apertures 112 and 113, re-imaging lenses 114 and 115, and image sensor 116 focus on a re-imaging type pupil division type phase difference detection method in which focus detection is performed at one position. Configure the detector. FIG. 28 schematically illustrates a focus detection device on the optical axis 191 and a focus detection device outside the optical axis. By combining a plurality of focus detection devices, it is possible to realize a focus detection dedicated sensor that performs focus detection by the re-imaging type pupil division type phase difference detection at the five focus detection positions G1 to G5 shown in FIG. .

  The focus detection apparatus including the condenser lens 110 is disposed between the condenser lens 110 disposed in the vicinity of the planned imaging surface of the interchangeable lens, the image sensor 116 disposed behind the condenser lens 110, and between the condenser lens 110 and the image sensor 116. A pair of re-imaging lenses 114 and 115 for re-imaging the primary image formed in the vicinity of the planned imaging surface on the image sensor 116, and the vicinity of the pair of re-imaging lenses (front surface in FIG. 28). The aperture mask 111 has a pair of aperture openings 112 and 113. The image sensor 116 is a line sensor in which a plurality of photoelectric conversion units are densely arranged along a straight line, and the arrangement direction of the photoelectric conversion units matches the dividing direction of the pair of distance measuring pupils, that is, the arrangement direction of the aperture openings. .

  Information corresponding to the intensity distribution of the pair of images re-imaged on the image sensor 116 is output from the image sensor 116, and image shift detection calculation processing (correlation processing, phase difference detection processing) is performed on this information. Thus, an image shift amount of the pair of images is detected by a so-called pupil division type phase difference detection method (re-imaging method). By multiplying the image shift amount by a predetermined conversion coefficient, the deviation (defocus amount) of the current image plane with respect to the planned image plane is calculated. In the case of the image shift amount detection calculation processing in this case, the information corresponding to the intensity distribution of the pair of images re-imaged on the image sensor 116 is logarithmically converted to obtain a pair of logarithmic data strings. The integrated correlation value C can be obtained. That is, the correlation calculation device according to the present invention can be applied to a re-imaging pupil division type focus detection device.

  The image sensor 116 is projected on the planned imaging plane by the re-imaging lenses 114 and 115, and the detection accuracy of the defocus amount (image deviation amount) is the detection pitch of the image deviation amount, particularly in the case of the re-imaging method. Is determined by the arrangement pitch of the photoelectric conversion units projected on the planned imaging plane.

  The condenser lens 110 projects the aperture openings 112 and 113 of the aperture mask 111 as areas 192 and 193 on the exit pupil 190. These regions 192 and 193 are called distance measurement pupils. That is, a pair of images re-imaged on the image sensor 116 is formed by the focus detection light beams 132 and 133 passing through the pair of distance measuring pupils 192 and 193 on the exit pupil 190, respectively.

(Modification 10) In addition, the present invention is not limited to focus detection using a method in which the light beam passing through the photographing optical system is divided into pupils, and can be applied to distance measurement using an external light triangulation method. Such effects can be achieved. With reference to FIG. 29, a description will be given of an external light triangulation distance measuring device to which the correlation calculation device according to this modification is applied. A first unit composed of a lens 320 and an image sensor 326 disposed on the imaging surface thereof, and a second unit composed of the lens 330 and an image sensor 336 disposed on the imaging surface thereof are separated from each other by a base line length. Be placed. A pair of units composed of the first unit and the second unit constitute a distance measuring device 347. An image of the distance measuring object 350 is formed on the image sensors 326 and 336 by the lenses 320 and 330, respectively.

  The positional relationship between the two images formed on the image sensors 326 and 336 changes according to the distance from the distance measuring device 347 to the distance measuring object 350. Therefore, the relative positional relationship between the two images formed on the image sensors 326 and 336 is detected by performing an image shift amount detection calculation process on the two sets of signal data output from the image sensors 326 and 336. Then, the distance to the distance measuring object 350 can be measured based on the detected relative positional relationship. In the image shift amount detection calculation process in this case, the two sets of signal data output from the image sensors 326 and 336 are logarithmically converted to obtain an integrated correlation value C based on two sets of logarithmic data strings obtained. Can be requested. That is, the correlation calculation device according to the present invention can also be applied to a distance measuring device of the external light triangulation method.

(Modification 11) An imaging device having a focus detection device including a correlation calculation device according to the present invention is not limited to a digital still camera composed of an interchangeable lens and a camera body. It can also be applied to video cameras. The correlation calculation device according to the present invention can also be applied to a small camera module, a surveillance camera, or the like built in a mobile phone. Furthermore, the correlation calculation device according to the present invention can be applied to a focus detection device, a distance measuring device, or a stereo distance measuring device other than the camera.

(Modification 12) The correlation calculation apparatus according to the present invention can also be applied to an apparatus that detects the correlation between the signals of the image sensors with different times and detects the movement of the subject image and camera shake. The correlation calculation device according to the present invention can also be applied to pattern matching between an image signal of an image sensor and a specific image signal. Furthermore, the correlation calculation device according to the present invention is not limited to a device that detects the correlation of image signal data, but can also be applied to any correlation calculation device that detects the correlation of data related to sound and generally the correlation of two signals. And the effects described above can be achieved.

10, 50, 60 micro lens,
11, 12, 13, 16, 17, 52, 53, 62, 63 photoelectric conversion unit,
20 Basic image, 21, 41 Face image, 22, 32 Reference pixel position,
23 correlation pixel position, 30 comparison target image, 33, 43 partial image,
72, 73, 82, 83 Focus detection luminous flux,
90 Exit pupil, 91 Optical axis, 92, 93 Distance pupil, 94, 95 area,
110, 120 condenser lens, 111, 121 aperture mask,
112, 113, 122, 123 Aperture aperture,
114, 115, 124, 125 Re-imaging lens,
116, 126 Image sensor,
132, 133, 142, 143 Focus detection light flux,
190 Exit pupil, 191 Optical axis, 192, 193 Distance pupil,
201 digital still camera, 202 interchangeable lens, 203 camera body,
204 Mount part, 205, 206, 207 Lens, 208 Aperture,
209 lens drive control device, 211 imaging device, 212 camera drive control device,
213 Memory card, 214 LCD driver, 215 LCD,
216 eyepieces, 217 electrical contacts,
310 imaging pixels, 311, 313, 314 focus detection pixels,
320, 330 lens, 326, 336 image sensor,
347 Distance measuring device, 350 Distance measuring object,
400, 401, 402, 403 image signal,
1110, 1120, 1130, 1140, 1150 region,
1410, 1420, 1430, 1440, 1450, 1460 region

Claims (15)

A plurality of first signal data forming a local partial array in a first signal data array including a plurality of first signal data, and a local partial array in a second signal data array including a plurality of second signal data. A plurality of second signal data having a local partial arrangement corresponding to the plurality of first signal data formed to calculate a first local correlation value, and the plurality of first signal data and the plurality of first signal data A first integrated correlation representing a degree of correlation between the first signal data array and the second signal data array by integrating the first local correlation values calculated each time a plurality of second signal data is changed. First correlation calculating means for calculating a value;
A third signal data array comprising a plurality of third signal data and a fourth signal data array comprising a plurality of fourth signal data by logarithmically converting the plurality of first signal data and the plurality of second signal data, respectively. Logarithmic conversion means for generating
A plurality of third signal data forming a local partial array in the third signal data array and a plurality of third signal data forming a local partial array in the fourth signal data array The second local correlation value is calculated by performing addition with a plurality of fourth signal data forming a partial array, and the calculation is performed each time the plurality of third signal data and the plurality of fourth signal data are changed. A second correlation calculating means for calculating a second integrated correlation value representing a degree of correlation between the third signal data array and the fourth signal data array by integrating the second local correlation values; A characteristic correlation calculation device. The correlation calculation device according to claim 1,
When the predetermined condition is satisfied, the second correlation calculating means calculates the second integrated correlation value,
When the predetermined condition is not satisfied, the first correlation calculation means calculates the first integrated correlation value. The correlation calculation device according to claim 2,
Determination means for determining whether or not the predetermined condition is satisfied;
According to the determination result by the determination means, select one of the calculation of the first integrated correlation value by the first correlation calculating means and the calculation of the second integrated correlation value by the second correlation calculating means is selected. A selection means;
The correlation between the first signal data array and the second signal data array based on one of the first integrated correlation value and the second integrated correlation value calculated according to the selection by the selection unit A correlation calculation device further comprising correlation degree detection means for detecting the degree. In the correlation calculation device according to claim 2 or 3,
The predetermined condition is that the average value of the plurality of first signal data and the average value of the plurality of second signal data are both equal to or greater than a predetermined value. In the first signal data array composed of a plurality of first signal data obtained by sampling the spatial intensity distribution of the first image at a plurality of spatial coordinate points arranged at predetermined intervals, the first of the plurality of spatial coordinate points. The plurality of spatial coordinates in which a plurality of first signal data obtained at a plurality of spatial coordinate points having a first positional relationship in the vicinity of one spatial coordinate point and the spatial intensity distribution of the second image are arranged at the predetermined interval. A second spatial coordinate point having a second positional relationship with respect to the first spatial coordinate point among the plurality of spatial coordinate points in a second signal data array comprising a plurality of second signal data obtained by sampling at a point; The first product data and the second product data are calculated by performing multiplication with a plurality of second signal data obtained at a plurality of spatial coordinate points in the first positional relationship in the vicinity of 1 product data and the above A plurality of first signal data in the vicinity of the first spatial coordinate point is obtained by performing subtraction with the product data to calculate first difference data, performing non-linear transformation on the first difference data and performing the same encoding. And a plurality of second signal data in the vicinity of the second spatial coordinate point, a first local correlation value is calculated, and the first spatial coordinate point is moved over a predetermined spatial coordinate area to First correlation calculating means for calculating a first integrated correlation value in the second positional relationship between the first signal data array and the second signal data array by integrating one local correlation value;
A plurality of third signal data and a plurality of fourth signal data are calculated by logarithmically converting the plurality of first signal data and the plurality of second signal data, respectively, and a third signal composed of the plurality of third signal data Logarithmic conversion means for generating a fourth signal data array comprising a data array and the plurality of fourth signal data;
In the third signal data array, in the fourth signal data array, a plurality of third signal data obtained at a plurality of spatial coordinate points in the first positional relationship in the vicinity of the first spatial coordinate point, Addition to a plurality of fourth signal data obtained at a plurality of spatial coordinate points having the first positional relationship in the vicinity of the second spatial coordinate point having the second positional relationship with respect to the first spatial coordinate point To calculate first sum data and second sum data, further subtract the first sum data from the second sum data to calculate second difference data, and calculate the second difference data By performing non-linear transformation and performing the same encoding, a plurality of third signal data in the vicinity of the first spatial coordinate point and a plurality of fourth signal data in the vicinity of the second spatial coordinate point are obtained. Two local correlation values are calculated, and the first space In the second positional relationship between the third signal data array and the fourth signal data array, the standard point is moved over the predetermined spatial coordinate area and the second local correlation value is integrated. Second correlation calculating means for calculating a second integrated correlation value;
One of calculation of the first integrated correlation value by the first correlation calculating means and calculation of the second integrated correlation value by the second correlation calculating means is selected according to whether or not a predetermined condition is satisfied. A selection means;
When the calculation of the first integrated correlation value by the first correlation calculation unit is selected by the selection unit, the second positional relationship corresponding to the second positional relationship every time the second positional relationship is changed over a predetermined range. A spatial relative displacement between the first signal data array and the second signal data array based on the second positional relationship when the first integrated correlation value is calculated and the first integrated correlation value shows an extreme value. When the calculation of the second integrated correlation value by the second correlation calculation unit is selected by the selection unit, the second positional relationship is changed every time the second positional relationship is changed over the predetermined range. Calculating the second integrated correlation value corresponding to the positional relationship, and based on the second positional relationship when the second integrated correlation value indicates an extreme value, the first signal data array and the second signal data array; Relative variation to calculate the spatial relative displacement of Correlation calculation apparatus characterized by comprising a quantity detecting means. The correlation calculation device according to claim 5,
The logarithmic conversion means converts signal data indicating a data value equal to or less than a predetermined value out of the plurality of first signal data and the plurality of second signal data into invalid data, and the plurality of third signal data and the plurality of third signal data and the plurality of second signal data When the invalid data is included in a plurality of fourth signal data, invalidate the second local correlation value corresponding to the invalid data,
The second correlation calculation means calculates the second integrated correlation value by integrating the second local correlation values that have not been invalidated. The correlation calculation device according to claim 6,
Data value distribution detecting means for detecting data value distributions of the plurality of first signal data and the plurality of second signal data;
The logarithmic conversion means determines the predetermined value according to the data value distribution detected by the data value distribution detection means. The correlation calculation device according to claim 6,
The logarithmic conversion means determines the predetermined value according to a noise level of the plurality of first signal data and the plurality of second signal data. The correlation calculation device according to claim 6,
A logarithmic lookup table in which a correspondence relationship between an input data value and an output data value obtained by logarithmically converting the input data value is defined;
The logarithmic conversion means refers to the logarithmic look-up table with the data value larger than the predetermined value as the input data value among the plurality of first signal data and the plurality of second signal data. The correlation calculation device, wherein the plurality of third signal data and the plurality of fourth signal data are calculated by acquiring data values. The correlation calculation device according to claim 9,
Data value distribution detecting means for detecting data value distributions of the plurality of first signal data and the plurality of second signal data;
The logarithmic lookup table includes a plurality of logarithmic lookup tables having different correspondence relationships;
The logarithmic conversion means acquires the output data value by referring to one logarithmic lookup table of the plurality of logarithmic lookup tables according to the data value distribution detected by the data value distribution detection means. Thus, the plurality of third signal data and the plurality of fourth signal data are calculated. The correlation calculation device according to claim 6,
The second correlation calculation means normalizes the second cumulative correlation value according to the number of the second local correlation values that are not invalidated. The correlation calculation device according to claim 6,
Subtraction of the plurality of third signal data obtained at the plurality of spatial coordinate points in the first positional relationship, and the plurality of fourth signals obtained at the plurality of spatial coordinate points in the first positional relationship. By subtracting the data by moving the first spatial coordinate point over the predetermined spatial coordinate region, a plurality of first intermediate processing data and a plurality of second second data corresponding to the plurality of spatial coordinate points are performed. Intermediate processing operation means for calculating intermediate processing data and generating a first intermediate processing data array composed of the plurality of first intermediate processing data and a second intermediate processing data array composed of the plurality of second intermediate processing data; Prepared,
The intermediate processing calculation means includes the invalid data among the plurality of third signal data and the plurality of fourth signal data used for calculating the plurality of first intermediate processing data and the plurality of second intermediate processing data. Is included, the intermediate processing data corresponding to the invalid data in the plurality of first intermediate processing data and the plurality of second intermediate processing data is invalidated,
The second correlation calculation means sets the second difference data calculated by subtracting the first sum data and the second sum data to the first positional relationship in the vicinity of the first spatial coordinate point. Each of the plurality of first intermediate processing data obtained at the plurality of spatial coordinate points and the first spatial data obtained at the plurality of spatial coordinate points in the first positional relationship in the vicinity of the second spatial coordinate point. When the second local correlation value is calculated by performing subtraction with each of the two intermediate processing data and performing the same encoding by nonlinearly converting the second difference data, the intermediate processing calculating means A correlation calculation device that invalidates the second local correlation value according to the intermediate processing data invalidated by. A correlation calculation device according to claim 6;
A plurality of focus detection pixels each having a pair of light fluxes that pass through different regions of the exit pupil of the optical system and form a pair of images on the planned focal plane of the optical system, respectively, arranged according to a spatial arrangement An image sensor that generates a pair of image data arrays corresponding to the pair of images as the first signal data array and the second signal data array when the photoelectric conversion unit receives light;
The spatial relative displacement amount calculated by the correlation calculation device relative displacement amount calculating means is acquired as an image displacement amount of the pair of images, and the predetermined focal plane is multiplied by a predetermined conversion coefficient. And a defocus amount calculating means for calculating a defocus amount between the optical system and the imaging plane of the optical system. The focus detection apparatus according to claim 13.
When the first signal data array and the second signal data array are generated by the image sensor, the selection unit calculates the first integrated correlation value and the second correlation operation by the selection unit. The logarithmic conversion unit generates the third signal data array and the fourth signal data array in advance before any one of the calculation of the second integrated correlation value by the unit is selected. Focus detection device. A plurality of first signal data forming a local partial array in a first signal data array including a plurality of first signal data, and a local partial array in a second signal data array including a plurality of second signal data. Multiplying a plurality of second signal data forming a local partial array corresponding to the plurality of first signal data forming the plurality of first signal data forming the local partial array, A first local correlation value between a plurality of second signal data forming a local partial array is calculated, and the local partial array in the first signal data array and the second signal data array is calculated. By integrating the first local correlation value calculated each time the plurality of first signal data formed and the plurality of second signal data forming the local partial array are changed, A first correlation calculating means for calculating a first cumulative correlation value representing a correlation between the serial second signal data sequences,
A third signal data array comprising a plurality of third signal data and a fourth signal data array comprising a plurality of fourth signal data by logarithmically converting the plurality of first signal data and the plurality of second signal data, respectively. Logarithmic conversion means for generating
A plurality of third signal data forming a local partial array in the third signal data array and a plurality of third signal data forming a local partial array in the fourth signal data array By performing addition between a plurality of fourth signal data forming a partial array, a plurality of third signal data forming the local partial array and a plurality of fourth signal data forming the local partial array; A plurality of third signal data forming the local partial array and the local partial array in the third signal data array and the fourth signal data array The second local correlation value calculated each time the plurality of fourth signal data forming the first is changed is integrated to obtain a second degree of correlation between the third signal data array and the fourth signal data array. Correlation calculation apparatus characterized by comprising a second correlation calculating means for calculating a calculated correlation value.

Images